Generating xCal calendar in python

{ Repost from my personal blog @ https://blog.codezero.xyz/generate-xcal-calendar-in-python }

“xCal”, is an XML format for iCalendar data.

The iCalendar data format (RFC5545) is a widely deployed interchange format for calendaring and scheduling data.

A Sample xCal document

<?xml version="1.0" encoding="utf-8"?>  
<iCalendar xmlns:xCal="urn:ietf:params:xml:ns:xcal">  
    <vcalendar>
        <version>2.0</version>
        <prodid>-//Pentabarf//Schedule 1.0//EN</prodid>
        <x-wr-caldesc>FOSDEM 2016</x-wr-caldesc>
        <x-wr-calname>Schedule for events at FOSDEM 2016</x-wr-calname>
        <vevent>
            <method>PUBLISH</method>
            <uid>123e4567-e89b-12d3-a456-426655440000</uid>
            <dtstart>20160131T090000</dtstart>
            <dtend>20160131T091000</dtend>
            <duration>00:10:00:00</duration>
            <summary>Introduction to the SDR Track- Speakers, Topics, Algorithm</summary>
            <description>&lt;p&gt;The opening talk for the SDR devroom at FOSDEM 2016.&lt;/p&gt;</description>
            <class>PUBLIC</class>
            <status>CONFIRMED</status>
            <categories>Software Defined Radio</categories>
            <url>https:/fosdem.org/2016/schedule/event/sdrintro/</url>
            <location>AW1.125</location>
            <attendee>Martin Braun</attendee>
        </vevent>
    </vcalendar>
</iCalendar>

Each event/session will be in a seperate vevent block. Each speaker/attendee of an event/session will be in an attendee block inside a vevent block.

Some important elements are:

  1. version – Has the version of the iCalendar data
  2. prodid – Contains the name of the application/generator that generated this document
  3. x-wr-caldesc – A descriptive name for this calendar
  4. x-wr-calname – A description of the calendar

The structure and keywords used in xCal are the same as those used in the iCal format. To generate the XML document, we’ll be using python’s ElementTreeXML API that is part of the Python standard library.

We’ll be using two main classes of the ElementTree API:

  1. Element – used to create a standard node. (Used for the root node)
  2. SubElement – used to create a sub element and attache the new node to a parent

Let’s start with the root iCalendar node and set the required attributes.

from xml.etree.ElementTree import Element, SubElement, tostring

i_calendar_node = Element('iCalendar')  
i_calendar_node.set('xmlns:xCal', 'urn:ietf:params:xml:ns:xcal')

Now, to add the vcalendar node to the iCalendar node.

v_calendar_node = SubElement(i_calendar_node, 'vcalendar')

Let’s add the other aspects of the calendar to the vcalendar node as separate sub nodes.

version_node = SubElement(v_calendar_node, 'version')  
version_node.text = '2.0'

prod_id_node = SubElement(v_calendar_node, 'prodid')  
prod_id_node.text = '-//fossasia//open-event//EN'

cal_desc_node = SubElement(v_calendar_node, 'x-wr-caldesc')  
cal_desc_node.text = "Calendar"

cal_name_node = SubElement(v_calendar_node, 'x-wr-calname')  
cal_name_node.text = "Schedule for sessions"

Now, we have added information about our calendar. Now to add the actual events to the calendar. Each event would be a vevent node, a child of vcalendar node. We can loop through all our available event/sessions and add them to the calendar.

for session in sessions:  
    v_event_node = SubElement(v_calendar_node, 'vevent')

    uid_node = SubElement(v_event_node, 'uid')
    uid_node.text = str(session.id)

    dtstart_node = SubElement(v_event_node, 'dtstart')
    dtstart_node.text = session.start_time.isoformat()

    dtend_node = SubElement(v_event_node, 'dtend')
    dtend_node.text = tz.localize(session.end_time).isoformat()

    duration_node = SubElement(v_event_node, 'duration')
    duration_node.text =  "00:30"

    summary_node = SubElement(v_event_node, 'summary')
    summary_node.text = session.title

    description_node = SubElement(v_event_node, 'description')
    description_node.text = session.short_abstract

    class_node = SubElement(v_event_node, 'class')
    class_node.text = 'PUBLIC'

    status_node = SubElement(v_event_node, 'status')
    status_node.text = 'CONFIRMED'

    categories_node = SubElement(v_event_node, 'categories')
    categories_node.text = session.session_type.name

    url_node = SubElement(v_event_node, 'url')
    url_node.text = "https://some.conf/event/" + str(session.id)

    location_node = SubElement(v_event_node, 'location')
    location_node.text = session.microlocation.name

    for speaker in session.speakers:
        attendee_node = SubElement(v_event_node, 'attendee')
        attendee_node.text = speaker.name

Please note that all the timings in the XML Document must comply with ISO 8601 and must have the date+time+timezone. Example: 2007-04-05T12:30-02:00.

We’re still not done yet. We now have the XML document as an Element object. But we’ll be needing it as a string to either store it somewhere or display it.

The document can be converted to a string by using the ElementTree API’s tostring helper method and passing the root node.

xml_as_string = tostring(i_calendar_node)

And that’s it. You now have a proper XML document representing your events.

KISS Datatable

Recenlty I’ve faced a problem with sorting columns in Datatable.

What is Datatable?

Datatable is a plug-in for Jquery library. It provides a lot of features like pagination, quick search or multi-column ordering. Besides, you can develop easily Bootstrap or Foundation ui css styles. There are also more other option but It doesn’t make sense to list it here, because you can visit their site and you can read clearly documentation. On Datatable website you can see a lot of examples. First of them shows how to improve your ordinary table to awesome and rich of features table. One function changes everything, It’s fantastic!  

$('#myTable').DataTable();

Returning to my problem which I’ve faced, as I told it was problem related to sorting column in table.

I know sorting is a trivial thing. I hope that everyone knows it 🙂 Sorting by a date is also implemented in a datatable library. So everything is clear when we don’t change date format to human readable format. I mean something like this ‘3 hours ago’, ‘1 year ago’.

When Open Event team tested how datatable manages ordering columns in that format it didn’t work. It’s quite hard to sort by that format, So I’ve invented an idea. Surely you are wondering what i’ve invented. I’ve postponed my minds about sort by this values. It can direct to overwork. When I thought about it, weird ideas came to my mind, a lots of conditions, If statements… Therefore I’ve resigned from this. I’ve used KISS principle. KISS means ‘keep it simple stupid’. I like it!

Therefore that sorting is implemented on frontend side. I’ve decided not to display human readable date format at the beginning. I find  all dates which have format “YYYY-MM-DD HH:mm” then I replace that format to human readable format. So it’s very quick and comfortable, and doesn’t require a lot conditions to write. Of course I’ve tried to implement it in Datatable library. I suppose that it would  require more effort than it’s now.

Below You can see great function which changes a date in frontend side but does not change data in a datatable. So sorting process takes place in a datatable using format  “YYYY-MM-DD HH:mm” but user see human readable format. Isn’t it awesome?!

function change_column_time_to_humanize_format(datatable_name, column_id) {
  $(datatable_name).each(function( key, value ) {
    $(value).children().each(function( key1, value2 ) {
       if(key1 === column_id ){
          var current_value = $(value2).text().slice(0, 16);
          var changed_value = moment(current_value, "YYYY-MM-DD hh:mm").fromNow()
          var isValid = moment(current_value, "YYYY-MM-DD HH:mm", true).isValid()
          if (changed_value !== current_value && isValid === true){
              $(value2).text(changed_value)
          }
      }
  });
});

 

Accepting Stripe payments on behalf of a third-party

{ Repost from my personal blog @ https://blog.codezero.xyz/accepting-stripe-payments-on-behalf-of-a-third-party }

In Open Event, we allow the organizer of each event to link their Stripe account, so that all ticket payments go directly into their account. To make it simpler for the organizer to setup the link, we have a Connect with stripe button on the event creation form.

Clicking on the button, the organizer is greeted with a signup flow similar to Login with Facebook or any other social login. Through this process, we’re able to securely and easily obtain the credentials required to accept payments on behalf of the organizer.

For this very purpose, stripe provides us with an OAuth interface called as Stripe Connect. Stripe Connect allows us to connect and interact with other stripe accounts through an API.

We’ll be using Python’s requests library for making all the HTTP Requests to the API.
You will be needing a stripe account for this.

Registering your platform
The OAuth Flow

The OAuth flow is similar to most platforms.

  • The user is redirected to an authorization page where they login to their stripe account and authorize your app to access their account
  • The user is then redirected back to a callback URL with an Authorization code
  • The server makes a request to the Token API with the Authorization code to retrieve the access_token, refresh_token and other credentials.

Implementing the flow

Redirect the user to the Authorization URL.
https://connect.stripe.com/oauth/authorize?response_type=code&client_id=ca_8x1ebxrl8eOwOSqRTVLUJkWtcfP92YJE&scope=read_write&redirect_uri=http://localhost/stripe/callback  

The authorization url accepts the following parameters.

  1. client_id – The client ID acquired when registering your platform.required.
  2. response_type – Response type. The value is always code. required.
  3. redirect_uri – The URL to redirect the customer to after authorization.
  4. scope – Can be read_write or read_only. The default is read_only. For analytics purposes, read_only is appropriate; To perform charges on behalf of the connected user, We will need to request read_write scope instead.

The user will be taken to stripe authorization page, where the user can login to an existing account or create a new account without breaking the flow. Once the user has authorized the application, he/she is taken back to the Callback URL with the result.

Requesting the access token with the authorization code

The user is redirected back to the callback URL.

If the authorization failed, the callback URL has a query string parameter error with the error name and a parameter error_description with the description of the error.

If the authorization was a success, the callback URL has the authorization code in the code query string parameter.

import requests

data = {  
    'client_secret': 'CLIENT_SECRET',
    'code': 'AUTHORIZATION_CODE',
    'grant_type': 'authorization_code'
}

response = requests.post('https://connect.stripe.com/oauth/token', data=data)

The client_secret is also obtained when registering your platform. The codeparameter is the authorization code.

On making this request, a json response will be returned.

If the request was a success, the following response will be obtained.

{
  "token_type": "bearer",
  "stripe_publishable_key": PUBLISHABLE_KEY,
  "scope": "read_write",
  "livemode": false,
  "stripe_user_id": USER_ID,
  "refresh_token": REFRESH_TOKEN,
  "access_token": ACCESS_TOKEN
}

If the request failed for some reason, an error will be returned.

{
  "error": "invalid_grant",
  "error_description": "Authorization code does not exist: AUTHORIZATION_CODE"
}

The access_token token obtained can be used as the secret key to accept payments like discussed in Integrating Stripe in the Flask web framework.

Responsive Image Overlay

Image overlay is a very common concept in front-end development. It is easy to implement but difficult when we deal it with different screen sizes, where we need to cover the image with the overlay each time the screen size is changed. I have gone through various blog posts when I need to implement the same for Open-event webapp and researched a solution that works for all screen sizes without any media query.

1234

How to add an overlay to an image ?

If we need four images in a single row nearly 300*300px.  The code below shows the markup.

image-holder : The parent class to take the image and overlay inside it.

background-image: This class takes image source.

responsive-overlay: This is the key point to make it responsive. Responsive-overlay contains a class hover-state to add overlay absolutely and a class social-links.

social-links: It adds content to hover-state.

 

<div class="image-holder">
  <img class="background-image" alt="" src="">
   <div class="responsive-overlay">
     <div class="hover-state text-center preserve3d">
       <div class="social-links vertical-align">

       </div>
     </div>
   </div>
 </div>

The styling is written with SASS in .scss file as shown below.

//overlayimage and backgroundshade can be set in config.scss

 .image-holder {
   position: relative;
   overflow: hidden;
   margin-bottom: 12px;

   .background-image {
     height: 300px;
     width: 300px;
     display: block;
     margin: 0 auto;
     background-color: $background-shade;
    }
 
   .responsive-overlay {
     @include responsiveoverlay;

    .preserve3d {
       height: 300px;
      }

    .hover-state {
     @include hoverstate;
     height: 300px;
     width: 300px;
    }

  @mixin responsiveoverlay {
     height: 100%;
     position: absolute;
     top: 0;
     width: 100%;
}

   @mixin hoverstate {
     background: $overlayimage;
     display: block;
     height: 300px;
     left: 0;
     margin: 0 auto;
     opacity: 0;
     position: relative;
     top: 0;
     -moz-transition: all 0.3s ease-out;
     -webkit-transition: all 0.3s ease-out;
     transition: all 0.3s ease-out;
     width: 300px;
     z-index: 2;
   }

This code will work for responsiveness as well. The main catch here is the responsive-overlay class which is made 100% in width but set to position absolute. The images which are 300 * 300 px in size will take an overlay of the same size because of hover-state class. Instead, if we adjust sizes of images in small screens the above code will adjust overlay on the image automatically.

Like, on tablets we can have an overlay like this.

345

And on mobile screen output is like that :

23

Conclusion

Responsiveness is easy if we follow correct concepts. Here, the concepts of absolute and relative positioning in CSS have done the magic. Now we can play by adding different contents and effect on hover following the same basics.

Programmer principles

As programmers we develop our programming skills and learn something every single day. We write code and solve many troubles. But is our aim to simply write code? I am sure it is not. I think writing code just for doing it is not interesting, and it’s definitely not Open Event team’s objective. Personally, I like reading code like a poem. We should always try to eliminate bad practises and ugly code. There are a few principles how to do it. Let me share them with you now.

SOLID principle

SOLID  is a mnemonic acronym introduced by Michael Feathers, and it simply means five basic principles of object oriented programming. These principles, when applied together, make it more likely that a programmer will create a system that is easy to maintain and extend over time. They are guidelines that can be applied while working on software to remove code smells by causing the programmer to refactor the software’s source code.  It is also a part of an overall strategy of agile. So, here they are:

S – Single responsibility principle

This principle means that there should never be more than one reason for a class to change.

In other words, a class should have only one potential change in a software’s specification. You should not add everything into your class. The best practise here is to check if the logic you are introducing should be in this class or not. Responsibility is the heart of this principle, so to rephrase there should never be more than one responsibility per class. Use layers for a help. And try to divide big classes into smaller ones.

O – Open/closed principle

Software entities like classes, module and functions should be open for extension, but closed for modification.

All of them should be private by default.

To make an object behaving differently without modifying it use abstractions, or place behavior(responsibility) in derivative classes. If properties of the abstracted class need to be compared or organized together, another abstraction should handle this. This is the basis of the “keep all object variables private” argument.

L – Liskov substitution principle

Functions that use pointers or references to base classes have to be able to use objects of derived classes without knowing/alerting the correctness of a program

A great example you can find here. If you are using a method defined at a base class upon an abstracted class, the function must be implemented properly on the subtype class. A great example provided here http://williamdurand.fr/2013/07/30/from-stupid-to-solid-code/  you can find below.

“ A rectangle is a plane figure with four right angles. It has a width, and a height. Now, take a look at the following pseudo-code:

rect = new Rectangle();

rect.width  = 10;
rect.height = 20;

assert 10 == rect.width
assert 20 == rect.height

We simply set a width and a height on a Rectangle instance, and then we assert that both properties are correct. So far, so good.

Now we can improve our definition by saying that a rectangle with four sides of equal length is called a square. A square is a rectangle so we can create aSquare class that extends the Rectangle one, and replace the first line above by the one below:

rect = new Square();

According to the definition of a square, its width is equal to its height. Can you spot the problem? The first assertion will fail because we had to change the behavior of the setters in the Square class to fit the definition “

I – Interface segregation principle

Many client-specific interfaces are better than one general-purpose interface.

Implementing methods that you don’t use is not recommended in this way. The idea here is to keep your components focused and try to minimize the dependencies between them. Enforcing that principle gives you low coupling, and high cohesion.

D – Dependency inversion principle

This means that “one should depends upon abstractions, do not depend upon concretions”

Interfaces should depend on other interfaces. Don’t add concrete classes to method signatures of an interface. However, use interfaces in your class methods.

So, we can also say that rather than working with classes that are tight coupled, use interfaces. This reduces dependency on implementation specifics and makes code more reusable.

Why SOLID?

I hope all of you understand the importance of using SOLID principles in your everyday code practise. Finally, let me underline again the main arguments why you should starting following them now. The most important thing is that thanks to them you can create easy to maintain software, then you can reuse your code, and finally it helps you to test easier. Do you need anymore to be  persuaded  to do it? I think it’s that’s crucial advantages and they are enough.

Source:

https://pl.wikipedia.org/wiki/SOLID_(programowanie_obiektowe)

https://scotch.io/bar-talk/s-o-l-i-d-the-first-five-principles-of-object-oriented-design

http://williamdurand.fr/2013/07/30/from-stupid-to-solid-code/

http://www.codeproject.com/Articles/60845/The-S-O-L-I-D-Object-Oriented-Programming-OOP-Prin

Mark Notifications Read on Click

Screenshot from 2016-08-01 07:31:22

Notification has become a really important way of informing users about the various activities related to them in web apps. There are different types of notification such as web app notification, email notification, desktop notification, push notification, etc. We are going to primarily talk about web app notification and mainly about how to mark them as read.

Create Notification

Creating a notification is plain and simple. You have a json or an object which stores the notification message corresponding to a particular activity. Whenever that activity occurs in the backend, you call the send notification module, which adds the information to the database and shows it in the notification page. As simple as that.

Screenshot from 2016-08-01 07:48:08

Marking Notification as Read

The main functioning of this is plain and simple as well. You have a URL, which on getting a request from the user, marks the notification as read in the database. That’s it.

Screenshot from 2016-08-01 07:48:17

We know how to do this using a button or a link. But the question here is how to mark a notification as read on clicking any part of the notification?? The obvious answer is, well, put the entire notification inside an anchor tag and you are done, right? Well, it would work in many cases. But what if the design structure is such that this doesn’t work somehow. Somehow enclosing the notification inside a particular anchor tag doesn’t solve the purpose. What do we do then?

Identify Whether Inside a DIV

The main problem here actually is how to identify whether the click is inside the enclosing div or somewhere else. Once we solve this problem, we can send an ajax request to the mark read URL and our job is done.

Screenshot from 2016-08-01 07:52:58

So, to identify that a click is indeed inside a div, we use the event.target property of the event clicked. The target event property returns the element that triggered the event. So we check whether event.target has the “notification” class in our case. If it does not have the “notification” class we check in all it’s parent nodes. We get the parent nodes using the “parent()” function and check whether any of that has notification. If either of the 2 occurs, we consider that the click is inside the div. And thus mark the notification as read.

Screenshot from 2016-08-01 07:51:09

So, once this is done, we mark the notification as read in the backend and our job is done…

Autocomplete Address Form using Google Map API

Google map is one of the most widely used API of Google as most of the websites use Google map for showing address location. For a static address it’s pretty simple. All you need to do is mention the address and the map will show the nearest location. Problem arrives when the address is dynamically putted by the user. Suppose for an event in event organizer server, one enters the location. The main component used while taking input location is Google Autocomplete. But we went a step further and parsed the entire address based on city, state, country, etc. and allowed user to input the details as well which gave them the nearest location marked in Map if autocomplete couldn’t find the address.

Autocomplete Location

Screenshot from 2016-07-27 06:52:37

As we can see, in the above input box we get suggestions by Google Map on entering first few letters of our address. To this, we need the API https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&libraries=places&callback=initMap. You can find an example code of how to do this here.

After this is done, what we wanted is not to just include this address, but to provide a form to fill up the entire address in case some parts were missing on this address. The function that the autocomplete listens to is “place_changed” . So once we click on one of the options, this event is triggered. Once the event is triggered, we use the autocomplete.getPlace() to get the complete address json. The json looks something like this:

Screenshot from 2016-07-27 07:04:49

Now what we do is we create a form with input fields having the id same as the ones we require(e.g., country, administrative_area_level_1, locality, etc.). After that we select the long_name or the short_name from this json and put the value in the corresponding input field with the ids. The code for the process after getting the json is elaborated here.

Editing Address Form

After doing this it looks something like this:
Screenshot from 2016-07-27 07:12:13

However, now the important part is to show the map according to this fields. Also, every time we update a field, the map should be updated. For this we use a hack. Instead of removing the initial location field completely, we hide the field but keep the binding to autocomplete intact. As a result the map is shown when we select a particular address.

Now when we update the fields in the address form, we append the value of this field to the value in the initial location field. Though the field is hidden but it is still bound to autocomplete. As a result as soon as we append something to the string contained in the field, the map gets updated. Also, the updated value gets stored to the DB. Thus, with every update in field, the pointer is moved to the nearest location formed by appending all the data from the form.

After saving the location data to DB, if we wish to edit it, we can get back the json by making the same request with the location value. And then we will get back the same address form and the map accordingly.

Finally it looks something like this:

Screenshot from 2016-07-27 07:19:56

Building interactive elements with HTML and javascript: Interact.js + resizing

{ Repost from my personal blog @ https://blog.codezero.xyz/building-interactive-elements-with-html-and-javascript-interact-js-resizing/ }

In a few of the past blog posts, we saw about implementing resizing with HTML and javascript. The functionality was pretty basic with simple resizing. In the last blog post we saw about interact.js.

interact.js is a lightweight, standalone JavaScript module for handling single-pointer and multi-touch drags and gestures with powerful features including inertia and snapping.

Getting started with Interact.js

You have multiple option to include the library in your project.

  • You can use bower to install (bower install interact) (or)
  • npm (npm install interact.js) (or)
  • You could directly include the library from a CDN (https://cdnjs.cloudflare.com/ajax/libs/interact.js/1.2.6/interact.min.js).
Implementing resizing

Let’s create a simple box using HTML. We’ll add a class called resizable to it so that we can reference it to initialize Interact.js

<div class="resizable">  
    Use right/bottom edge to resize
</div>

We need to create an interact instance. Once the instance is created, we have to call the resizable method on it to add resize support to the div.

interact('.resizable')
  .resizable({
    edges: { right: true, bottom: true }
  })
  .on('resizemove', function (event) {
    

  });

Inside the resizable method, we can pass configuration options. The edgesconfig key allows us to specify on which all edges, resizing should be allowed. Right now, we have allowed on the right and bottom edges. Similarly we can have resizing support in the top and left edges too.

The resizemove event is triggered by interact every time the user tries to resize the div. From the event, we can get the box that is being resized, (i.e) the target by accessing event.target.

The event object also provides us event.rect.width and event.rect.height which is the width and height of the div after resizing. We’ll not set this as the width of the div so that, the user is able to see the width change.

var target = event.target;
    // update the element's style
    target.style.width  = event.rect.width + 'px';
    target.style.height = event.rect.height + 'px';

We can also instruct Interact.js to preserve the aspect ratio of the box by adding an option preserveAspectRatio: true to the configuration object passed to resizable method during initialization.

JavaScript
interact('.resizable')
  .resizable({
    edges: { right: true, bottom: true }
  })
  .on('resizemove', function (event) {
    var target = event.target;

    // update the element's style
    target.style.width  = event.rect.width + 'px';
    target.style.height = event.rect.height + 'px';
  });

Resizing and drag-drop (with Interact.js) were used to create the Scheduler tool at Open Event. The tool allows event/track organizers to easily arrange the sessions into their respective rooms by drag-drop and also to easily change the timings of the events by resizing the event block. The entire source code of the scheduler can be viewed at app/static/js/admin/event/scheduler.js in the Open Event Organizer server’s GitHub repository.

Demo:
https://jsfiddle.net/xdfocdty/

Import/Export feature of Open Event – Challenges

We have developed a nice import/export feature as a part of our GSoC project Open Event. It allows user to export an event and then further import it back.

Event contains data like tracks, sessions, microlocations etc. When I was developing the basic part of this feature, it was a challenge on how to export and then further import the same data. I was in need of a format that completely stores data and is recognized by the current system. This is when I decided to use the APIs.

API documentation of Open Event project is at http://open-event.herokuapp.com/api/v2. We have a considerably rich API covering most aspects of the system. For the export, I adopted this very simple technique.

  1. Call the corresponding GET APIs (tracks, sessions etc) for a database model internally.
  2. Save the data in separate json files.
  3. Zip them all and done.

This was very simple and convenient. Now the real challenge came of importing the event from the data exported. As exported data was nothing but json, we could have created the event back by sending the data back as POST request. But this was not that easy because the data formats are not exactly the same for GET and POST requests.

Example –

Sessions GET –

{
	"speakers": [
		{
			"id": 1,
			"name": "Jay Sean"
		}
	],
	"track": {
		"id": 1,
		"name": "Warmups"
	}
}

Sessions POST –

{
	"speaker_ids": [1],
	"track_id": 1
}

So the exported data can only be imported when it has been converted to POST form. Luckily, the only change between POST and GET APIs was of the related attributes where dictionary in GET was replaced with just the ID in POST/PUT. So when importing I had to make it so such that the dicts are converted to their POST counterparts. For this, all that I had to do was to list all dict-type keys and extract the id key from them. I defined a global variable as the following listing all dict keys and then wrote a function to extract the ids and convert the keys.

RELATED_FIELDS = {
    'sessions': [
        ('track', 'track_id', 'tracks'),
        ('speakers', 'speaker_ids', 'speakers'),
    ]
}

Second challenge

Now I realized that there was even a tougher problem, and that was how to re-create the relations. In the above json, you must have realized that a session can be related to speaker(s) and track. These relations are managed using the IDs of the items. When an event is imported, the IDs are bound to change and so the old IDs will become outdated i.e. a track which was at ID 62 when exported can be at ID 92 when it is imported. This will cause the relationships to break. So to counter this problem, I did the following –

  1. Import items in a specific order, independent first
  2. Store a map of old IDs v/s new IDs.
  3. When dependent items are to be created, get new ID from the map and relate with it.

Let me explain the above –

The first step was to import/re-create the independent items first. Here independent items are tracks and speakers, and the dependent item is session. Now while creating the independent items, store their new IDs after create. Create a map of old ids v/s new ids and store it. This map will hold a clue to what became what after they were recreated from the json. Now the key final step is that when dependent items are to be created, find the indepedent related keys in their json using the above defined RELATED_FIELDS listing. Once they are found, extract their IDs and find the new ID corresponding to their old ID. Link the new ID with the dependent item and that would be all.

This post covers the main challenges I faced when developing the import/export feature and how I overcame them. I hope it will provide some help when you are dealing with similar problems.

 

{{ Repost from my personal blog http://aviaryan.in/blog/gsoc/open-event-import-export-algo.html }}

Unit Testing

There are many stories about unit testing. Developers sometimes say that they don’t write tests because they write a good quality code. Does it make sense, if no one is infallible?.

At studies only a  few teachers talk about unit testing, but they only show basic examples of unit testing. They require to write a few tests to finish final project, but nobody really  teaches us the importance of unit testing.

I have also always wondered what benefits can it bring. As time is a really important factor in our work it often happens that we simply resign of this part of process development to get “more time” rather than spend time on writing stupid tests. But now I know that it is a vicious circle.

Customers requierments does not help us. They put a high pressure to see visible results not a few statistics about coverage status. None of them cares about some strange numbers. So, as I mentioned above, we usually focuses on building new features and get riid of tests. It may seem to save time, but it doesn’t.

In reality tests save us a lot of time because we can identify and fix bugs very quickly. If a bug ocurrs because someone’s change we don’t have to spend long hours trying to figure out wgat is going out. That’s why we need tests.  

It is especially visible in huge open source projects. FOSSASIA organization has about 200 contributors. In OpenEvent project we have about 20 active developers, who generate many lines of code every single day. Many of them change over and over again as well as interfere  with each other.

Let me provide you with a simple example. In our team we have about 7 pull requests per day. As I mentioned above we want to make our code high quality and free of bugs, but without testing identifying if pull request causes a bug is very difficult task. But fortunately this boring job makes Travis CI for us. It is a great tool which uses our tests and runs them on every PR  to check if bugs occur. It helps us to quickly notice bugs and maintain our project very well.

What is unit testing?

Unit testing is a software development method in which the smallest testable parts of an application are tested

Why do we need writing unit tests?

Let me point all arguments why unit testing is really important while developing a project.

  • To prove that our code works properly

If developer adds another condition, test checks if method returns correct results. You simply don’t need to wonder if something is wrong with you code.

  • To reduce amount of bugs

It let you to know what inputs params’ function should get and what results should be returned. You simply don’t  write unused code

  • To save development time

Developers don’t waste time on checking every code’s change if his code works correctly

  • Unit tests help to understand software design
  • To provide quick feedback about method which you are testing
  • To help document a code

How to write unit test in Python

In my work I write use tests in Python. I am going to share my sample code  with you now

  • Import module unittest
  • Choose function to test
  • Write unit test

Example OpenEvent test in Python

class TestPagesUrls(OpenEventTestCase):

   def setUp(self):

       self.app = Setup.create_app()

   def test_if_urls_exist(self):

       """Test all urls via GET method"""

       with app.test_request_context():

           for rule in app.url_map.iter_rules():

               if excluded_paths(rule):

                   status_code = self.app.get(request.url[:-1] + str(rule).replace('//', '/'),        follow_redirects=True).status_code

                   self.assertTrue(status_code in [200, 302, 401])

 

I want to check if all views exist but it required a lot of time. That’s why I wonder I how to avoid writing similar tests. Finally, based  on our list of routes I am able to write test which checks code’s status  on every page.

If some of them response returns status_code different than 200, 302 or 401, test fails.This results means that somethings is wrong. Simple, isn’t it ?  Try to test it manually…. This one short test cover about 40 use cases…

This example shows an incredible value of unit tests! If developer makes a bug in response he receives an error that something is wrong with a view. Travis CI allows to reject all  wrong pull requests and merge only these which fulfill our quality requirements.   

Fixing  error is one part but finding a bug is even harder task. But an ability to detect bug on early stage of process development reduces cost of software.