open event

Read more about the article Implementing Notifications API in Open Event Frontend

Implementing Notifications API in Open Event Frontend

In Open Event Frontend, at the index page of the application, we have a notification dropdown in which a user gets the notifications regarding the events, sessions, etc. Thus, a user gets notified for the particular event or session he wants to receive notifications about. While dealing with an issue, we had to integrate the API with the frontend. We achieved it as follows:

First, we create a model of notifications so that we have basic structure ready. It goes as follows:

export default ModelBase.extend({
  title      : attr('string'),
  message    : attr('string'),
  isRead     : attr('boolean', { defaultValue: false }),
  receivedAt : attr('moment'),

  user: belongsTo('user')
});

Thus, we have fields like title, message, isRead, receivedAt which we will get from the server response as JSON which we will need to show on the page. To show the notifications to the user, first we need to query the notifications for a specific user using ember data. Since we are querying the notifications for a specific user when he is logged in, we are also having relationship between user and notification as shown in the above notification model. In user model we do:

notifications: hasMany('notification')

Now, we query the notifications in our application route i.e routes/application.js

model() {
    if (this.get('session.isAuthenticated')) {
      return new RSVP.Promise((resolve, reject) => {
        this.store.findRecord('user', this.get('authManager.currentUser.id'), { reload: true })
          .then(user => {
            user.get('notifications').then(resolve).catch(reject);
          })
          .catch(reject);
      });
    }
  }

The reason why we used a RSVP promise here was because the authManager couldn’t load the user befor the notifications were queried and returned. Thus, we query the notifications by using currentUser from authManager. Thus, in our template, we iterate over our notifications as follows:

    {{#each notifications as  notification }}
      <div class="item">
        <div class="header">
          {{notification.title}}
        </div>
        <div class="content weight-600">
          {{notification.description}}
        </div>
        <div class="left floated content">
          {{moment-from-now notification.createdAt}}
        </div>
      </div>
    {{/each}}

The notifications are thus shown to the user when he clicks the icon in the nav-bar. As a result, we get the following notifications in the dropdown:

Resources:

Ember data official guide

Blog on Ember data by Embedly.

Continue ReadingImplementing Notifications API in Open Event Frontend
Read more about the article Implementing Email Notifications in Open Event Frontend

Implementing Email Notifications in Open Event Frontend

In Open Event Frontend, we have a ‘settings/email-preferences’ route where we give the user an access to change the email-notifications that he wants to subscribe for a specific event. For now, we are having three notifications for an event which the user can toggle to on and off. To achieve this, we did the following things:

First, we create a model for email-notifications so as to have a skeleton of the data that we are going to receive from the JSON API.

export default ModelBase.extend({

  /**
   * Attributes
   */

  nextEvent           : attr('boolean'),
  newPaper            : attr('boolean'),
  sessionAcceptReject : attr('boolean'),
  sessionSchedule     : attr('boolean'),
  afterTicketPurchase : attr('boolean'),

  /**
   * Relationships
   */
  user  : belongsTo('user'),
  event : belongsTo('event')
});

Thus, above code shows the attributes which we are going to receive via our JSON API and we will render the data accordingly on the page. We have established the relationship of the email-notifications with user and event so that in future wherever needed, we can query the records from either side. The client side has checkboxes to show the data to the user. Following is the format of the checkboxes:

<div class="row">
        <div class="column eight wide">
          {{t 'New Paper is Submitted to your Event'}}
        </div>
        <div class="ui column eight wide right aligned">
          {{ui-checkbox class='toggle' checked=preference.newPaper onChange=(pipe-action (action (mut preference.newPaper)) (action 'savePreference' preference))}}
        </div>
      </div>
      <div class="row">
        <div class="column eight wide">
          {{t 'Change in Schedule of Sessions in your Event'}}
        </div>
        <div class="ui column eight wide right aligned">
          {{ui-checkbox class='toggle' checked=preference.sessionSchedule onChange=(pipe-action (action (mut preference.sessionSchedule)) (action 'savePreference' preference))}}
        </div>
      </div>

The states of the checkboxes are determined by the data that we receive from the API. For example, if for a record, the API returns:

nextEvent           : true,
newPaper            : false,
sessionAcceptReject : true,
sessionSchedule     : false,
afterTicketPurchase : false,

Then, the respective states will be shown by the checkbox and the user can toggle the states to change the email-preferences as they want.

Thus to get the data sent by the server to the client, we return it as a model and query it as:

model() {
    return this.get('authManager.currentUser').query('emailNotifications', { include: 'event' });
}

As we can see, as mentioned earlier, we kept the relationships so that we can query the email-notifications specific to the particular user or specific to particular event. Here, we are showing a user’s email-notifications and hence we queried it with the user relationship.
The authManager loads the currentUser and queries the email-notifications for a particular use. We also want the event details to show the email-preferences, hence we include the event model to be fetched in the query also.

We also let the user change the preferences of the email-notifications so that he can customise the notifications and keep the ones he wants to receive. We implement the updating of email-preferences API as follows:

Whenever a user toggles the checkbox, we are having an action as called ‘savePreference’, which handles the updation of the preferences.

 

{{ui-checkbox class='toggle' checked=preference.newPaper onChange=(pipe-action (action (mut preference.newPaper)) (action 'savePreference' preference))}}

savePreference(emailPreference) {
      emailPreference.save()
        .then(() => {
          this.get('notify').success(this.l10n.t('Email notifications updated successfully'));
        })
        .catch(() => {
          emailPreference.rollbackAttributes();
          this.get('notify').error(this.l10n.t('An unexpected error occurred.'));
        });
    }

We are passing the parameter(the whole preference object to the action), and then just performing a ‘save’ method on it which will send a PATCH request to the server to update the data.

Thus, in this way, the user can change the email-notification preferences in the Open Event Frontend.

Resources:
Ember data Official guide
Blog on Models and Ember data by Daniel Lavigne: Learning Ember.js Part 4: Models

Source code: https://github.com/fossasia/open-event-frontend/pull/537/files

Continue ReadingImplementing Email Notifications in Open Event Frontend
Read more about the article Implementing the UI of the Scheduler in Open Event Frontend with Sub Rooms and External Events

Implementing the UI of the Scheduler in Open Event Frontend with Sub Rooms and External Events

This blog article will illustrate how exactly the UI of the scheduler is implemented in  Open Event Frontend, using the fullcalendar library. Our discussion primarily will involve the events/view/scheduler  route.

FullCalendar is an open source javascript scheduler with an option to use it’s scheduler functionality. To use it with ember JS, we make use of it’s ember-wrapper (ember-fullcalendar). We begin by installing it via CLI

npm install  ember-fullcalendar

Next to initialise it, we need to specify some properties in the config/environment.js File.

Note: It is wrongly mentioned in the official documentation. (issue) Hence specified in this blog.

emberFullCalendar: {
includeScheduler: true
}

This enables the scheduler functionality of the calendar, next we need to specify the model, which will include the details of the rooms and the events which need to be displayed, the full calendar scheduler requires them in a specific way, hence the model will hook will be:

model() {
return RSVP.hash({
events: [{
title      : 'Session 1',
start      : '2017-07-26T07:08:08',
end        : '2017-07-26T09:08:08',
resourceId : 'a'
}],
rooms: [
{ id: 'a', title: 'Auditorium A' },
{ id: 'b', title: 'Auditorium B', eventColor: 'green' },
{ id: 'c', title: 'Auditorium C', eventColor: 'orange' },
{ id       : 'd', title    : 'Auditorium D', children : [
{ id: 'd1', title: 'Room D1' },
{ id: 'd2', title: 'Room D2' }
] },
{ id: 'e', title: 'Auditorium E' },
{ id: 'f', title: 'Auditorium F', eventColor: 'red' }
]
});

Now we begin with the basic structure we will require for the scheduler in the template files.Since we need an option of dragging and dropping external events, we will split the page into two columns, and make a separate component for storing the list of external events.The component is aptly named external-event-list. Hence the scheduler.hbs will have the following grid column structure.

<div class="ui grid">
<div class="row">
<div class="three wide column">
{{scheduler/external-event-list}}
</div>
<div class="thirteen wide column">
{{full-calendar events=model.events
editable=true
resources=model.rooms
header=header
views=views
viewName='timelineDay'
drop=(action 'drop')
eventReceive=(action 'eventReceive')
ondragover=(action 'eventDrop')
}}
</div>
</div>
</div>

It is important to note here that the full-calendar has various callbacks which are triggered when the events are dragged and dropped. The editable property allows the user to change the events’ venue and timeline, and is enabled only for the organiser of the event. The resources refer to the rooms/locations where the session/events will be held. Since ember executes functionality via functions, each of the standard callback of the full calendar has been translated into an action. Please see the official docs for what each callback does.The only thing which remains is the list of external events. It is necessary for the actual event to be wrapped in fc-event spans, as  they are the default classes for full calendar, and the calendar is able to fetch the event or session name from these spans.

<div id='external-events'>
<h4 class="ui header">{{t 'Events'}}</h4>
<span class='fc-event' draggable="true">My Event 1</span>
<span class='fc-event' draggable="true">My Event 2</span>
<span class='fc-event' draggable="true">My Event 3</span>
</div>

Whenever the events will be dragged and dropped onto the scheduler, they will trigger the drop action inside it, which will fetch the data from them and create an event object for the scheduler.

Resources

Tags :

Open event, Open event frontend, ember JS, ember service, semantic UI, ember-data, ember controllers,  tickets, Open Event API, Ember models

Continue ReadingImplementing the UI of the Scheduler in Open Event Frontend with Sub Rooms and External Events
Read more about the article Enhancing SUSI Line Bot UI by Showing Carousels and Location Map

Enhancing SUSI Line Bot UI by Showing Carousels and Location Map

A good UI primarily focuses on attracting large numbers of users and any good UI designer aims to achieve user satisfaction with a user-friendly design. In this blog, we will learn how to show carousels and location map in SUSI line bot to make UI better and easy to use. We can show web search result from SUSI in form of text responses in line bot but it doesn’t follow design rule as it is not a good approach to show more text in constricted space. Users deem such a layout as less attractive. In SUSI webchat, location is returned with a map which looks more promising to users as illustrated below:

Web search is returned as carousels and is viewable as:

We can implement web search in line by sending an array of text responses. We can do this with the code below:

for (var i = 0; i < 5; i++) {
   msg[i] = "";
   msg[i] = {
       type: 'text',
       text: 'text to be sent here'
   }
}
return client.replyMessage(event.replyToken, msg);

If we send web search using text response it looks messy, a user will not like it that much as it is difficult for a user to read and understand a lot of text in one place. You can see it in the screenshot below:

To make it better looking we will implement carousels for web search in SUSI line bot. Carousels are tiles that can be scrolled horizontally to show rich content. We can implement carousels using this code: 

for (var i = 1; i < 4; i++) {
   title = 'title of carousel';
   query = title;
   msg = 'description of carousel here';
   link = 'link to be opened here';
   if (title.length >= 40) {
       title = title.substring(0, 36);
       title = title + "...";
   }
   if (msg.length >= 60) {
       msg = msg.substring(0, 56);
       msg = msg + "...";
   }
   carousel[i] = {
       "title": title,
       "text": msg,
       "actions": [{
               "type": "uri",
               "label": "View detail",
               "uri": link
           },
           {
               "type": "message",
               "label": "Ask SUSI again",
               "text": query
           }
       ]
   };
}

In above code, we are checking the length of title and description because line API has a limit for the title up to 40 characters and description up to 60 characters. If limit exceeds we then trim the title or description accordingly and use it. Next, we assign title, description, and link to be opened in action to carousel so that we can use it in below code. 

const answer = [{
       type: 'text',
       text: ans
   },
   {
       "type": "template",
       "altText": "this is a carousel template",
       "template": {
           "type": "carousel",
           "columns": [
               carousel[1],
               carousel[2],
               carousel[3]
           ]
       }
   }
]
return client.replyMessage(event.replyToken, answer);

Above code shows an array names answer in which we have added carousels that we have created in last code. After implementing this web search will look like this:

This UI looks neat and easy to read and users will like this. Ask SUSI again will send the title of the carousel to SUSI. To show location map on SUSI line bot just like it is shown on SUSI webchat we will follow this code: 

const answer = [{
       type: 'text',
       text: ans
   },
   {
       "type": "location",
       "title": "name of the place here",
       "address": address,
       "latitude": latitude of location here,
       "longitude": longitude of location here
   }
]
return client.replyMessage(event.replyToken, answer);

We will send a location type message which will include latitude and longitude of the location so that Line bot can show location using map. After implementing location type response it will look like this:

On clicking on this location it will open a map inside line app on which we will see a pointer pointing at the location that we have provided.

If you want to learn more about line messaging API refer to this https://devdocs.line.me/en/#messaging-api

Resources
https://devdocs.line.me/en/#messaging-api

Continue ReadingEnhancing SUSI Line Bot UI by Showing Carousels and Location Map
Read more about the article Handling Requests for hasMany Relationships in Open Event Front-end

Handling Requests for hasMany Relationships in Open Event Front-end

In Open Event Front-end we use Ember Data and JSON API specs to integrate the application with the server. Ember Data provides an easy way to handle API requests, however it does not support a direct POST for saving bulk data which was the problem we faced while implementing event creation using the API.

In this blog we will take a look at how we implemented POST requests for saving hasMany relationships, using an example of sessions-speakers route to see how we saved the tracks, microlocations & session-types. Lets see how we did it.

Fetching the data from the server

Ember by default does not support hasMany requests for getting related model data. However we can use external add on which enable the hasMany Get requests, we use ember-data-has-many-query which is a great add on for querying hasMany relations of a model.

let data = this.modelFor('events.view.edit');
data.tracks = data.event.get('tracks');
data.microlocations = data.event.get('microlocations');
data.sessionTypes = data.event.get('sessionTypes');
return RSVP.hash(data);

In the above example we are querying the tracks, microlocations & sessionTypes which are hasMany relationships, related to the events model. We can simply do a to do a GET request for the related model.

data.event.get('tracks');

In the above example we are retrieving the all the tracks of the event.

Sending a POST request for hasMany relationship
Ember currently does not saving bulk data POST requests for hasMany relations. We solved this by doing a POST request for individual data of the hasMany array.

We start with creating a `promises` array which contains all the individual requests. We then iterate over all the hasMany relations & push it to the `promises` array. Now each request is an individual promise.

let promises = [];

promises.push(this.get('model.event.tracks').toArray().map(track => track.save()));
promises.push(this.get('model.event.sessionTypes').toArray().map(type => type.save()));
promises.push(this.get('model.event.microlocations').toArray().map(location => location.save()));

Once we have all the promises we then use RSVP to make the POST requests. We make use of all() method which takes an array of promises as parameter and resolves all the promises. If the promises are not resolved successfully then we simply notify the user using the notify service, else we redirect to the home page.

RSVP.Promise.all(promises)
  .then(() => {
    this.transitionToRoute('index');
  }, function() {
    this.get('notify').error(this.l10n.t(‘Data did not save. Please try again'));
  });

The result of this we can now retrieve & create new tracks, microlocations & sessionTypes on sessions-speakers route.

Thank you for reading the blog, you can check the source code for the example here.

Resources

 

Continue ReadingHandling Requests for hasMany Relationships in Open Event Front-end
Read more about the article Data Access Layer in Open Event Organizer Android App

Data Access Layer in Open Event Organizer Android App

Open Event Organizer is an Android App for Organizers and Entry Managers. Its core feature is scanning a QR Code to validate Attendee Check In. Other features of the App are to display an overview of sales and tickets management. The App maintains a local database and syncs it with the Open Event API Server. The Data Access Layer in the App is designed such that the data is fetched from the server or taken from the local database according to the user’s need. For example, simply showing the event sales overview to the user will fetch the data from the locally saved database. But when the user wants to see the latest data then the App need to fetch the data from the server to show it to the user and also update the locally saved data for future reference. I will be talking about the data access layer in the Open Event Organizer App in this blog.

The App uses RxJava to perform all the background tasks. So all the data access methods in the app return the Observables which is then subscribed in the presenter to get the data items. So according to the data request, the App has to create the Observable which will either load the data from the locally saved database or fetch the data from the API server. For this, the App has AbstractObservableBuilder class. This class gets to decide which Observable to return on a data request.

Relevant Code:

final class AbstractObservableBuilder<T> {
   ...
   ...
   @NonNull
   private Callable<Observable<T>> getReloadCallable() {
       return () -> {
           if (reload)
               return Observable.empty();
           else
               return diskObservable
                   .doOnNext(item -> Timber.d("Loaded %s From Disk on Thread %s",
                       item.getClass(), Thread.currentThread().getName()));
       };
   }

   @NonNull
   private Observable<T> getConnectionObservable() {
       if (utilModel.isConnected())
           return networkObservable
               .doOnNext(item -> Timber.d("Loaded %s From Network on Thread %s",
                   item.getClass(), Thread.currentThread().getName()));
       else
           return Observable.error(new Throwable(Constants.NO_NETWORK));
   }

   @NonNull
   private <V> ObservableTransformer<V, V> applySchedulers() {
       return observable -> observable
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread());
   }

   @NonNull
   public Observable<T> build() {
       if (diskObservable == null || networkObservable == null)
           throw new IllegalStateException("Network or Disk observable not provided");

       return Observable
               .defer(getReloadCallable())
               .switchIfEmpty(getConnectionObservable())
               .compose(applySchedulers());
   }
}

 

The class is used to build the Abstract Observable which contains both types of Observables, making data request to the API server and the locally saved database. Take a look at the method build. Method getReloadCallable provides an observable which will be the default one to be subscribed which is a disk observable which means data is fetched from the locally saved database. The method checks parameter reload which if true suggests to make the data request to the API server or else to the locally saved database. If the reload is false which means data can be fetched from the locally saved database, getReloadCallable returns the disk observable and the data will be fetched from the locally saved database. If the reload is true which means data request must be made to the API server, then the method returns an empty observable.

The method getConnectionObservable returns a network observable which makes the data request to the API server. In the method build, switchIfEmpty operator is applied on the default observable which is empty if reload is true, and the network observable is passed to it. So when reload is true, network observable is subscribed and when it is false disk observable is subscribed. For example of usage of this class to make a events data request is:

public Observable<Event> getEvents(boolean reload) {
   Observable<Event> diskObservable = Observable.defer(() ->
       databaseRepository.getAllItems(Event.class)
   );

   Observable<Event> networkObservable = Observable.defer(() ->
       eventService.getEvents(JWTUtils.getIdentity(getAuthorization()))
           ...
           ...

   return new AbstractObservableBuilder<Event>(utilModel)
       .reload(reload)
       .withDiskObservable(diskObservable)
       .withNetworkObservable(networkObservable)
       .build();
}

 

So according to the boolean parameter reload, a correct observable is subscribed to complete the data request.

Links:
1. Documentation about the Operators in ReactiveX
2. Information about the Data Access Layer on Wikipedia

Continue ReadingData Access Layer in Open Event Organizer Android App
Read more about the article Using Data Access Object to Store Information

Using Data Access Object to Store Information

We often need to store the information received from the network to retrieve that later. Although we can store and read data directly but by using data access object to store information enables us to do data operations without exposing details of the database. Using data access object is also a best practice in software engineering. Recently I modified Connfa app to store the data received in Open Event format. In this blog, I describe how to use data access object.

The goal is to abstract and encapsulate all access to the data and provide an interface. This is called Data Access Object pattern. In a nutshell, the DAO “knows” which data source (that could be a database, a flat file or even a WebService) to connect to and is specific for this data source. It makes no difference in applications when it accesses a relational database or parses xml files (using a DAO). The DAO is usually able to create an instance of a data object (“to read data”) and also to persist data (“to save data”) to the datasource.

Consider the example from Connfa app in which get the tracks from API and store them in SQL database. We use DAO to create a layer between model and database. Where AbstractEntityDAO is an abstract class which have the functions to perform CRUD operation. We extend it to implement them in our DAO model. Here is TrackDAO structure,

public class TrackDao extends AbstractEntityDAO<Track, Long> {

    public static final String TABLE_NAME = "table_track";

    @Override
    protected String getSearchCondition() {
        return "_id=?";
    }
    
    ...
}

Find the complete class to see the detailed methods to implement search conditions, get key columns, create instance etc.  here.

Here is a general method to get the data from the database. Where getFacade() for the given layer element, this method returns the requested facade object to represent the passed in layer element.

public List<ClassToSave> getAllSafe() {
   ILAPIDBFacade facade = getFacade();
   try {
       facade.open();
       return getAll();

   } finally {
       facade.close();
   }
}

Now we can create an instance to use these methods instead of directly using SQL operations. This functions gets the data and sort it accordingly.

private TrackDao mTrackDao;
 public List<Track> getTracks() {
   List<Track> tracks = mTrackDao.getAllSafe();
   Collections.sort(tracks, new Comparator<Track>() {
       @Override
       public int compare(Track track, Track track2) {
           return Double.compare(track.getOrder(), track2.getOrder());
       }
   });
   return tracks;
}

References:

 

Continue ReadingUsing Data Access Object to Store Information
Read more about the article Receiving Data From the Network Asynchronously

Receiving Data From the Network Asynchronously

We often need to receive information from networks in Android apps. Although it is a simple process but the problem arises when it is done through the main user interaction thread. To understand this problem better consider using an app which shows some text downloaded from the online database. As the user clicks a button to display a text it may take some time to get the HTTP response. So what does an activity do in that time? It creates a lag in the user interface and makes the app to stop responding. Recently I implemented this in Connfa App to download data in Open Event Format.

To solve that problem we use concurrent thread to download the data and send the result back to be processed separating it from main user interaction thread. AsyncTask allows you to perform background operations and publish results on the UI thread without having to manipulate threads and/or handlers.

AsyncTask is designed to be a helper class around Thread and Handler and does not constitute a generic threading framework. AsyncTasks should ideally be used for short operations (a few seconds at the most). If you need to keep threads running for long periods of time, it is highly recommended you use the various APIs provided by the java.util.concurrent package such as Executor, ThreadPoolExecutor and FutureTask.

An asynchronous task is defined by a computation that runs on a background thread and whose result is published on the UI thread. An asynchronous task is defined by 3 generic types, called Params, Progress and Result, and 4 steps, called onPreExecute, doInBackground, onProgressUpdate and onPostExecute.

In this blog, I describe how to download data in an android app with AsyncTask using OkHttp library.

First, you need to add dependency for OkHttp Library,

compile 'com.squareup.okhttp:okhttp:2.4.0'


Extend AsyncTask class and implement all the functions in the class by overriding them to modify them accordingly. Here I declare a class named AsyncDownloader extending AsyncTask. We took the string parameter, integer type progress and string type result to return instead which will be our JSON data and param is our URL. Instead of using get the function of AsyncTask we implement an interface to receive data so the main user interaction thread does not have to wait for the return of the result.

public class AsyncDownloader extends AsyncTask<String, Integer, String> {
    private String jsonData = null;

    public interface JsonDataSetter {
        void setJsonData(String str);
    }


    JsonDataSetter jsonDataSetterListener;

    public AsyncDownloader(JsonDataSetter context) {

        this.jsonDataSetterListener = context;
    }

    @Override
    protected void onPreExecute() {
        super.onPreExecute();
    }

    @Override
    protected String doInBackground(String... params) {

        String url = params[0];

        OkHttpClient client = new OkHttpClient();
        Request request = new Request.Builder()
                .url(url)
                .build();
        Call call = client.newCall(request);

        Response response = null;

        try {
            response = call.execute();

            if (response.isSuccessful()) {
                jsonData = response.body().string();

            } else {
                jsonData = null;
            }

        } catch (IOException e) {
            e.printStackTrace();
        }


        return jsonData;
    }

    @Override
    protected void onPostExecute(String jsonData) {
        super.onPostExecute(jsonData);
        jsonDataSetterListener.setJsonData(jsonData);
    }
}

We download the data in doInBackground making a request using OkHttp library and send the result back in onPostExecute method using our interface method. See above to check out the complete class implementation.

You can make an instance to AsyncDownloader in an activity. Implement interface to receive that data.

Tip: Always close AsyncTask after using to avoid many concurrent tasks at one time.

Find the complete code here.

References:

Continue ReadingReceiving Data From the Network Asynchronously
Read more about the article Advantage of Open Event Format over xCal, pentabarf/frab XML and iCal

Advantage of Open Event Format over xCal, pentabarf/frab XML and iCal

Event apps like Giggity and Giraffe use event formats like xCal, pentabarf/frab XML, iCal etc. In this blog, I present some of the advantages of using FOSSASIA’s Open Event data format over other formats. I added support for Open Event format in these two apps so I describe the advantages and improvements that were made with respect to them.

  • The main problem that is faced in Giggity app is that all the data like social links, microlocations, the link for the logo file etc., can not be fetched from a single file, so a separate raw file is being added to provide this data. Our Open Event format provides all this information from the single URL that could be received from the server so no need to use any separate file.
  • Here is the pentabarf format data for FOSSASIA 2016 conference excluding sessions. Although it provides all the necessary information it leaves the information for logo URL, details for longitude and latitude for microlocations (rooms) and links to social media and website. While the open event format provides all the missing details including some extra information like language, comments etc. See FOSSASIA 2016 Open Event format sample.
<conference>
<title>FOSSASIA 2016</title>
<subtitle/>
<venue>Science Centre Road</venue>
<city>Singapore</city>
<start>2016-03-18</start>
<end>2016-03-20</end>
<days>3</days>
<day_change>09:00:00</day_change>
<timeslot_duration>00:00:00</timeslot_duration>
</conference>
  • The parsing of received file format gets very complicated in case of iCal, xCal etc. as tags needs to be matched to get the data. Howsoever there are various libraries available for parsing JSON data. So we can create simply an array list of the received data to send it to the adapter. See this example for more information of working code. You can also see the parser for iCal to compare the complexity of the code.
  • The other more common problem is the structure of the formats received is sometimes it becomes complicated to define the sub parts of a single element. For example for the location we define latitude and longitude separately while in iCal format it is just separated by a comma. For example for

iCal 

GEO:1.333194;103.736132

JSON

{
           "id": 1,
           "name": "Stage 1",
           "floor": 0,
           "latitude": 37.425420,
           "longitude": -122.080291,
           "room": null
}

And the information provided is more detailed.

  • Open Event format is well documented and it makes it easier for other developers to work on it. Find the documentation here.

References:

 

Continue ReadingAdvantage of Open Event Format over xCal, pentabarf/frab XML and iCal
Read more about the article Export an Event using APIs of Open Event Server

Export an Event using APIs of Open Event Server

We in FOSSASIA’s Open Event Server project, allow the organizer, co-organizer and the admins to export all the data related to an event in the form of an archive of JSON files. This way the data can be reused in some other place for various different purposes. The basic workflow is something like this:

  • Send a POST request in the /events/{event_id}/export/json with a payload containing whether you require the various media files.
  • The POST request starts a celery task in the background to start extracting data related to event and jsonifying them
  • The celery task url is returned as a response. Sending a GET request to this url gives the status of the task. If the status is either FAILED or SUCCESS then there is the corresponding error message or the result.
  • Separate JSON files for events, speakers, sessions, micro-locations, tracks, session types and custom forms are created.
  • All this files are then archived and the zip is then served on the endpoint /events/{event_id}/exports/{path}
  • Sending a GET request to the above mentioned endpoint downloads a zip containing all the data related to the endpoint.

Let’s dive into each of these points one-by-one

POST request ( /events/{event_id}/export/json)

For making a POST request you firstly need a JWT authentication like most of the other API endpoints. You need to send a payload containing the settings for whether you want the media files related with the event to be downloaded along with the JSON files. An example payload looks like this:

{
   "image": true,
   "video": true,
   "document": true,
   "audio": true
 }

def export_event(event_id):
    from helpers.tasks import export_event_task

    settings = EXPORT_SETTING
    settings['image'] = request.json.get('image', False)
    settings['video'] = request.json.get('video', False)
    settings['document'] = request.json.get('document', False)
    settings['audio'] = request.json.get('audio', False)
    # queue task
    task = export_event_task.delay(
        current_identity.email, event_id, settings)
    # create Job
    create_export_job(task.id, event_id)

    # in case of testing
    if current_app.config.get('CELERY_ALWAYS_EAGER'):
        # send_export_mail(event_id, task.get())
        TASK_RESULTS[task.id] = {
            'result': task.get(),
            'state': task.state
        }
    return jsonify(
        task_url=url_for('tasks.celery_task', task_id=task.id)
    )


Taking the settings about the media files and the event id, we pass them as parameter to the export event celery task and queue up the task. We then create an entry in the database with the task url and the event id and the user who triggered the export to keep a record of the activity. After that we return as response the url for the celery task to the user.

If the celery task is still underway it show a response with ‘state’:’WAITING’. Once, the task is completed, the value of ‘state’ is either ‘FAILED’ or ‘SUCCESS’. If it is SUCCESS it returns the result of the task, in this case the download url for the zip.

Celery Task to Export Event

Exporting an event is a very time consuming process and we don’t want that this process to come in the way of user interaction with other services. So we needed to use a queueing system that would queue the tasks and execute them in the background with disturbing the main worker from executing the other user requests. We have used celery to queue tasks in the background and execute them without disturbing the other user requests.

We have created a celery task namely “export.event” which calls the event_export_task_base() which in turn calls the export_event_json() where all the jsonification process is carried out. To start the celery task all we do is export_event_task.delay(event_id, settings) and it return a celery task object with a task id that can be used to check the status of the task.

@celery.task(base=RequestContextTask, name='export.event', bind=True)
def export_event_task(self, email, event_id, settings):
    event = safe_query(db, Event, 'id', event_id, 'event_id')
    try:
        logging.info('Exporting started')
        path = event_export_task_base(event_id, settings)
        # task_id = self.request.id.__str__()  # str(async result)
        download_url = path

        result = {
            'download_url': download_url
        }
        logging.info('Exporting done.. sending email')
        send_export_mail(email=email, event_name=event.name, download_url=download_url)
    except Exception as e:
        print(traceback.format_exc())
        result = {'__error': True, 'result': str(e)}
        logging.info('Error in exporting.. sending email')
        send_export_mail(email=email, event_name=event.name, error_text=str(e))

    return result


After exporting a path to the export zip is returned. We then get the downloading endpoint and return it as the result of the celery task. In case there is an error in the celery task, we print an entire traceback in the celery worker and return the error as a result.

Make the Exported Zip Ready

We have a separate export_helpers.py file in the helpers module of API for performing various tasks related to exporting all the data of the event. The most important function in this file is the export_event_json(). This file accepts the event_id and the settings dictionary. In the export helpers we have global constant dictionaries which contain the order in which the fields are to appear in the JSON files created while exporting.

Firstly, we create the directory for storing the exported JSON and finally the archive of all the JSON files. Then we have a global dictionary named EXPORTS which contains all the tables and their corresponding Models which we want to extract from the database and store as JSON.  From the EXPORTS dict we get the Model names. We use this Models to make queries with the given event_id and retrieve the data from the database. After retrieving data, we use another helper function named _order_json which jsonifies the sqlalchemy data in the order that is mentioned in the dictionary. After this we download the media data, i.e. the slides, images, videos etc. related to that particular Model depending on the settings.

def export_event_json(event_id, settings):
    """
    Exports the event as a zip on the server and return its path
    """
    # make directory
    exports_dir = app.config['BASE_DIR'] + '/static/uploads/exports/'
    if not os.path.isdir(exports_dir):
        os.mkdir(exports_dir)
    dir_path = exports_dir + 'event%d' % int(event_id)
    if os.path.isdir(dir_path):
        shutil.rmtree(dir_path, ignore_errors=True)
    os.mkdir(dir_path)
    # save to directory
    for e in EXPORTS:
        if e[0] == 'event':
            query_obj = db.session.query(e[1]).filter(
                e[1].id == event_id).first()
            data = _order_json(dict(query_obj.__dict__), e)
            _download_media(data, 'event', dir_path, settings)
        else:
            query_objs = db.session.query(e[1]).filter(
                e[1].event_id == event_id).all()
            data = [_order_json(dict(query_obj.__dict__), e) for query_obj in query_objs]
            for count in range(len(data)):
                data[count] = _order_json(data[count], e)
                _download_media(data[count], e[0], dir_path, settings)
        data_str = json.dumps(data, indent=4, ensure_ascii=False).encode('utf-8')
        fp = open(dir_path + '/' + e[0], 'w')
        fp.write(data_str)
        fp.close()
    # add meta
    data_str = json.dumps(
        _generate_meta(), sort_keys=True,
        indent=4, ensure_ascii=False
    ).encode('utf-8')
    fp = open(dir_path + '/meta', 'w')
    fp.write(data_str)
    fp.close()
    # make zip
    shutil.make_archive(dir_path, 'zip', dir_path)
    dir_path = dir_path + ".zip"

    storage_path = UPLOAD_PATHS['exports']['zip'].format(
        event_id=event_id
    )
    uploaded_file = UploadedFile(dir_path, dir_path.rsplit('/', 1)[1])
    storage_url = upload(uploaded_file, storage_path)

    return storage_url


After we receive the json data from the _order_json() function, we create a dump of the json using json.dumps with an indentation of 4 spaces and utf-8 encoding. Then we save this dump in a file named according to the model from which the data was retrieved. This process is repeated for all the models that are mentioned in the EXPORTS dictionary. After all the JSON files are created and all the media is downloaded, we make a zip of the folder.

To do this we use shutil.make_archive. It creates a zip and uploads the zip to the storage service used by the server such as S3, google storage, etc. and returns the url for the zip through which it can be accessed.

Apart from this function, the other major function in this file is to create an export job entry in the database so that we can keep a track about which used started a task related to which event and help us in debugging and security purposes.

Downloading the Zip File

After the exporting is completed, if you send a GET request to the task url, you get a response similar to this: 

{
   "result": {
     "download_url": "http://localhost:5000/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
   },
   "state": "SUCCESS"
 }

So on opening the download url in the browser or using any other tool, you can download the zip file.

One big question however remains is, all the workflow is okay but how do you understand after sending the POST request, that the task is completed and ready to be downloaded? One way of solving this problem is a technique known as polling. In polling what we do is we send a GET request repeatedly after every fixed interval of time. So, what we do is from the POST request we get the url for the export task. You keep polling this task url until the state is either “FAILED” or “SUCCESS”. If it is a SUCCESS you append the download url somewhere in your website which can then clicked to download the archived export of the event.

 

Reference:

 

Continue ReadingExport an Event using APIs of Open Event Server