Introducing Priority Kaizen Harvester for loklak server

In the previous blog post, I discussed the changes made in loklak’s Kaizen harvester so it could be extended and other harvesting strategies could be introduced. Those changes made it possible to introduce a new harvesting strategy as PriorityKaizen harvester which uses a priority queue to store the queries that are to be processed. In this blog post, I will be discussing the process through which this new harvesting strategy was introduced in loklak.

Background, motivation and approach

Before jumping into the changes, we first need to understand that why do we need this new harvesting strategy. Let us start by discussing the issue with the Kaizen harvester.

The produce consumer imbalance in Kaizen harvester

Kaizen uses a simple hash queue to store queries. When the queue is full, new queries are dropped. But numbers of queries produced after searching for one query is much higher than the consumption rate, i.e. the queries are bound to overflow and new queries that arrive would get dropped. (See loklak/loklak_server#1156)

Learnings from attempt to add blocking queue for queries

As a solution to this problem, I first tried to use a blocking queue to store the queries. In this implementation, the producers would get blocked before putting the queries in the queue if it is full and would wait until there is space for more. This way, we would have a good balance between consumers and producers as the consumers would be waiting until producers can free up space for them –

public class BlockingKaizenHarvester extends KaizenHarvester {
   ...
   public BlockingKaizenHarvester() {
       super(new KaizenQueries() {
           ...
           private BlockingQueue<String> queries = new ArrayBlockingQueue<>(maxSize);

           @Override
           public boolean addQuery(String query) {
               if (this.queries.contains(query)) {
                   return false;
               }
               try {
                   this.queries.offer(query, this.blockingTimeout, TimeUnit.SECONDS);
                   return true;
               } catch (InterruptedException e) {
                   DAO.severe("BlockingKaizen Couldn't add query: " + query, e);
                   return false;
               }
           }
           @Override
           public String getQuery() {
               try {
                   return this.queries.take();
               } catch (InterruptedException e) {
                   DAO.severe("BlockingKaizen Couldn't get any query", e);
                   return null;
               }
           }
           ...
       });
   }
}

[SOURCE, loklak/loklak_server#1210]

But there is an issue here. The consumers themselves are producers of even higher rate. When a search is performed, queries are requested to be appended to the KaizenQueries instance for the object (which here, would implement a blocking queue). Now let us consider the case where queue is full and a thread requests a query from the queue and scrapes data. Now when the scraping is finished, many new queries are requested to be inserted to most of them get blocked (because the queue would be full again after one query getting inserted).

Therefore, using a blocking queue in KaizenQueries is not a good thing to do.

Other considerations

After the failure of introducing the Blocking Kaizen harvester, we looked for other alternatives for storing queries. We came across multilevel queues, persistent disk queues and priority queues.

Multilevel queues sounded like a good idea at first where we would have multiple queues for storing queries. But eventually, this would just boil down to how much queue size are we allowing and the queries would eventually get dropped.

Persistent disk queues would allow us to store greater number of queries but the major disadvantage was lookup time. It would terribly slow to check if a query already exists in the disk queue when the queue is large. Also, since the queries would always increase practically, the disk queue would also go out of hand at some point in time.

So by now, we were clear that not dropping queries is not an alternative. So what we had to use the limited size queue smartly so that we do not drop queries that are important.

Solution: Priority Queue

So a good solution to our problem was a priority queue. We could assign a higher score to queries that come from more popular Tweets and they would go higher in the queue and do not drop off until we have even higher priority queried in the queue.

Assigning score to a Tweet

Score for a tweet was decided using the following formula –

α= 5* (retweet count)+(favourite count)

score=α/(α+10*exp(-0.01*α))

This equation generates a score between zero and one from the retweet and favourite count of a Tweet. This normalisation of score would ensure we do not assign an insanely large score to Tweets with a high retweet and favourite count. You can see the behaviour for the second mentioned equation here.

Graph?

Changes required in existing Kaizen harvester

To take a score into account, it became necessary to add an interface to also provide a score as a parameter to the addQuery() method in KaizenQueries. Also, not all queries can have a score associated with it, for example, if we add a query that would search for Tweets older than the oldest in the current timeline, giving it a score wouldn’t be possible as it would not be associated with a single Tweet. To tackle this, a default score of 0.5 was given to these queries –

public abstract class KaizenQueries {

   public boolean addQuery(String query) {
       return this.addQuery(query, 0.5);
   }

   public abstract boolean addQuery(String query, double score);
   ...
}

[SOURCE]

Defining appropriate KaizenQueries object

The KaizenQueries object for a priority queue had to define a wrapper class that would hold the query and its score together so that they could be inserted in a queue as a single object.

ScoreWrapper and comparator

The ScoreWrapper is a simple class that stores score and query object together –

private class ScoreWrapper {

   private double score;
   private String query;

   ScoreWrapper(String m, double score) {
       this.query = m;
       this.score = score;
   }

}

[SOURCE]

In order to define a way to sort the ScoreWrapper objects in the priority queue, we need to define a Comparator for it –

private Comparator<ScoreWrapper> scoreComparator = (scoreWrapper, t1) -> (int) (scoreWrapper.score - t1.score);

[SOURCE]

Putting things together

Now that we have all the ingredients to declare our priority queue, we can also declare the strategy to getQuery and putQuery in the corresponding KaizenQueries object –

public class PriorityKaizenHarvester extends KaizenHarvester {

   private static class PriorityKaizenQueries extends KaizenQueries {
       ...
       private Queue<ScoreWrapper> queue;
       private int maxSize;

       public PriorityKaizenQueries(int size) {
           this.maxSize = size;
           queue = new PriorityQueue<>(size, scoreComparator);
       }

       @Override
       public boolean addQuery(String query, double score) {
           ScoreWrapper sw = new ScoreWrapper(query, score);
           if (this.queue.contains(sw)) {
               return false;
           }
           try {
               this.queue.add(sw);
               return true;
           } catch (IllegalStateException e) {
               return false;
           }
       }

       @Override
       public String getQuery() {
           return this.queue.poll().query;
       }
       ...
}

[SOURCE]

Conclusion

In this blog post, I discussed the process in which PriorityKaizen harvester was introduced to loklak. This strategy is a flavour of Kaizen harvester which uses a priority queue to store queries that are to be processed. These changes were possible because of a previous patch which allowed extending of Kaizen harvester.

The changes were introduced in pull request loklak/loklak#1240 by @singhpratyush (me).

Resources

Continue ReadingIntroducing Priority Kaizen Harvester for loklak server

Fetching URL for Embedded Twitter Videos in loklak server

The primary web service that loklak scrapes is Twitter. Being a news and social networking service, Twitter allows its users to post videos directly to Twitter and they convey more thoughts than what text can. But for an automated scraper, getting the links is not a simple task.

Let us see that what were the problems we faced with videos and how we solved them in the loklak server project.

Previous setup and embedded videos

In the previous version of loklak server, the TwitterScraper searched for videos in 2 ways –

  1. Youtube links
  2. HTML5 video links

To fetch the video URL from HTML5 video, following snippet was used –

if ((p = input.indexOf("<source video-src")) >= 0 && input.indexOf("type=\"video/") > p) {
   String video_url = new prop(input, p, "video-src").value;
   videos.add
   continue;
}

Here, input is the current line from raw HTML that is being processed and prop is a class defined in loklak that is useful in parsing HTML attributes. So in this way, the HTML5 videos were extracted.

The Problem – Embedded videos

Though the previous setup had no issues, it was useless as Twitter embeds the videos in an iFrame and therefore, can’t be fetched using simple HTML5 tag extraction.

If we take the following Tweet for example,

the requested HTML from the search page contains video in following format –

<src="https://twitter.com/i/videos/tweet/881946694413422593?embed_source=clientlib&player_id=0&rpc_init=1" allowfullscreen="" id="player_tweet_881946694413422593" style="width: 100%; height: 100%; position: absolute; top: 0; left: 0;">

So we needed to come up with a better technique to get those videos.

Parsing video URL from iFrame

The <div> which contains video is marked with AdaptiveMedia-videoContainer class. So if a Tweet has an iFrame containing video, it will also have the mentioned class.

Also, the source of iFrame is of the form https://twitter.com/i/videos/tweet/{Tweet-ID}. So now we can programmatically go to any Tweet’s video and parse it to get results.

Extracting video URL from iFrame source

Now that we have the source of iFrame, we can easily get the video source using the following flow –

public final static Pattern videoURL = Pattern.compile("video_url\\\":\\\"(.*?)\\\"");

private static String[] fetchTwitterIframeVideos(String iframeURL) {
   // Read fron iframeURL line by line into BufferReader br
   while ((line = br.readLine()) != null ) {
       int index;
       if ((index = line.indexOf("data-config=")) >= 0) {
           String jsonEscHTML = (new prop(line, index, "data-config")).value;
           String jsonUnescHTML = HtmlEscape.unescapeHtml(jsonEscHTML);
           Matcher m = videoURL.matcher(jsonUnescHTML);
           if (!m.find()) {
               return new String[]{};
           }
           String url = m.group(1);
           url = url.replace("\\/", "/");  // Clean URL
           /*
            * Play with url and return results
            */
       }
   }
}

MP4 and M3U8 URLs

If we encounter mp4 URLs, we’re fine as it is the direct link to video. But if we encounter m3u8 URL, we need to process it further before we can actually get to the videos.

For Twitter, the hosted m3u8 videos contain link to further m3u8 videos which are of different resolution. These m3u8 videos again contain link to various .ts files that contain actual video in parts of 3 seconds length each to support better streaming experience on the web.

To resolve videos in such a setup, we need to recursively parse m3u8 files and collect all the .ts videos.

private static String[] extractM3u8(String url) {
   return extractM3u8(url, "https://video.twimg.com/");
}

private static String[] extractM3u8(String url, String baseURL) {
   // Read from baseURL + url line by line
   while ((line = br.readLine()) != null) {
       if (line.startsWith("#")) {  // Skip comments in m3u8
           continue;
       }
       String currentURL = (new URL(new URL(baseURL), line)).toString();
       if (currentURL.endsWith(".m3u8")) {
           String[] more = extractM3u8(currentURL, baseURL);  // Recursively add all
           Collections.addAll(links, more);
       } else {
           links.add(currentURL);
       }
   }
   return links.toArray(new String[links.size()]);
}

And then in fetchTwitterIframeVideos, we can return the all .ts URLs for the video –

if (url.endsWith(".mp4")) {
   return new String[]{url};
} else if (url.endsWith(".m3u8")) {
   return extractM3u8(url);
}

Putting things together

Finally, the TwitterScraper can discover the video links by tweaking a little –

if (input.indexOf("AdaptiveMedia-videoContainer") > 0) {
   // Fetch Tweet ID
   String tweetURL = props.get("tweetstatusurl").value;
   int slashIndex = tweetURL.lastIndexOf('/');
   if (slashIndex < 0) {
       continue;
   }
   String tweetID = tweetURL.substring(slashIndex + 1);
   String iframeURL = "https://twitter.com/i/videos/tweet/" + tweetID;
   String[] videoURLs = fetchTwitterIframeVideos(iframeURL);
   Collections.addAll(videos, videoURLs);
}

Conclusion

This blog post explained the process of extracting video URL from Twitter and the problem faced. The discussed change enabled loklak to extract and serve URLs to video for tweets. It was introduced in PR loklak/loklak_server#1193 by me (@singhpratyush).

The service was further enhanced to collect single mp4 link for videos (see PR loklak/loklak_server#1206), which is discussed in another blog post.

Resources

Continue ReadingFetching URL for Embedded Twitter Videos in loklak server

Creating Sensor Libraries in PSLab

The I2C bus of the PSLab enables access to a whole range of sensors capable of measuring parameters ranging from light intensity, humidity, and temperature, to acceleration, passive infrared, and magnetism.

Support for each sensor in the communication library is implemented as a small Python library that depends in the I2C communication module for PSLab.

However, most sensors have capabilities that are not just limited to data readouts, but also enable various configuration options.

This blog post explains how a common format followed across the sensor libraries enables the graphical utilities such as data loggers and control panels to dynamically create widgets pertaining to the various configuration options.

The following variables and methods must be present in each sensor file in order to enable the graphical utilities to correctly function:

Name: A generic name for the sensor to be shown in menus and such. e.g. ‘Altimeter BMP180’

GetRaw(): A function that returns a list of values read from the sensor. The values may have been directly read from the sensor, or derived based on some parameters/equations.

For example, the BMP180 altitude sensor is actually a pressure and temperature sensor. The altitude value is derived from these two using an equation. Therefore, the getRaw function for the BMP180 returns a list of three values, viz, [temperature, pressure, altitude]

NUMPLOTS: A constant that stores the number of available dataPoints in the list returned by the getRaw function. This enables the graphical utilities to create required number of traces . In case of the BMP180, it is 3

PLOTNAMES: A list of text strings to be displayed in the plot legend . For the BMP180, the following list is defined : [‘Temperature’, ‘Pressure’, ‘Altitude’]

params: A dictionary that stores the function names for configuring various features of the sensor, and options that can be passed to the function. For example, for the BMP180 altimeter, and oversampling parameter is available, and can take values 0,1,2 and 3 . Therefore, params = {‘setOversampling’: [0, 1, 2, 3]}

The Sensor data Logger application uses this dictionary to auto-generate menus with the ‘key’ as the name , and corresponding ‘values’ as a submenu . When the user opens a menu and clicks on a ‘value’ , the ‘value’ is passed to a function whose name is the corresponding key , and which must be defined in the sensor’s class.

When the above are defined, menus and plots are automatically generated, and saves considerable time and effort for graphical utility development since no sensor specific code needs to be added on that front.

The following Params dictionary defined in the class of MPU6050 creates a menu as shown in the second image:

self.params = { 'powerUp':['Go'],
'setGyroRange':[250,500,1000,2000],
'setAccelRange':[2,4,8,16],
'KalmanFilter':[.01,.1,1,10,100,1000,10000,'OFF']
}

As shown in the image , when the user clicks on ‘8’ , MPU6050.setAccelRange(8) is executed.

Improving the flexibility of the auto-generated menus

The above approach is a little limited, since only a fixed set of values can be used for configuration options, and there may be cases where a flexible input is required.

This is the case with the Kalman filter option, where the user may want to provide the intensity of the filter as a decimal value. Therefore we shall implement a more flexible route for the params dictionary, and allow the value to certains keys to be objects other than lists.

Functions with user defined variable inputs are defined as Spinbox/QDoubleSpinBox.

KalmanFilter is defined in the following entry in Params:

‘KalmanFilter’:{‘dataType’:’double’,’min’:0,’max’:1000,’prefix’:’value: ‘}

Screenshot of the improved UI with MPU6050.

In this instance, the user can input a custom value, and the KalmanFilter function is called with the same.

Additional Reading:

[1]: Using sensors with PSLab Android

[2]: Analyzing sensor data with PSLab android
[3]: YouTube video to understand analysis of data from MPU6050 with Arduino – https://www.youtube.com/watch?v=taZHl4Mr-Pk

Continue ReadingCreating Sensor Libraries in PSLab

Declaring Ember Data Model Defaults in Open Event Frontend

This article will illustrate the ways to declare the model defaults for JavaScript data types ‘boolean’, ‘string’, ‘number’, ‘object’ and ‘array’ as used in Open Event frontend project.

While working with ember-data models which define the properties and behaviour of data, in Open Event frontend project, we needed to preset the values for certain attributes if they are not supplied along with data. To preset the values in ember-data we need to specify the default values to the attribute. The process of setting the default values is very much similar to the way we handle it in other databases like MySQL and we might have certainly done it many times there. In ember, we set the values like this.

export default Model.extend({
 type : attr('string', { defaultValue: 'text' }),
 isRequired : attr('boolean', { defaultValue: false })
});


This way we could set values for attributes having data types, ‘boolean’, ‘string’ and ‘number’. But when it came to data types, ‘object’ and ‘array’ we were unable to set the default values in similar way reason being ember data does not provide support for ‘array’ and ‘object’ data types. Also, we tried to play around by not providing the first argument to the DS.attr() method, then Ember does not remain the value for that particular attribute empty or unhandled rather it forced to matching JavaScript type. The other and the obvious way which clicked to us was this.The attributes are properties defined to convert the JSON data coming from our server into a record, and serializing a record to save back to the server after it has been modified. In above code we defined that ‘type’  has a default value of ‘text’ and ‘isRequired’ has default value ‘false’ which will be present at the time of model’s creation.

export default Model.extend({
 startTime : attr('object', { defaultValue: {} })
});


But soon after executing the code, we got a warning which can be seen below.The above code implies that whenever we create a new
event model then it would be expected to have an attribute startTime with value {}.

Non primitive defaultValues are deprecated because they are shared between all instances. If you would like to use a complex object as a default value please provide a function that returns the complex object.

The warning explains that ember-data model extends from Ember.Object, which means that arrays and objects will be shared among all instances of that model hence the use of non-primitive (apart from boolean, number & string) default values are deprecated.

Now again we thought of something that can handle ‘array’ and ‘object’ data types. We thought of adding a function which would convert them to the custom type. To add that we needed to provide proper serialize and deserialize methods for processing the data properly which was a bit tedious process and increase the overall complexity of code both in terms of execution as well as understanding.

After reading the available resources, we got to know that the defaultValue accepts the function as well. So what we finally did was passed a function which will add the months and days to the date object based on the current time which can be calculated using moment.

export default Model.extend({
 startTime : attr('date', { defaultValue: () => moment().add(1, 'months').startOf('day').toDate() }) 
});


The ability to pass afunction to defaultValue proved very helpful. It suits best if we want to set custom defaults.That’s it! Passing the function will ensure that every
startTime attribute contains it’s own date instance.

Additional Resources:

**image is licenced under free to use CC0 Public Domain

Continue ReadingDeclaring Ember Data Model Defaults in Open Event Frontend

Dynamic Ticket Analysis UI using Data Binding in Open Event Android Orga App

Any event manager application has the responsibility to show the analytics about the event to the organiser and in Open Event Android Orga App (Github Repo), we wanted to achieve a way to display the analytics of total and sold tickets with the data present to us.
To analyse, we have a list of tickets, which are divided into 3 categories:

  • Free
  • Paid
  • Donation

Our goal was to show information about total tickets and the amount of sold tickets per category. This blog will focus on the dynamic UI creation for the ticket analysis component of the Event Details Dashboard using Android Layout Data Binding. By using Data Binding, we not only reduced the amount of Java Boilerplate code we would have to write, but also accomplished UI reuse in just XML which wouldn’t have been possible without it. You’ll see in a moment what I mean.

Properties

So first, we’d need to define some properties which will be bound in the UI. These properties are declared in the Event model and their type is ObservableLong provided by the Android DataBinding package. The reason why we are using these instead of primitives is because these fields being Observable, will update the UI as soon as they are updated, without requiring the programmer to set the View Property at all.

There are six fields, 3 for total tickets of each type and 3 for sold tickets

public final ObservableLong freeTickets = new ObservableLong();
public final ObservableLong paidTickets = new ObservableLong();
public final ObservableLong donationTickets = new ObservableLong();

public final ObservableLong soldFreeTickets = new ObservableLong();
public final ObservableLong soldPaidTickets = new ObservableLong();
public final ObservableLong soldDonationTickets = new ObservableLong();

Some more advantages we get from using these are the batch view update and the use of computed properties in UI. Imagine having a TextView display the amount of free tickets and a progress bar showing the percentage of free tickets sold. Traditionally, you’d have to set the text and compute the percentage and set the progress bar as the data changes, whereas you can just use the fields in layout as is in both TextView and ProgressBar with the computations required and they’ll work in harmony.

We have leveraged this feature to show the analytics component of tickets with a

  • Ticket Type
  • Circular Progress Bar
  • Sold Tickets
  • Total Tickets

All using the XML layout and databinding

Ticket Component

For each ticket component, we have 4 variables, namely

  • Ticket Type Name
  • Total Amount
  • Completed Amount (Sold Tickets)
  • Color

First 3 are fairly self explanatory, the color attribute we used in our component needs a little bit of description. We decided to give each ticket category its own color for circular progress bar for aesthetics. So, we need each component to have its own color attribute too. But this is not a normal android color ID or a hex. We needed 2 variants of the same color to show in the circular progress to discern the total and completed part. As we are using Material Color Palette, which has a color divided by intensities, we used 500 variant for completed portion and 100 (lighter) variant for the background of circular progress.

Let’s look at the layout now:

<data>
    <variable name="color" type="String" />
    <variable name="ticketName" type="String" />
    <variable name="total" type="long" />
    <variable name="completed" type="long" />
</data>

<LinearLayout
     android:orientation="vertical">
    <TextView
        android:text="@{ticketName}" />
    <FrameLayout>
        <com.mikhaellopez.circularprogressbar.CircularProgressBar
            app:circular_progress_color="@{color}"
            app:progress_with_animation="@{total == 0 ? 0 : (int) ((completed*100)/total)}" />

        <LinearLayout
            android:orientation="horizontal">

            <TextView
                android:text="@{completed}" />
            <TextView
                android:text='@{"/" + total}' />

        </LinearLayout>
    </FrameLayout>

    <TextView
        android:text='@{(total == 0 ? 0 : (int) ((completed*100)/total)) + "%"}' />

</LinearLayout>

Note: The layout snippet is not complete. Only attribute names to be discussed in the blog are shown for brevity

As you can see, after the data variable declarations, we have a CardView first showing the ticket name on top, and then we have a FrameLayout wrapping the circular progress and a textview showing the Sold/Total tickets.

Circular Progress Bar

Let’s discuss the circular progress first, we have used this library to create a circular progress bar, the two other attributes circular_progress_color and progress_with_animation are specific to Open Event Orga Application and we have created custom adapters for them:

@BindingAdapter("progress_with_animation")
public static void bindCircularProgress(CircularProgressBar circularProgressBar, int progress) {
    circularProgressBar.setProgressWithAnimation(progress, 500);
}

@BindingAdapter("circular_progress_color")
public static void bindCircularProgressColor(CircularProgressBar circularProgressBar, String colorName) {
    Context context = circularProgressBar.getContext();
    Resources resources = circularProgressBar.getResources();

    int color = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_500", "color", context.getPackageName()));
    int bgColor = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_100", "color", context.getPackageName()));

    circularProgressBar.setColor(color);
    circularProgressBar.setBackgroundColor(bgColor);
}
  • progress_with_animation sets the provided integer value as the progress of the circular progress bar with an animation of 500 ms
  • circular_progress_color finds the 100 and 500 variant of the color name string provided and sets them as background and foreground color of the progress bar

These are the color definitions we have used in the app:

<color name="light_blue_100">#B3E5FC</color>
<color name="light_blue_500">#03A9F4</color>
<color name="purple_100">#E1BEE7</color>
<color name="purple_500">#9C27B0</color>
<color name="red_100">#ffcdd2</color>
<color name="red_500">#f44336</color>

As you can that if we pass purple as the color name, it’ll load purple_100 and purple_500 and set it as corresponding background and foreground color

Other Properties

Now, let’s talk about other properties like the progress value :

  • total == 0 ? 0 : (int) ((completed*100)/total)The conditional is used to prevent divide by zero error.
    The same expression is used to display the circular progress and percentage text in the TextView at the bottom of the layout
  • completed and “/” + total are used to in TextViews of different sizes to create a nice design with completed/total format

This completes our ticket component design and now we’ll see how to reuse this component to display different ticket types.

Composite Layout

To use the ticket component, we just include the layout and bind specific variables from Event model to create a dynamic layout like this:

<data>
    <variable
        name="event"
        type="org.fossasia.openevent.app.data.models.Event" />
</data>

<LinearLayout
    android:orientation="vertical">

    <TextView
        android:text="@string/tickets" />

    <LinearLayout
        android:orientation="horizontal">

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"light_blue"}'
            bind:completed="@{event.soldFreeTickets}"
            bind:ticketName="@{@string/ticket_free}"
            bind:total="@{event.freeTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"purple"}'
            bind:completed="@{event.soldPaidTickets}"
            bind:ticketName="@{@string/ticket_paid}"
            bind:total="@{event.paidTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"red"}'
            bind:completed="@{event.soldDonationTickets}"
            bind:ticketName="@{@string/ticket_donation}"
            bind:total="@{event.donationTickets}" />
    </LinearLayout>

</LinearLayout>

The layout consists of a horizontal with 3 equally divided ticket components,

  • Free Ticket Component -> Light Blue
  • Paid Ticket Component -> Purple
  • Donation Ticket Component -> Red

This is how it looks on a device

 

So this is how data binding made us accomplish easily which would have been a very convoluted solution using traditional ID based view binding. For more info about data binding, refer to these sites:

https://developer.android.com/topic/libraries/data-binding/index.html

http://www.vogella.com/tutorials/AndroidDatabinding/article.html

Continue ReadingDynamic Ticket Analysis UI using Data Binding in Open Event Android Orga App

Calculation of the Frame Size of the Chat Bubble in SUSI iOS

We receive intelligent responses from the SUSI Server based on our query. Each response contains a different set of actions and the content of the action can be of variable sizes, map, string, table, pie chart, etc. To make the chat bubble size dynamic in the SUSI iOS client, we need to check the action type. For each action, we calculate a different frame size which makes the size of the chat bubble dynamic and hence solving the issue of dynamic size of these bubbles.

In order to calculate the frame size, as mentioned above, we need to check the action type of that message. Let’s start by first making the API call sending the query and getting the action types as a response.

func queryResponse(_ params: [String : AnyObject], _ completion: @escaping(_ messages: List<Message>?, _ success: Bool, _ error: String?) -> Void) {

   let url = getApiUrl(UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.ipAddress) as! String, Methods.Chat)

       _ = makeRequest(url, .get, [:], parameters: params, completion: { (results, message) in
           if let _ = message {
               completion(nil, false, ResponseMessages.ServerError)
           } else if let results = results {

               guard let response = results as? [String : AnyObject] else {
                   completion(nil, false, ResponseMessages.InvalidParams)
                   return
               }

               let messages = Message.getAllActions(data: response)
               completion(messages, true, nil)
           }
           return
})
}

Here, we are sending the query in the params dictionary. The `makeRequest` method makes the actual API call and returns a results object and an error object if any which default to `nil`. First, we check if the error variable is `nil` or not and if it is, we parse the complete response by using a helper method created in the Message object called `getAllActions`. This basically takes the response and gives us a list of messages of all action types returned for that query.

In order to display this in the UI, we need to call this method in the View Controller to actually use the result. Here is how we call this method.

var params: [String : AnyObject] = [
 Client.WebsearchKeys.Query: inputTextView.text! as AnyObject,
 Client.ChatKeys.TimeZoneOffset: ControllerConstants.timeZone as AnyObject,
       Client.ChatKeys.Language: Locale.current.languageCode as AnyObject
]

if let location = locationManager.location {
 params[Client.ChatKeys.Latitude] = location.coordinate.latitude as AnyObject
       params[Client.ChatKeys.Longitude] = location.coordinate.longitude as AnyObject
}

if let userData = UserDefaults.standard.dictionary(forKey: ControllerConstants.UserDefaultsKeys.user) as [String : AnyObject]? {
 let user = User(dictionary: userData)
        params[Client.ChatKeys.AccessToken] = user.accessToken as AnyObject
       }

Client.sharedInstance.queryResponse(params) { (messages, success, _) in
 DispatchQueue.main.async {
 if success {
 self.collectionView?.performBatchUpdates({
                            for message in messages! {
                                try! self.realm.write {
                                    self.realm.add(message)
                                    self.messages.append(message)
                                    let indexPath = IndexPath(item: self.messages.count - 1, section: 0)
                                    self.collectionView?.insertItems(at: [indexPath])
                                }
                            }
                        }, completion: { (_) in
                            self.scrollToLast()
                     })
 }
 }
}

Here, we are creating a params object sending the query and some additional parameters such as time zone, location coordinates and access token identifying the user. After the response is received, which contains a list of messages, we use a method called `performBatchUpdates` on the collection view where we loop through all the messages, writing each one of them to the database and then adding at the end of the collection view. Here we got all the messages inside the `messages` list object where each message can be checked for its action type and a frame can be calculated for the same.

Since the frame for each cell is returned in the `sizeForItemAt` delegate method of the collectionViewDelegate, we first grab the message using its indexPath and check the action type for each such message added to the collection view.

if message.actionType == ActionType.map.rawValue {
   // map action
} else if message.actionType == ActionType.rss.rawValue {
   // rss action type
} else if message.actionType == ActionType.websearch.rawValue {
   // web search action
}

Since map action will be a static map, we use a hard coded value for the map’s height and the width equals to the width of the cell frame.

CGSize(width: view.frame.width, height: Constants.mapActionHeight)

Next, web search and rss action having the same UI, will have the same frame size but the number of cells inside each of the frame for these action depends on number of responses were received from the server.

CGSize(width: view.frame.width, height: Constants.rssActionHeight)

And the check can be condensed as well, instead of checking each action separately, we use a `||` (pipes) or an `OR`.

else if message.actionType == ActionType.rss.rawValue ||
            message.actionType == ActionType.websearch.rawValue {
           // web search and rss action
 }

The anchor and answer action types, are supposed to display a string in the chat bubble. So the chat bubble size can be calculated using the following method:

let size = CGSize(width: Constants.maxCellWidth, height: Constants.maxCellHeight)
let options = NSStringDrawingOptions.usesFontLeading.union(.usesLineFragmentOrigin)
let estimatedFrame = NSString(string: messageBody).boundingRect(with: size, options: options, attributes: [NSFontAttributeName: UIFont.systemFont(ofSize: Constants.messageFontSize)], context: nil)

Here, we first create a maximum frame size that can exist. Then, using the drawingOptions create an options variable. The actual calculation of the frame happens in the last method where we use the complete message string and this returns us the actual frame for the above action types. Use the above method to get the frame in the `CGRect` type.

Below, is the complete method used for this calculation:

func collectionView(_ collectionView: UICollectionView, layout collectionViewLayout: UICollectionViewLayout, sizeForItemAt indexPath: IndexPath) -> CGSize {
        let message = messages[indexPath.row]
 
        let estimatedFrame = self.estimatedFrame(messageBody: message.message)
        if message.actionType == ActionType.map.rawValue {
            return CGSize(width: view.frame.width, height: Constants.mapActionHeight)
        } else if message.actionType == ActionType.rss.rawValue ||
            message.actionType == ActionType.websearch.rawValue {
            return CGSize(width: view.frame.width, height: Constants.rssActionType)
        }
        return CGSize(width: view.frame.width, height: estimatedFrame.height + Constants.answerActionMargin)
    }

rn CGSize(width: view.frame.width, height: estimatedFrame.height + Constants.answerActionMargin)
}

Below are the results of the app in action.

References:

Continue ReadingCalculation of the Frame Size of the Chat Bubble in SUSI iOS

Adding Snackbar to undo recent change in theme of SUSI.AI Web Chat

SUSI.AI Web Chat has two primary themes: Light and Dark. The user can switch between these two from settings, but what if the user does not prefer the theme change. He/She has to repeat the entire process of Going to Settings -> Selecting another theme -> saving it. To enable the user to instantly change the theme, we decided to add snackbar on theme change.

What is Snackbar ?

Snackbar contains info regarding the operation just performed and it displays them in a visual format. Snackbar can also have a button like “Undo” to revert the recent operation just made.

It appears at the bottom of screen by default. A simple snackbar looks like this:

Now we needed this to pop-up, whenever the theme is changed. When a user changes theme we run a function named “implementSettings” which checks what the theme is changed to.

The method is:

implementSettings = (values) => {
    this.setState({showSettings: false});
    if(values.theme!==this.state.currTheme){
      this.setState({SnackbarOpen: true});
    }
    this.changeTheme(values.theme);
    this.changeEnterAsSend(values.enterAsSend);
    setTimeout(() => {
       this.setState({
           SnackbarOpen: false
       });
   }, 2500);
  }

The argument values is an object that contains all the change that user has made in settings. Here values.theme contains the value of theme user selected from settings. We first check if the theme is same as the current one if so, we do nothing. If the theme is different from current, then we update the theme by calling this.changeTheme(values.theme) and also initiate snackbar by setting SnackbarOpen to open.

The snackbar component looks like:

<Snackbar
     open={this.state.SnackbarOpen}
     message={'Theme Changed'}
     action="undo"
     autoHideDuration={4000}
     onActionTouchTap={this.handleActionTouchTap}
     onRequestClose={this.handleRequestClose}
/>

This appears on the screen as follows :

Now if a user wants to change the theme instantly he/she can press the undo button. For this undo button, we have a method handleActionTouchTap. In this method, we change the theme back to previous one. The method is implemented in the following manner:

handleActionTouchTap = () => {
    this.setState({
      SnackbarOpen: false,
    });
    switch(this.state.currTheme){
      case 'light': {
          this.changeTheme('dark');
          break;
      }
      case 'dark': {
          this.changeTheme('light');
          break;
      }
      default: {
          // do nothing
      }
    }
  };

The above method is pretty self-explanatory which checks the current theme(“light” or “dark”) and then revert it. We also change the state of SnackbarOpen to “false” so that on clicking “UNDO” button, the theme changes back to previous one and the snackbar closes.Now user is having an option of instantly changing back to previous theme.

Resources:

Testing Link:

http://chat.susi.ai/

Continue ReadingAdding Snackbar to undo recent change in theme of SUSI.AI Web Chat

How SUSI WebChat Implements RSS Action Type

SUSI.AI now has a new action type called RSS. As the name suggests, SUSI is now capable of searching the internet to answer user queries. This web search can be performed either on the client side or the server side. When the web search is to be performed on the client side, it is denoted by websearch action type. When the web search is performed by the server itself, it is denoted by rss action type. The server searches the internet and using RSS feeds, returns an array of objects containing :

  • Title
  • Description
  • Link
  • Count

Each object is displayed as a result tile and all the results are rendered as swipeable tiles.

Lets visit SUSI WebChat and try it out.

Query : Google
Response: API response

SUSI WebChat uses the same code abstraction to render websearch and rss results as both are results of websearch, only difference being where the search is being performed i.e client side or server side.

How does the client know that it is a rss action type response?

"actions": [
  {
    "type": "answer",
    "expression": "I found this on the web:"
  },
  {
    "type": "rss",
    "title": "title",
    "description": "description",
    "link": "link",
    "count": 3
  }
],

The actions attribute in the JSON API response has information about the action type and the keys to be parsed for title, link and description.

  • The type attribute tells the action type is rss.
  • The title attribute tells that title for each result is under the key – title for each object in answers[0].data.
  • Similarly keys to be parsed for description and link are description and link respectively.
  • The count attribute tells the client how many results to display.

We then loop through the objects in answers,data[0] and from each object we extract title, description and link.

let rssKeys = Object.assign({}, data.answers[0].actions[index]);

delete rssKeys.type;

let count = -1;

if(rssKeys.hasOwnProperty('count')){
  count = rssKeys.count;
  delete rssKeys.count;
}

let rssTiles = getRSSTiles(rssKeys,data.answers[0].data,count);

We use the count attribute and the length of answers[0].data to fix the number of results to be displayed.

// Fetch RSS data

export function getRSSTiles(rssKeys,rssData,count){

  let parseKeys = Object.keys(rssKeys);
  let rssTiles = [];
  let tilesLimit = rssData.length;

  if(count > -1){
    tilesLimit = Math.min(count,rssData.length);
  }

  for(var i=0; i<tilesLimit; i++){
    let respData = rssData[i];
    let tileData = {};

    parseKeys.forEach((rssKey,j)=>{
      tileData[rssKey] = respData[rssKeys[rssKey]];
    });

    rssTiles.push(tileData);
  }

return rssTiles;

}

We now have our list of objects with the information parsed from the response.We then pass this list to our renderTiles function where each object in the rssTiles array returned from getRSSTiles function is converted into a Paper tile with the title and description and the entire tile is hyperlinked to the given link using Material UI Paper Component and few CSS attributes.

// Draw Tiles for Websearch RSS data

export function drawTiles(tilesData){

let resultTiles = tilesData.map((tile,i) => {

  return(
    <div key={i}>
      <MuiThemeProvider>
        <Paper zDepth={0} className='tile'>
          <a rel='noopener noreferrer'
          href={tile.link} target='_blank'
          className='tile-anchor'>
            {tile.icon &&
            (<div className='tile-img-container'>
               <img src={tile.icon}
               className='tile-img' alt=''/>
             </div>
            )}
            <div className='tile-text'>
              <p className='tile-title'>
                <strong>
                  {processText(tile.title,'websearch-rss')}
                </strong>
              </p>
              {processText(tile.description,'websearch-rss')}
            </div>
          </a>
        </Paper>
      </MuiThemeProvider>
    </div>
  );

});

return resultTiles;
}

The tile title and description is processed for HTML special entities and emojis too using the processText function.

case 'websearch-rss':{

let htmlText = entities.decode(text);
processedText = <Emojify>{htmlText}</Emojify>;
break;

}

We now display our result tiles as a carousel like swipeable display using react-slick. We initialise our slider with few default options specifying the swipe speed and the slider UI.

import Slider from 'react-slick';

// Render Websearch RSS tiles

export function renderTiles(tiles){

  if(tiles.length === 0){
    let noResultFound = 'NO Results Found';
    return(<center>{noResultFound}</center>);
  }

  let resultTiles = drawTiles(tiles);
  
  var settings = {
    speed: 500,
    slidesToShow: 3,
    slidesToScroll: 1,
    swipeToSlide:true,
    swipe:true,
    arrows:false
  };

  return(
    <Slider {...settings}>
      {resultTiles}
    </Slider>
  );
}

We finally add CSS attributes to style our result tile and add overflow for text maintaining standard width for all tiles.We also add some CSS for our carousel display to show multiple tiles instead of one by default. This is done by adding some margin for child components in the slider.

.slick-slide{
  margin: 0 10px;
}

.slick-list{
  max-height: 100px;
}

We finally have our swipeable display of rss data tiles each tile hyperlinked to the source of the data. When the user clicks on a tile, he is redirected to the link in a new window i.e the entire tile is hyperlinked. And when there are no results to display, we show a `NO Results Found` message.

The complete code can be found at SUSI WebChat Repo. Feel free to contribute

Resources

 

Continue ReadingHow SUSI WebChat Implements RSS Action Type

Serialisation, Deserialisation and gson in Open Event Android

JSON is a format to exchange and inter-change data. It has almost become a universal means for transferring data because of it being lightweight, human understandable, machine readable and most importantly, it’s language independence. It can be used with, and in any programming language. (Reference)

FOSSASIA’s Open Event project makes extensive use of JSON for transferring information about events, their speakers, sessions and other event information. The Open Event Android application itself loads all it’s data in JSON format. This JSON is uploaded by the user in the form of a .zip compressed file or by giving an API link. Now before we use this data in the app, we have to parse the data to get Java objects that can be used in Android. Deserialisation is the process of converting JSON to Java objects. In the same way, Serialisation refers to converting Java objects into JSON.

In most applications and frameworks, JSON is serialized and deserialized on multiple instances. The most common approach is to create objects corresponding to the JSON format and then use functions to convert them to JSON line-by-line, attribute by attribute. While this approach will work, it will mean writing unnecessary code and spending a lot more time on it. In Open-event-Android, we are using Google’s gson library to  serialise and deserialise JSON and corresponding Java objects respectively.

Why are we using gson specifically? And why do we need any library in the first place?

Yes, we can obviously make functions in Java by defining all the JSON parameters and converting the JSON into Java objects manually. It is indeed the obvious approach, but the entire process will be time-consuming, complex and definitely not error-free. Also, as projects become bigger, it is inevitable to take care of the code size and reduce it to only what is necessary.

Let’s take a look at the one of the Open Event JSON file as a sample to try and understand things in a better way. This is an example of event.json in the FBF8’2017 sample.

{

"id": 180,
"name": "F8-Facebook Developer Conference 2017",
"latitude": 37.329008,
"longitude": -121.888794,
"location_name": "San Jose Convention Center, 150 W San Carlos St  San Jose  CA USA 95113 ",
"start_time": "2017-04-18T10:00:00-07:00",
"end_time": "2017-04-19T10:00:00-07:00",
"timezone": "US / Pacific",
"description": "Join us for our annual 2-day event, where developers and businesses come together to explore the future of technology. Learn how Facebook connects the world through new products and innovation. This year's event is bigger than ever – with more than 50 sessions, interactive experiences, and the opportunity to meet one-on-one with members of the Facebook team.",
"background_image": "/images/background.jpg",
"logo": "/images/fbf8.png",
"organizer_name": "Facebook",
"organizer_description": "Join us for our annual 2-day event, where developers and businesses come together to explore the future of technology. Learn how Facebook connects the world through new products and innovation. This year's event is bigger than ever – with more than 50 sessions, interactive experiences, and the opportunity to meet one-on-one with members of the Facebook team.",
"event_url": "https://www.fbf8.com",
"social_links": [
{"id": 1,
"link": "https://www.facebook.com/FacebookForDevelopers",
"name": "Facebook"},
{"id": 2,
"link": "https://twitter.com/fbplatform",
"name": "Twitter"},
{"id": 3,
"link": "https://www.instagram.com/explore/tags/fbf8/",
"name": "Instagram"},
{"id": 4,
"link": "https://github.com/Facebook",
"name": "GitHub"}],
"ticket_url": null,
"privacy": "public",
"type": "",
"topic": "Science & Technology",
"sub_topic": "High Tech",
"code_of_conduct": "",
"copyright": {
"logo": "",
"licence_url": "https://en.wikipedia.org/wiki/All_rights_reserved",
"holder": "Facebook",
"licence": "All rights reserved",
"holder_url": null,
"year": 2017},
"call_for_papers": null,
"email": null,
"has_session_speakers": false,
"identifier": "7d16c124",
"large": null,
"licence_details": {
"logo": "",
"compact_logo": "",
"name": "All rights reserved",
"url": "https://en.wikipedia.org/wiki/All_rights_reserved",
"long_name": "All rights reserved",
"description": "The copyright holder reserves, or holds for their own use, all the rights provided by copyright law under one specific copyright treaty."},
"placeholder_url": null,
"schedule_published_on": null,
"searchable_location_name": null,
"state": "Published",
"version": {
"event_ver": 0,
"microlocations_ver": 0,
"sessions_ver": 0,
"speakers_ver": 0,
"sponsors_ver": 0,
"tracks_ver": 0}}

For a file with so many attributes, it can be very tedious to make a parser class in Java. Instead, we simply use the gson library to do the job.

To install gson in maven central, add the following dependency:

<dependencies>
   <!--  Gson: Java to Json conversion -->
   <dependency>
     <groupId>com.google.code.gson</groupId>
     <artifactId>gson</artifactId>
     <version>2.8.0</version>
     <scope>compile</scope>
   </dependency>
</dependencies>                                             

[source:https://github.com/google/gson/blob/master/UserGuide.md]

To use gson in Android, we need to add the following dependency in build.gradle of the project.

compile ‘com.google.code.gson:gson:2.6.2’

Using gson in your code is as simple as using any built-in class in java. There are two main functions – toJson()and fromJson() that will be used throughout the processes of serialisation and deserialisation.

First off, let’s declare a basic Java object. I will take an example of a Java object that represents a car to simplify things a bit. This object car contains 3 variables- Name, Model number and color of the car. For example:

class Car(){
String  name;
int model_no;
String colour;
Car(){}
}

Serialisation:

Serialising the car object now, that is converting the object into JSON format:

Car one = new Car();
Gson gson = new GSON();
String serialised_output= gson.toJson(car);//serialised_output=>JSON

Deserialisation:

Deserializing the JSON now, that is converting the JSON into java objects:

Car car = gson.fromJson(serialised_output,Car.class);

Finally, you can find the GitHub link for gson library here in case you want to know more about it or if you are facing some issues with the library, you can report them there. This is the link to the official Google group for discussions related to gson.

Continue ReadingSerialisation, Deserialisation and gson in Open Event Android