Visualising Data acquired by loklak Aggregation API

Data is always more interesting to look at when it is visualised by means of graphs and various plots rather than just viewing data as raw json or xml. There are various open source libraries available for visualising data. The one which I will be using today is called morris.js. Although it does not provide a lot of plots but it is extremely simple to use and can be of great help when quick data visualisation is required.

This blog shows one of the many ways in which data obtained from loklak service can be visualised using morris.js. This is same library which I am using presently to build MultiLinePlotter app whicch will soon be displayed on Loklak app store.

What is morris.js?

It is an open source library for drawing graphs using javascript. Presently it provides APIs to draw line charts, area charts, bar charts and donut charts. To know more about morris.js please visit this link.

Loklak aggregation service

A loklak search result can also contain field aggregations if they are requested in the api call. Fields to be aggregated must be listed in the fields attribute in the api request. The following url is an example for requesting aggregation on hashtags and mentions:

http://api.loklak.org/api/search.json?q=spacex%20since:2015-04-01%20until:2015-04-06&source=cache&count=0&fields=mentions,hashtags&limit=6

‘created_at’ is a special field which will return a date histogram if mentioned in the fields attribute. To know more about the loklak aggregation service and the loklak project in general please visit this link.

Getting started

An example of morris.js can be found here. Feel free to browse this code to see morris.js in working.

So what we will be doing today is plotting a simple bar chart using morris.js and the data returned by loklak aggregation service. For this we will be using the following api call :

https://api.loklak.org/api/search.json?callback=JSON_CALLBACK&q=fossasia%20since:2015-12-10&source=cache&count=0&fields=created_at

The above request will return an date histogram for the query fossasia, that is, date vs number of tweets containing the term ‘fossasia’.

So the first thing that we need to do is create a directory for this project and create a file named index.html in it. This is the only file we will be requiring.

Next we need to add morris.js and its dependencies:

<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.css">
    <script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
    <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/raphael/2.1.0/raphael-min.js"></script>
    <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.min.js"></script>

This will add the css and js script files for morris.js and its dependencies : Jquery and Raphael.
Next we need to add the div element that will contain our chart.

<div id="myfirstchart" style="height: 380px;margin-top:50px;overflow-x: auto;"></div>

The most important thing to note over here is that the div must have an id so that we can refer it later from our js code.

Next comes the most important and interesting part : adding the js code that will fetch the data and plot the bar chart.
The first thing that the script will do is make an ajax call to the above mentioned url and get the data.This is done using the following code :

$.ajax({
          url: "https://api.loklak.org/api/search.json?callback=JSON_CALLBACK&q=fossasia%20since:2015-12-10&source=cache&count=0&fields=created_at",
          jsonp: "callback",
          dataType: "jsonp",
          success: function( response ) {
              data=[];
              for (var date in response.aggregations.created_at) {
                  data.push({day: date, tweets: response.aggregations.created_at[date]});
              }
              console.log(data);
              plotData(data);
          }
      });

It is to be noted here that a callback function is provided in the url and data type is set to jsonp and not json.This is done to avoid cross origin resource sharing (CORS) issue so that our browser does not block the ajax request.
Once our request is successful we populate the data array from the response returned to us.
We iterate over all the key value pairs present in the created_at object and add them as individual objects in the data array. This will be required in the next part.

Next we have to plot the data acquired by us. For this we will be using plotData(data) function as written below:

function plotData(data) {
          Morris.Bar({
              element: 'myfirstchart',
            data: data,
            xkey: 'day',
       ykeys: ['tweets'],
            labels: ['tweets']
          });
      }

We send this function the data array we created earlier. Inside this function we create a Morris.Bar object which takes an object as parameter. The object has the following properties:

‘element’ : This takes the id of the div where we want to load the chart.

‘data’ : This is the data array we created earlier and passed to this function. It is the data to plot.It is an array of objects containing the x and y  attributes.

xkey’ : A string containing the name of the attribute that contains X labels.

‘ykeys’ : A list of strings containing names of attributes that contain Y values (one for each series of data to be plotted).

‘labels’ : A list of strings containing labels for the data series to be plotted (corresponding to the values in the ykeys option).

That’s it. These are the basic parameters required to plot a bar chart using morris.js. Simple enough, isn’t it!!

Here is the complete js code:

<script type="text/javascript">
      $.ajax({
          url: "https://api.loklak.org/api/search.json?callback=JSON_CALLBACK&q=fossasia%20since:2015-12-10&source=cache&count=0&fields=created_at",
          jsonp: "callback",
          dataType: "jsonp",
          success: function( response ) {
              data=[];
              for (var date in response.aggregations.created_at) {
                  data.push({day: date, tweets: response.aggregations.created_at[date]});
              }
              console.log(data);
              plotData(data);
          }
      });
      function plotData(data) {
          Morris.Bar({
              element: 'myfirstchart',
              data: data,
              xkey: 'day',
              ykeys: ['tweets'],
              labels: ['tweets']
          });
      }
    </script>

Next, open terminal and navigate to your project directory containing the index.html file, then execute the following command:

Python -m SimpleHTTPServer

As the command suggests, it starts a simple HTTP server at port 8000.

So next ,as you might have already guessed, fire up your favourite browser and visit http://127.0.0.1:8000 and you will be able to see the chart present below.

Python’s simple HTTP server comes in handy if you are developing a frontend app and you want to test it on real small screen devices. For example if you want to test it on your mobile then just connect your computer and your mobile to your router, start python http server on your computer, find out your computer’s ip address (ifconfig) and visit http://<computer’s ip address>:8000 on your mobile. You will be able to view your app on your mobile. If you don’t use a router then create_ap may help.

Important Resources

  • Find more information on morris.js here
  • Visit Loklak apps GitHub page here
  • You can find a similar implementation of the above tutorial here

Published by

Deepjyoti Mondal

A web and mobile application developer and an enthusiastic learner. I like trying out new technologies. JavaScript and Python are my favorites