loklak

The CountryTweetMap app is a web application which uses Loklak api and visualises loklak data on a map. The app is presently a part of Loklak apps site and can be used here. In this blog I have discussed in details about how I have developed CountryTweetMap. Before delving right into the development process let us know what the app does?

Related issue: https://github.com/fossasia/apps.loklak.org/issues/244

Overview

Loklak CoutryTweetMap uses a map to plot data returned by the Loklak api. The user needs to enter a query word in the input field and press search. The app uses Loklak API to determine all the tweets which contain the query words. Finally the various places from where tweets have been made containing the given query word are plotted on the map with markers of  specific color. Color can be red (highest number of tweets), blue (medium number of tweets) and green (less number of tweets). The three categories of markers are added as layers. That is, users can filter the markers. For example, the users can view only the green markers and hide the other markers if they want to know from which countries the number of tweets are low. Once the data is plotted there is also an option to plot distribution. It simply lists the returned data in a tabular form showing country v/s number of tweets.

Developing CountryTweetMap app

Now we know what CountryTweetMap does. Let us find out how it works. Like other loklak apps present on the app site, this app also uses the loklak API to fetch the data. Next it uses a framework called leaflet.js to plot the acquired data on map. ‘leaflet.js’ provides various APIs to deal with maps and corresponding layers. For date input, the app uses jquery UI, a library based on jquery which provides various ready to use components.

Let us iterate through each step involved in creating the app. At first we take input from user and get the data from Loklak server. This is done by a simple ajax call. However before making the ajax call and actually getting the data, we need to ensure certain things. Firstly, we need to make sure that the user has actually entered something in the query field, dates are in order, that is start date is less than end date and count is a number. All these checks are done in the search function itself.

$scope.error = "";
        if ($scope.tweet === undefined || $scope.tweet === "" || $scope.isLoading === true) {
            if ($scope.tweet === undefined || $scope.tweet === "") {
                $scope.error = "Please enter a valid query word."
            }
            if ($scope.isLoading === true) {
                $scope.error = "Previous search not completed. Please wait...";
            }
            $scope.showSnackbar();
            return;
        }

        var count = $(".count").val();
        if (count.length !== 0) {
            if (/^\d+$/.test(count) === false) {
                $scope.error = "Count should be a valid number.";
                $scope.showSnackbar();
                return;
            }
        }

        var sinceDate = $(".start-date").val();
        var endDate = $(".end-date").val();
        if ((sinceDate !== undefined && endDate !== "") && (endDate !== undefined && endDate !== "")) {
            var date1 = new Date(sinceDate);
            var date2 = new Date(endDate);
            if (endDate < sinceDate) {
                $scope.error = "End date should be larger than start date";
                $scope.showSnackbar();
                return;
            }
        }

We first check that the query field is not empty and whether the app is already processing some query or not. In either case we show a suitable message in a snackbar and return from the function. Next we validate the dates. For date comparison we use the inbuilt Date class already present in javascript. For validating the count field we use regex for identifying only unsigned numbers. Finally if everything is alright we fetch the the data making an ajax call to the loklak search service.

$scope.isLoading = true;
        var query = "q=" + $scope.tweet;

        if (sinceDate !== undefined && sinceDate !== "" ) {
            query += "%20since:" + sinceDate;
        }
        if (endDate !== undefined && endDate !== "") {
            query += "%20until:" + endDate;
        }

        // Change base url to api.loklak.org later
        var url = "http://35.184.151.104/api/search.json?callback=JSON_CALLBACK&" + query;
        var count = $(".count").val();
        if (count !== undefined && count !== "") {
            url  += "&count=" + count;
        }
        $http.jsonp(url)
            .then(function (response) {
                $scope.reset();
                $scope.prepareFreq(response.data.statuses);
                $scope.displayMap();
            });

Once we get the desired data from the ajax call, we prepare a country v/s number of tweets frequency and store it in a javascript object and then plot the map. For calculating the frequency we simply iterate over each status returned, insert each country code as key in the object, and corresponding statuses as value in a list. In this way we create a country v/s tweets distribution where against each country code we have a list of statuses.

data.forEach(function(status) {
            if (status.place_country_code !== undefined) {
                if ($scope.tweetFreq[status.place_country_code] === undefined) {
                    $scope.tweetFreq[status.place_country_code] = [];
                }
                $scope.tweetFreq[status.place_country_code].push(status);
            }
        });

Next we need a suitable point which will act as the center of the world map. We need to chose the center in such a way so that maximum number of markers are visible without any need to scroll. For this we calculate the average latitude and longitude and feed it to leaflet as the map center. Once we have all the data in place, we are ready to plot the map. But before that we need to create our markers. For each country we require a marker and the color of the marker is determined by the number of tweets associated with the corresponding country.
Next, we need to group all the markers of the same color into a single layer so that the user can add and remove all the markers at once.

for (var key in $scope.tweetFreq) {
            var country = $scope.tweetFreq[key][0];
            var size = $scope.tweetFreq[key].length;
            var markerIcon = null;
            var marker = null;
            if (size === $scope.tweetMax) {
                markerIcon = L.marker([country.place_country_center[1], country.place_country_center[0]], {icon: redIcon});
                marker = $scope.getMarker(markerIcon, country, key);
                countriesHigh.push(marker);
            } else if (size > rangeMid) {
                markerIcon = L.marker([country.place_country_center[1], country.place_country_center[0]], {icon: blueIcon});
                marker = $scope.getMarker(markerIcon, country, key);
                countriesMedium.push(marker)
            } else {
                markerIcon = L.marker([country.place_country_center[1], country.place_country_center[0]], {icon: greenIcon});
                marker = $scope.getMarker(markerIcon, country, key);
                countriesLow.push(marker);
            }
        }

The above code snippet classifies the markers into three different classes (identified by three colors) and groups them into layers. Finally we need to plot them on our map. This is extremely easy with the simple interface provided by leaflet.

var backgroundLight = L.tileLayer(
            'http://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}.png',
            {
                maxZoom: 18,
                minZoom: 1,
                noWrap: true
            });
        $scope.map = L.map('map', {
            center: [mapCenterLat, mapCenterLon],
            zoom: 2,
            'maxBounds': [
                [-90,-180],
                [90, 180]
            ],
            layers: [backgroundLight, countryHighGroup, countryMediumGroup, countryLowGroup]
        });
       var overlayMaps = {
            "Maximum tweets": countryHighGroup,
            "Medium tweets": countryMediumGroup,
            "Low tweets": countryLowGroup,
            "Heatmap": heat
        };
        var baseMaps = {
            "basemap": backgroundLight
        };
        L.control.layers(baseMaps, overlayMaps).addTo($scope.map);

What is happening in the above code snippet? First we create a title layer for our map. The title consists of the map tiles which we actually see. Here we are using basemaps from cartocdn.com. There are several other map layouts like openlayer and openstreets. Next we specify the zoom values for this layer and set nowrap to false. This means that the map tiles will not get repeated and we will not get multiple copies of the world.
Next we create our map, we set the map’s center, zoom and bounds. Bounds restrict the users from scrolling beyond a particular portion of the map. Finally we set our layers. We have four layers in our map, the background layer containing our map tiles and the three marker layers signifying high, medium and low number of tweets. We separate the layers into two categories, baseMaps and overlayMaps. After this we add these layers to our map and we are done.

Future roadmap

• Adding a heatmap overlay.
• Visualising country v/s tweet frequency using a suitable graph.

Important resources

• Visit Loklak apps site here.
• See Loklak documentation here.
• Find leaflet.js documentation here.
• Learn more about AngularJS on the Angular.org documentation.
• Learn more about Jquery UI here.