Automatic Signing and Publishing of Android Apps from Travis

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login using travis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tar fastlane.json key.jks

 

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

 

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

#!/bin/sh
set -e

export DEPLOY_BRANCH=${DEPLOY_BRANCH:-master}

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_REPO_SLUG" != "iamareebjamal/android-test-fastlane" -o "$TRAVIS_BRANCH" != "$DEPLOY_BRANCH" ]; then
    echo "We decrypt key only for pushes to the master branch and not PRs. So, skip."
    exit 0
fi

openssl aes-256-cbc -K $encrypted_4dd7_key -iv $encrypted_4dd7_iv -in ./scripts/secrets.tar.enc -out ./scripts/secrets.tar -d
tar xvf ./scripts/secrets.tar -C scripts/

 

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

cp app-release-unsigned.apk app-release-unaligned.apk
jarsigner -verbose -tsa http://timestamp.comodoca.com/rfc3161 -sigalg SHA1withRSA -digestalg SHA1 -keystore ../scripts/key.jks -storepass $STORE_PASS -keypass $KEY_PASS app-release-unaligned.apk $ALIAS

 

Then run this command to zipalign the app

${ANDROID_HOME}/build-tools/25.0.2/zipalign -v -p 4 app-release-unaligned.apk app-release.apk

 

Remember that the build tools version should be the same as the one specified in .travis.yml

This will create an apk named app-release.apk

Publishing App

This is the easiest step. First install fastlane using this command

gem install fastlane

 

Then run this command to publish the app to alpha channel on Play Store

fastlane supply --apk app-release.apk --track alpha --json_key ../scripts/fastlane.json --package_name com.iamareebjamal.fastlane

 

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

Continue ReadingAutomatic Signing and Publishing of Android Apps from Travis

Using Universal Image Loader to Display Image on Phimpme Android Application

In Phimpme Android application we needed to load image on the sharing Activity fast so that there won’t be any delay that is visible by a user in the loading of any activity. We used Universal Image Loader to load the image on the sharing Activity to load Image faster.

Getting Universal Image Loader

To get Universal Image Loader in your application go to Gradle(app)-> and then add the following line of code inside dependencies:

dependencies{

compile 'com.nostra13.universalimageloader:universal-image-loader:1.9.4'

}

Initialising Universal Image Loader and Displaying Image

To display image on using Universal Image Loader we need to convert the image into a URI from a file path:

saveFilePath = getIntent().getStringExtra(EXTRA_OUTPUT);
Uri uri = Uri.fromFile(new File(saveFilePath));

How an image should be displayed

We need to display the image in such a way that it covers the whole image view in the sharing Activity. The image should be zoomed out. The quality of the image should not be distorted or reduced. The image should look as it is. The image should be zoomable so that the user can pinch to zoom in and zoom out. For the image to adjust the whole Image View we set ImageScaleType.EXACTLY_STRETCHED. We will also set cacheInMemory to true and cacheOnDisc to true.  

private void initView() {
   saveFilePath = getIntent().getStringExtra(EXTRA_OUTPUT);
   Uri uri = Uri.fromFile(new File(saveFilePath));
   ImageLoader imageLoader = ((MyApplication)getApplicationContext()).getImageLoader();
   DisplayImageOptions options = new DisplayImageOptions.Builder()
           .cacheOnDisc(true)
           .imageScaleType(ImageScaleType.EXACTLY_STRETCHED)
           .cacheInMemory(true)
           .bitmapConfig(Bitmap.Config.RGB_565)
           .build();
   imageLoader.displayImage(uri.toString(), shareImage, options);
}

Image Loader function in MyApplication class:

private void initImageLoader() {
   File cacheDir = com.nostra13.universalimageloader.utils.StorageUtils.getCacheDirectory(this);
   int MAXMEMONRY = (int) (Runtime.getRuntime().maxMemory());
   // System.out.println("dsa-->"+MAXMEMONRY+"   "+(MAXMEMONRY/5));//.memoryCache(new
   // LruMemoryCache(50 * 1024 * 1024))
   DisplayImageOptions defaultOptions = new DisplayImageOptions.Builder()
           .cacheInMemory(true)
           .cacheOnDisk(true)
           .build();

   ImageLoaderConfiguration config = new ImageLoaderConfiguration.Builder(
           this).memoryCacheExtraOptions(480, 800).defaultDisplayImageOptions(defaultOptions)
           .diskCacheExtraOptions(480, 800, null).threadPoolSize(3)
           .threadPriority(Thread.NORM_PRIORITY - 2)
           .tasksProcessingOrder(QueueProcessingType.FIFO)
           .denyCacheImageMultipleSizesInMemory()
           .memoryCache(new LruMemoryCache(MAXMEMONRY / 5))
           .diskCache(new UnlimitedDiskCache(cacheDir))
           .diskCacheFileNameGenerator(new HashCodeFileNameGenerator()) // default
           .imageDownloader(new BaseImageDownloader(this)) // default
           .imageDecoder(new BaseImageDecoder(false)) // default
           .defaultDisplayImageOptions(DisplayImageOptions.createSimple()).build();

   this.imageLoader = ImageLoader.getInstance();
   imageLoader.init(config);
}

Image View in Sharing Activity XML file:

In the Sharing Activity Xml resource, we need to specify the width of the image view and the height of the image view. In Phimpme Android application we are using ImageViewTouch so that we have features like touch to zoom in zoom out. The scale type of the imageView is centerCrop so that image which is loaded is zoomed out and focus is in the center of the image.  

<org.fossasia.phimpme.editor.view.imagezoom.ImageViewTouch
   android:layout_width="match_parent"
   android:layout_height="wrap_content"
   android:id="@+id/share_image"
   android:layout_below="@+id/toolbar"
   android:layout_weight="10"
   android:layout_alignParentStart="true"
   android:scaleType="centerCrop"/>

Conclusion

To load image faster on any ImageView we should use Universal Image Loader. It helps load the activity faster and allows many features as discussed in the blog.

 

Github

Resources

Continue ReadingUsing Universal Image Loader to Display Image on Phimpme Android Application

Auto Updating SUSI Android APK and App Preview on appetize.io

This blog will cover the way in which the SUSI Android APK is build automatically after each commit and pushed to “apk” branch in the github repo. Other thing which will be covered is that how the app preview on appetize.io can be updated after each commit. This is basically for the testers who wish to test the SUSI Android App. There are four ways to test the SUSI Android App. One is to simply download the alpha version of the app from the Google PlayStore. Here is the link to the app. Join the alpha testing and report bugs on the github issue tracker of the repo. Other way is to build the app from Android Studio but you may need to set the complete project. If you are looking to contribute in the project, this is the advised way to test the app. The other two ways are explained below.

Auto Building of APK and pushing to “apk” branch

We have written a script which does following steps whenever a PR is merged:

  1. Checks if the commit is of a PR or a commit to repo
  2. If not of PR, configures a user whose github account will be used to push the APKs.
  3. Clones the repo, generates the debug and release APK.
  4. Deletes everything in the apk branch.
  5. Commits and Pushes new changes to apk branch.

This script is written for people or testers who do not have android studio installed in their computer and want to test the app. So, they can directly download the apk from the apk branch and install it in their phone. The APK is always updated after each commit. So, whenever a tester downloads the APK from apk branch, he will always get the latest app.

if [[ $CIRCLE_BRANCH != pull* ]]
then
    git config --global user.name "USERNAME"
    git config --global user.email "EMAIL"

    git clone --quiet --branch=apk https://USERNAME:$GITHUB_API_KEY@github.com/fossasia/susi_android apk > /dev/null
    ls
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-debug.apk apk/susi-debug.apk
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-release-unsigned.apk apk/susi-release.apk
    cd apk

    git checkout --orphan workaround
    git add -A

    git commit -am "[Circle CI] Update Susi Apk"

    git branch -D apk
    git branch -m apk

    git push origin apk --force --quiet > /dev/null
fi

Auto Updating of App Preview on appetize.io

The APKs generated in the above step can now be used to set up the preview of the app on the appetize.io. Appetize.io is an online simulator to run mobile apps ( IOS and Android). Appetize.io provides a nice virtual mobile frame to run native apps with various options like screen size, mobile, OS version, etc. Appetize.io provides some API to update/publish the app. In SUSI, we once uploaded the app on appetize.io and now we are using the API provided by them to update the APK everytime a commit is pushed in the repository.

API information (Derived from official docs of appetize.io):

You may upload a new version of an existing app, or update app settings.

Send an HTTP POST request to

https://APITOKEN@api.appetize.io/v1/apps/PUBLICKEY

Replace APITOKEN with your API token and PUBLICKEY with the public key of the app you’re updating. Your API token must be permissioned to the same account as was used to upload the app. The POST body must be a JSON object. To delete a previously set field, use a value of null.

Optional Fields

  1. url: (string) a publicly accessible link to your .zip, .tar.gz, or .apk file, used to upload a new version of your app.
  2. note: (string) a note for your own purposes, will appear on your management dashboard.

For the url parameter, we have used https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk and note can be anything. We have used Update SUSI Preview.

curl https://$APPETIZE_API_TOKEN@api.appetize.io/v1/apps/mbpprq4xj92c119j7nxdhttjm0 -H 'Content-Type: application/json' -d '{"url":"https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk", "note": "Update SUSI Preview"}'

Summary

This blog covered about how to implement an automatic structure to generate APKs for testing and using that APK to build a preview on websites like appetize.io and then using the APIs provided by them to update the APK after each PR merge in the repo. Check out the resources below to learn more about the topic. So, if you are thinking of contributing to SUSI Android App, this may help you a little in testing the app. But if not, then you can also use the similar technique for your android app as well and ease the life of testers.

Resources

  1. Docs of appetize.io to learn more about the API https://appetize.io/docs
  2. Tutorial on using curl to make API requests https://curl.haxx.se/docs/httpscripting.html
  3. Tutorial on writing basic shell scripts https://ryanstutorials.net/bash-scripting-tutorial/
Continue ReadingAuto Updating SUSI Android APK and App Preview on appetize.io

Setting Loklak Server with SSL

Loklak Server is based on embedded Jetty Server which can work both with or without SSL encryption. Lately, there was need to setup Loklak Server with SSL. Though the need was satisfied by CloudFlare. Alternatively, there are 2 ways to set up Loklak Server with SSL. They are:-

1) Default Jetty Implementation

There is pre-existing implementation of Jetty libraries. The http mode can be set in configuration file. There are 4 modes on which Loklak Server can work: http mode, https mode, only https mode and redirect to https mode. Loklak Server listens to port 9000 when in http mode and to port 9443 when in https mode.

There is also a need of SSL certificate which is to be added in configuration file.

2) Getting SSL certificate with Kube-Lego on Kubernetes Deployment

I got to know about Kube-Lego by @niranjan94. It is implemented in Open-Event-Orga-Server. The approach is to use:

a) Nginx as ingress controller

For setting up Nginx ingress controller, a yml file is needed which downloads and configures the server.

The configurations for data requests and response are:

proxy-connect-timeout: "15"
 proxy-read-timeout: "600"
 proxy-send-imeout: "600"
 hsts-include-subdomains: "false"
 body-size: "64m"
 server-name-hash-bucket-size: "256"
 server-tokens: "false"

Nginx is configured to work on both http and https ports in service.yml

ports:
- port: 80
  name: http
- port: 443
  name: https

 

b) Kube-Lego for fetching SSL certificates from Let’s Encrypt

Kube-Lego was set up with default values in yml. It uses the host-name, email address and secretname of the deployment to validate url and fetch SSL certificate from Let’s Encrypt.

c) Setup configurations related to TLS and no-TLS connection

These configuration files mentions the path and service ports for Nginx Server through which requests are forwarded to backend Loklak Server. Here for no-TLS and TLS requests, the requests are directly forwarded to localhost at port 80 of Loklak Server container.

rules:
- host: staging.loklak.org
  http:
  paths:
  - path: /
    backend:
    serviceName: server
    servicePort: 80

For TLS requests, the secret name is also mentioned. Kube-Lego fetches host-name and secret-name from here for the certificate

tls:
- hosts:
- staging.loklak.org
  secretName: loklak-api-tls

d) Loklak Server, ElasticSearch and Mosquitto at backend

These containers work at backend. ElasticSearch and Mosquitto are only accessible to Loklak Server. Loklak Server can be accessed through Nginx server. Loklak Server is configured to work at http mode and is exposed at port 80.

ports:
- port: 80
  protocol: TCP
  targetPort: 80

To deploy the Loklak Server, all these are deployed in separate pods and they interact through service ports. To deploy, we use deploy script:

# For elasticsearch, accessible only to api-server
kubectl create -R -f ${path-to-config-file}/elasticsearch/

# For mqtt, accessible only to api-server
kubectl create -R -f ${path-to-config-file}/mosquitto/

# Start KubeLego deployment for TLS certificates
kubectl create -R -f ${path-to-config-file}/lego/
kubectl create -R -f ${path-to-config-file}/nginx/

# Create web namespace, this acts as bridge to Loklak Server
kubectl create -R -f ${path-to-config-file}/web/

# Create API server deployment and expose the services
kubectl create -R -f ${path-to-config-file}/api-server/

# Get the IP address of the deployment to be used
kubectl get services --namespace=nginx-ingress

References

Continue ReadingSetting Loklak Server with SSL

Creating Animations in GTK+ with Pycairo in SUSI Linux App

SUSI Linux has an assistant user interface to answer your queries. You may ask queries and SUSI answers interactively using a host of skills which range from entertainment, knowledge, media to science, technology and sports. While SUSI is finding the answer to the query, it makes sense to add some animations depicting the process. The UI for SUSI Linux app is wholly made using GTK+ 3 using PyGObject (PyGTK). Thus, I needed to find a way create animations in GTK+.

Animations in most frameworks are generally created using repetitive drawing after an interval which leads to the effect of an object’s movement. Thus, basic need was to find a way to draw a custom object in GTK+. Reading the documentation of the GTK+, I realized that this could be done with a GTKDrawingArea.

GTKDrawingArea defines an area on which application developers can do the drawing on their own. GTK itself does not provide Canvas and related object to draw the shapes. For that, we need to use Cairo graphics library. Cairo is an open source graphics library with support for multiple window systems. It can run on a variety of backends, though here, we are not concerned about them.

Cairo can be accessed in Python using Pycairo. Pycairo is a set of Python 2 & 3 bindings for the cairo graphics library. Resources for usage of Pycairo for animations with GTK+ 3 are very less thus I will try to explain it in this blog in detail. We will start by creating an Animator class extending the GTK DrawingArea class.

class Animator(Gtk.DrawingArea):
   def __init__(self, **properties):
       super().__init__(**properties)
       self.set_size_request(200, 80)
       self.connect("draw", self.do_drawing)
       GLib.timeout_add(50, self.tick)

   def tick(self):
       self.queue_draw()
       return True

   def do_drawing(self, widget, ctx):
       self.draw(ctx, self.get_allocated_width(), self.get_allocated_height())

   def draw(self, ctx, width, height):
       pass

In the above code stub, we created the Animator class extending the GTK.DrawingArea. Animator class is meant to be an abstract class for the other animators. We defined the size request of the area we want and connected the “draw” signal to the do_drawing method. On notable thing to note here is that, we have “draw” signal on GTK+ 3 while on GTK+2 we have “on_expose” signal. On the creation of the widget, draw signal is fired. In the handler do_drawing method, we receive widget and ctx. Here ctx is the Cairo context. We can perform our drawing with the help of the of Cairo context. We further call the draw method passing the context, width and height of the widget. On notable thing here is that, even though we requested for a size for the widget, allocated size might be different depending upon a number of factors. Thus, drawing must be done according to allocated area instead of the absolute area.

The draw is an abstract method. All the animators must override this method to implement custom drawing on the widget area. Lastly, we add a timeout based call to tick method. This is what drives the animation. This is done with the help of GLib.timeout_add(). Here the first argument is the time in milliseconds after which callback should be fired and second argument is the callback that should be fired. We are calling tick method in the class. It is required for the method to return True if successful for proper functioning. We call queue_draw method from within the tick method. queue_draw leads to invalidation of current area and again generates the draw signal.

Now that we know how the core of the animation will work, let us define some cool animations for the Listening phase of the application. We define the Listening Animator for the same.

class ListeningAnimator(Animator):
   def __init__(self, window, **properties):
       super().__init__(**properties)
       self.window = window
       self.tc = 0

   def draw(self, ctx, width, height):

       self.tc += 0.2
       self.tc %= 2 * math.pi

       for i in range(-4, 5):
           ctx.set_source_rgb(0.2, 0.5, 1)
           ctx.set_line_width(6)
           ctx.set_line_cap(cairo.LINE_CAP_ROUND)
           if i % 2 == 0:
               ctx.move_to(width / 2 + i * 10, height / 2 + 3 - 
                           8 * math.sin(self.tc + i))
               ctx.line_to(width / 2 + i * 10, height / 2 - 3 + 
                           8 * math.sin(self.tc + i))
           else:
               ctx.set_source_rgb(0.2, 0.7, 1)
               ctx.move_to(width / 2 + i * 10, height / 2 + 3 - 
                           8 * math.cos(self.tc - i))
               ctx.line_to(width / 2 + i * 10, height / 2 - 3 + 
                           8 * math.cos(self.tc - i))
               ctx.stroke()

In this we are drawing some lines with round cap to create an effect like below.

Since, the lines must move we are using trigonometric functions to create sinusoidal movement with some phase difference between adjacent lines. To set color of the brush, we use set_source_rgb method. We may then move the pointer to desired position, draw lines or other shapes. Then, we can either fill the shape or draw strokes using the relevant methods. The full list of methods can be accessed here in the official documentation.
After creating the widget, it can be easily added to the UI depending on the type of the container, generally by add method. You may access the full code of SUSI Linux repository to learn more about the usage in SUSI Linux App. The final result can be seen in the following video.

Resources:

Continue ReadingCreating Animations in GTK+ with Pycairo in SUSI Linux App

Showing location response in SUSI.AI bots

SUSI.AI has a capability to tell the basic information about a location, it is asked for. Along with the basic information about that place, it shows a map (i.e. open street map) pointing to that location. The task at hand is to inculcate this “location” feature to the SUSI.AI messenger bots. The SUSI Tweetbot and SUSI Fbbot are used as examples in this blog.

Let’s first check on what response do we get, when a person asks a query like “where is london” to the SUSI API. Along with the basic information about that location, the result as shown below has the type of reply (i.e. map), latitude, longitude and a link to the open street map.

"actions": [
      {
        "type": "answer",
        "language": "en",
        "expression": "Ludhiana is a city and a municipal corporation in Ludhiana district in the Indian state of Punjab, and is the largest city north of Delhi."
      },
      {
        "type": "anchor",
        "link": "https://www.openstreetmap.org/#map=13/30.912040570244187/75.85379021980509",
        "text": "Here is a map",
        "language": "en"
      },
      {
        "type": "map",
        "latitude": "30.912040570244187",
        "longitude": "75.85379021980509",
        "zoom": "13",
        "language": "en"
      }
    ]

The response for a location type query has these 3 main parts:

  1. Clickable static map image.
  2. A basic information of the place asked about.
  3. The link i.e. when the static map image is clicked it should redirect to the corresponding map location.

Let’s try to make up with the 1st part of the response i.e. Static map image.

The map quest API is used to result in a static map image of the location. We need an API key to access the map quest API, which can be requested from their developer site.

Along with that we need the latitude and longitude of the location at hand.

The SUSI API’s response helps us to get the latitude value:

// if body represents the response object 
var lat = body.answers[0].actions[2].latitude;

And the longitude value:

var lon = body.answers[0].actions[2].longitude;

Using the three values that are API key, latitude and longitude, the static image is rendered by this link:

var static_image_url = "https://open.mapquestapi.com/staticmap/v4/getmap?key=API_KEY&size=600,400&zoom=13&center="+lat+","+lon;

The second part is, basic information about the place asked, can be fetched from:

// if body is the JSON response object from SUSI API
var mapMessage = body.answers[0].actions[0].expression;

The link to the map location can be easily fetched from the SUSI API’s response:

var link = body.answers[0].actions[1].link;

As all the three parts are ready, let’s look on how to render them on the SUSI.AI bot’s screen.

Facebook:

Sending a generic template message object:

message = {
        "type":"template",
        "payload":{
                    "template_type":"generic",
                    "elements":[{
                        "title": mapMessage,
                       "image_url": static_image_url,
                       "Item_url": link
                    }]
        }
};

sendTextMessage(sender, message, 1);

Twitter:

The Twitter API does not need a static image of the map to be rendered. It does that work for us. We just need to send an event to the Twitter API with the message data object constituting of the map message, the latitude and longitude values:

"message_data": {
            "text": mapMessage,
            "attachment": {
                "type": "location",
                "location": {
                    "type": "shared_coordinate",
                    "shared_coordinate": {
                        "coordinates": {
                            "type": "Point",
                            "coordinates": [lon, lat]
                        }
                    }
                }
            }
}

Resources:

  1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
  2. Drive discovery of bots and other customer experiences in direct messages by By Travis Lull from Twitter blog.
  3. By Seth Rosenberg from Facebook developers blogLink Ads to Messenger, Enhanced Mobile Websites, Payments and More.
Continue ReadingShowing location response in SUSI.AI bots

Adding Tweet Streaming Feature in World Mood Tracker loklak App

The World Mood Tracker was added to loklak apps with the feature to display aggregated data from the emotion classifier of loklak server. The next step in the app was adding the feature to display the stream of Tweets from a country as they are discovered by loklak. With the addition of stream servlet in loklak, it was possible to utilise it in this app.

In this blog post, I will be discussing the steps taken while adding to introduce this feature in World Mood Tracker app.

Props for WorldMap component

The WorldMap component holds the view for the map displayed in the app. This is where API calls to classifier endpoint are made and results are displayed on the map. In order to display tweets on clicking a country, we need to define react props so that methods from higher level components can be called.

In order to enable props, we need to change the constructor for the component –

export default class WorldMap extends React.Component {
    constructor(props) {
        super(props);
        ...
    }
    ...
}

[SOURCE]

We can now pass the method from parent component to enable streaming and other components can close the stream by using props in them –

export default class WorldMoodTracker extends React.Component {
    ...
    showStream(countryName, countryCode) {
        /* Do something to enable streaming component */
        ...
    }
 
    render() {
        return (
             ...
                <WorldMap showStream={this.showStream}/>
             ...
        )
    }
}

[SOURCE]

Defining Actions on Clicking Country Map

As mentioned in an earlier blog post, World Mood Tracker uses Datamaps to visualize data on a map. In order to trigger a piece of code on clicking a country, we can use the “done” method of the Datamaps instance. This is where we use the props passed earlier –

done: function(datamap) {
    datamap.svg.selectAll('.datamaps-subunit').on('click', function (geography) {
        props.showStream(geography.properties.name, reverseCountryCode(geography.id));
    })
}

[SOURCE]

The name and ID for the country will be used to display name and make API call to stream endpoint respectively.

The StreamOverlay Component

The StreamOverlay components hold all the utilities to display the stream of Tweets from loklak. This component is used from its parent components whose state holds info about displaying this component –

export default class WorldMoodTracker extends React.Component {
    ...
    getStreamOverlay() {
        if (this.state.enabled) {
            return (<StreamOverlay
                show={true} channel={this.state.channel}
                country={this.state.country} onClose={this.onOverlayClose}/>);
        }
    }

    render() {
        return (
            ...
                {this.getStreamOverlay()}
            ...
        )
    }
}

[SOURCE]

The corresponding props passed are used to render the component and connect to the stream from loklak server.

Creating Overlay Modal

On clicking the map, an overlay is shown. To display this overlay, react-overlays is used. The Modal component offered by the packages provides a very simple interface to define the design and interface of the component, including style, onclose hook, etc.

import {Modal} from 'react-overlays';

<Modal aria-labelledby='modal-label'
    style={modalStyle}
    backdropStyle={backdropStyle}
    show={true}
    onHide={this.close}>
    <div style={dialogStyle()}>
        ...
    </div>
</Modal>

[SOURCE]

It must be noted that modalStyle and backdropStyle are React style objects.

Dialog Style

The dialog style is defined to provide some space at the top, clicking where, the overlay is closed. To do this, vertical height units are used –

const dialogStyle = function () {
    return {
        position: 'absolute',
        width: '100%',
        top: '5vh',
        height: '95vh',
        padding: 20
        ...
    };
};

[SOURCE]

Connecting to loklak Tweet Stream

loklak sends Server Sent Events to clients connected to it. To utilise this stream, we can use the natively supported EventSource object. Event stream is started with the render method of the StreamOverlay component –

render () {
    this.startEventSource(this.props.channel);
    ...
}

[SOURCE]

This channel is used to connect to twitter/country/<country-ID> channel on the stream and then this can be passed to EventStream constructor. On receiving a message, a list of Tweets is appended and later rendered in the view –

startEventSource(country) {
    let channel = 'twitter%2Fcountry%2F' + country;
    if (this.eventSource) {
        return;
    }
    this.eventSource = new EventSource(host + '/api/stream.json?channel=' + channel);
    this.eventSource.onmessage = (event) => {
        let json = JSON.parse(event.data);
        this.state.tweets.push(json);
        if (this.state.tweets.length > 250) {
            this.state.tweets.shift();
        }
        this.setState(this.state);
    };
}

[SOURCE]

The size of the list is restricted to 250 here, so when a newer Tweet comes in, the oldest one is chopped off. And thanks to fast DOM actions in React, the rendering doesn’t take much time.

Rendering Tweets

The Tweets are displayed as simple cards on which user can click to open it on Twitter in a new tab. It contains basic information about the Tweet – screen name and Tweet text. Images are not rendered as it would make no sense to load them when Tweets are coming at a high rate.

function getTweetHtml(json) {
    return (
        <div style={{padding: '5px', borderRadius: '3px', border: '1px solid black', margin: '10px'}}>
            <a href={json.link} target="_blank">
            <div style={{marginBottom: '5px'}}>
                <b>@{json['screen_name']}</b>
            </div>
            <div style={{overflowX: 'hidden'}}>{json['text']}</div>
            </a>
        </div>
    )
}

[SOURCE]

They are rendered using a simple map in the render method of StreamOverlay component –

<div className={styles.container} style={{'height': '100%', 'overflowY': 'auto',
    'overflowX': 'hidden', maxWidth: '100%'}}>
    {this.state.tweets.reverse().map(getTweetHtml)}
</div>

[SOURCE]

Closing Overlay

With the previous setup in place, we can now see Tweets from the loklak backend as they arrive. But the problem is that we will still be connected to the stream when we click-close the modal. Also, we would need to close the overlay from the parent component in order to stop rendering it.

We can use the onclose method for the Modal here –

close() {
    if (this.eventSource) {
        this.eventSource.close();
        this.eventSource = null;
    }
    this.props.onClose();
}

[SOURCE]

Here, props.onClose() disables rendering of StreamOverlay in the parent component.

Conclusion

In this blog post, I explained how the flow of props are used in the World Mood Tracker app to turn on and off the streaming in the overlay defined using react-overlays. This feature shows a basic setup for using the newly introduced stream API in loklak.

The motivation of such application was taken from emojitracker by mroth as mentioned in fossasia/labs.fossasia.org#136. The changes were proposed in fossasia/apps.loklak.org#315 by @singhpratyush (me).

The app can be accessed live at https://singhpratyush.github.io/world-mood-tracker/index.html.

Resources

Continue ReadingAdding Tweet Streaming Feature in World Mood Tracker loklak App

Loklak Timeline Using Sphinx Extension In Yaydoc

In Yaydoc, I decided to add option, to show show the twitter timeline which showthe latest twitter feed. But I wanted to implement it using loklak instead of twitter embedded plugin. I started to search for an embedded plugin that exists for loklak. There is no such plugin, hence I built my own plugin. You can see the source code here.

Now that I have the plugin, the next phase is to add the plugin to the documentation. Adding the plugin by appending the plugin code to HTML is not viable. Therefore I decided to make Directive for Sphinx which adds a timeline based on the query parameter which user provides.

In order to make a Directive, I had to make a Sphinx extension which creates a timeline Directive. The Directive has to look like this

.. timeline :: fossasia

from docutils import nodes

from docutils.parsers import rst

class timeline(nodes.General, nodes.Element):
  pass

def visit(self, node):
  tag=u'''

”’

.format(node.display_name)
  self.body.append(tag)
  self.visit_admonition(node)

def depart(self, node):
  self.depart_admonition(node)

class TimelineDirective(rst.Directive):
  name = 'timeline'
  node_class = timeline
  has_content = True
  required_argument = 1
  optional_argument = 0
  final_argument_whitespace = False
  option_spec = {}

 def run(self):
    node = self.node_class()
    node.display_name = self.content[0]
    return [node]

def setup(app):            app.add_javascript("https://cdn.rawgit.com/fossasia/loklak-timeline-plugin/master/plugi
 n.js")
  app.add_node(timeline, html=(visit, depart))
  app.add_directive('timeline', TimelineDirective)

We have to create an empty class for Nodes that inherits`Node.General` and `Node.Elements`. This class is used for storing the value which will be passed by the directive.

I wrote a `Visit` function which executes when sphinx visits the `timeline` directive. `Visit` function basically appends the necessary html code needed to render the twitter timeline. Then I created TimelineDirective class which inherits rst.Directive. In that class, I defined a run method which read the argument from the directive and passed it to the node. Finally I defined a setup method which adds the loklak-timeline-plugin js to the render html node, and directive to the sphinx. Setup function has to be defined, in order to detect module as an extension by the sphinx.

Resources:

Continue ReadingLoklak Timeline Using Sphinx Extension In Yaydoc

Updating of Linux Standard Base (lsb) and Shorten of log of Meilix Build

Updating the Linux Standard Base of the Meilix

Originally Meilix uses the Ubuntu mirror to fetch the Kernel source for the building of the Operating System. Therefore whenever a user type lsb_release -a for fetching the required information, they get the Ubuntu build info. The task is to change the config file to update the Linux Base Information from Ubuntu to Meilix.

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty

We need to patch a file in the location meilix-default-settings/etc/lsb-release which contains the information of the lsb release and this will overwrite the original configuration of Meilix.
This is how the lsb-release file looks like now:

DISTRIB_ID=Meilix
DISTRIB_RELEASE=17.04
DISTRIB_CODENAME=meilix
DISTRIB_DESCRIPTION="Meilix 17.04"

We made the required changes in the file as shown above and the output is as follows:

Shorten the log length of Meilix in Travis

We are facing issues while deployment of Meilix ISO in Travis and the error follows is ReadTimeout Error. Example log of one of fail build is:

This error gets solved automatically and the the ISO gets deployment after 1 or 2 restart of the build. But this time the error doesn’t get solved and after several attempts of restart of build, the ISO doesn’t get deployed.

Reason behind the error:

Travis is taking a lot of time to build the ISO. Travis logs are exceeding the time limit.

Proposed solution:

Reduce the time of build or shift to a new CI.
Reduce the log of the build so as to get the log within 9999 lines.

Solution Implemented

The best solution is to reduce the number of lines used in the log and this will also reduce the time of the build.
I tried concealing some command outputs by appending >/dev/null 2>&1 to some of the commands that has long outputs and adding -y to the commands like:

apt-get -qq -y --purge install file-roller unrar

References

Wiki of Linux Standard Base
Linux Foundation lsb
Ubuntu answer to reduce log

Continue ReadingUpdating of Linux Standard Base (lsb) and Shorten of log of Meilix Build

Turning off Power Management and Blanking of Screen in Meilix

Meililx has the LXQt desktop environment which already have the lxqt-powermanagement package. We don’t need to configure the power management during an event. Every time when an ISO is booted a pop-up arrives which asks to configure power. On the one hand, the pop-ups are a disturbing element during a presentation, slide-show, etc. On the other hand, while a presentation is going on and the presenter is explaining a slide for a longer duration then there is a possibility of the screen to turn blank or screen saver to get started due to inactivity. In this post we will discuss to resolve the above two issues.

Disabling Power Management
The issue has been filed #133 to solve. We can follow two approaches to shutdown the power daemon which is responsible for starting the power-management everytime during startup.

In the beginning the issue looks as:

One is to copy the script to /etc/X11/xorg.conf.d/10-monitor.conf by editing the meilix-default-settings metapackage. 

1. [panel1]
2. alignment=-1
3. animation-duration=0
4. background-color=@Variant(\0\0\0\x43\0\xff\xff\0\0\0\0\0\0\0\0)
5. background-image=
6. desktop=0
7. font-color=@Variant(\0\0\0\x43\0\xff\xff\0\0\0\0\0\0\0\0)
8. hidable=true
9. iconSize=22
10. lineCount=1
11. lockPanel=false
12. opacity=100
13.panelSize=32
14. plugins=mainmenu, desktopswitch, quicklaunch, taskbar, tray, statusnotifier, mount, volume, clock, showdesktop
15. position=Bottom
16. show-delay=0
17. width=100
18. width-percent=true

Explanation of code:

In the Monitor section, DPMS (Desktop Power Management System) is turned off by setting the value to false
In the ServerLayout section, Blanktime, Offtime, SuspendTime, StandbyTime are set to “0”, so that they will be disabled.
Through this way we are suspending the power-management to use it feature by making changes in the script. This approach will make sure that this feature can’t be changed through the GUI, too.

And the other approach is the basic one but it’s also fulfilling the similar need. Adding a line to chroot.sh

apt-get purge lxqt-powermanagement

Both will solve the issue and the pop will not arrive at the startup of the OS.
Now LXQt desktop starts as:

Disabling Blanking of Screen in Meilix
A presenter never wants to turn his monitor blank or screensaver while presenting his presentation. Here we will edit the .profile file in the home folder to fulfill our requirement.
For get the same file in the ISO, we need to patch the file meilix-default-settings/etc/skel/.profile.
Skel folder has the property to transfer the tree inside the folder to the home folder of the new user created. For example here, the new user created will have .profile folder inside its home directory and it will apply the requested changes.

Important code which is responsible for turning the blanking of screen off:

xset s off && xset -dpms

 

Link to important pages:
Disabling Power Management – Arch Wiki
Code for turning Power Management (PR) – @meets2tarun (author)
Code for turning off blanking of screen (PR) – @meets2tarun (author)

Continue ReadingTurning off Power Management and Blanking of Screen in Meilix