GSoC

Read more about the article Persistently Storing loklak Server Dumps on Kubernetes

Persistently Storing loklak Server Dumps on Kubernetes

In an earlier blog post, I discussed loklak setup on Kubernetes. The deployment mentioned in the post was to test the development branch. Next, we needed to have a deployment where all the messages are collected and dumped in text files that can be reused.

In this blog post, I will be discussing the challenges with such deployment and the approach to tackle them.

Volatile Disk in Kubernetes

The pods that hold deployments in Kubernetes have disk storage. Any data that gets written by the application stays only until the same version of deployment is running. As soon as the deployment is updated/relocated, the data stored during the application is cleaned up.


Due to this, dumps are written when loklak is running but they get wiped out when the deployment image is updated. In other words, all dumps are lost when the image updates. We needed to find a solution to this as we needed a permanent storage when collecting dumps.

Persistent Disk

In order to have a storage which can hold data permanently, we can mount persistent disk(s) on a pod at the appropriate location. This ensures that the data that is important to us stays with us, even
when the deployment goes down.


In order to add persistent disks, we first need to create a persistent disk. On Google Cloud Platform, we can use the gcloud CLI to create disks in a given region –

gcloud compute disks create --size=<required size> --zone=<same as cluster zone> <unique disk name>

After this, we can mount it on a Docker volume defined in Kubernetes configurations –

      ...
      volumeMounts:
        - mountPath: /path/to/mount
          name: volume-name
  volumes:
    - name: volume-name
      gcePersistentDisk:
        pdName: disk-name
        fsType: fileSystemType

But this setup can’t be used for storing loklak dumps. Let’s see “why” in the next section.

Rolling Updates and Persistent Disk

The Kubernetes deployment needs to be updated when the master branch of loklak server is updated. This update of master deployment would create a new pod and try to start loklak server on it. During all this, the older deployment would also be running and serving the requests.


The control will not be transferred to the newer pod until it is ready and all the probes are passing. The newer deployment will now try to mount the disk which is mentioned in the configuration, but it would fail to do so. This would happen because the older pod has already mounted the disk.


Therefore, all new deployments would simply fail to start due to insufficient resources. To overcome such issues, Kubernetes allows persistent volume claims. Let’s see how we used them for loklak deployment.

Persistent Volume Claims

Kubernetes provides Persistent Volume Claims which claim resources (storage) from a Persistent Volume (just like a pod does from a node). The higher level APIs are provided by Kubernetes (configurations and kubectl command line). In the loklak deployment, the persistent volume is a Google Compute Engine disk –

apiVersion: v1
kind: PersistentVolume
metadata:
  name: dump
  namespace: web
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: slow
  gcePersistentDisk:
    pdName: "data-dump-disk"
    fsType: "ext4"

[SOURCE]

It must be noted here that a persistent disk by the name of data-dump-index is already created in the same region.


The storage class defines the way in which the PV should be handled, along with the provisioner for the service –

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
  namespace: web
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  zone: us-central1-a

[SOURCE]

After having the StorageClass and PersistentVolume, we can create a claim for the volume by using appropriate configurations –

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dump
  namespace: web
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
  storageClassName: slow

[SOURCE]

After this, we can mount this claim on our Deployment –

  ...
  volumeMounts:
    - name: dump
      mountPath: /loklak_server/data
volumes:
  - name: dump
    persistentVolumeClaim:
      claimName: dump

[SOURCE]

Verifying persistence of Dumps

To verify this, we can redeploy the cluster using the same persistent disk and check if the earlier dumps are still present there –

$ http http://link.to.deployment/dump/
HTTP/1.1 200 OK
Cache-Control: public, max-age=60
Content-Type: text/html;charset=utf-8
...


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
…
<h1>Index of /dump</h1>
<pre>      Name 
[gz ] <a href="messages_20170802_71562040.txt.gz">messages_20170802_71562040.txt.gz</a>   Thu Aug 03 00:07:21 GMT 2017   132M
[gz ] <a href="messages_20170803_69925009.txt.gz">messages_20170803_69925009.txt.gz</a>   Mon Aug 07 15:40:04 GMT 2017   532M
[gz ] <a href="messages_20170807_36357603.txt.gz">messages_20170807_36357603.txt.gz</a>   Wed Aug 09 10:26:24 GMT 2017   377M
[txt] <a href="messages_20170809_27974404.txt">messages_20170809_27974404.txt</a>      Thu Aug 10 08:51:49 GMT 2017  1564M
<hr></pre>
...

Conclusion

In this blog post, I discussed the process of deployment of loklak with persistent dumps on Kubernetes. This deployment is intended to work as root.loklak.org in near future. The changes were proposed in loklak/loklak_server#1377 by @singhpratyush (me).

Resources

Continue ReadingPersistently Storing loklak Server Dumps on Kubernetes
Read more about the article Making Customized and Mobile Responsive Drop-down Menus in Susper using Angular

Making Customized and Mobile Responsive Drop-down Menus in Susper using Angular

In  Susper, the drop-down menu is customized with colorful search icons and we wanted to maintain the same menu for mobile screens too, however the drop-down menu disappeared for all screens with width less than 767px. This blog can be used to learn how to create css classes for such drop-down menus without using any bootstrap.
This is how the issue was solved.

  1. Replacing standard bootstrap classes : The drop-down menu blocks had a source code as follows:

class=“dropdown-menu”>

class=“row”>

class=“col-sm-4”>

class=“block”>
…
…

</div>
</div>

Using col-sm-4 will do the following

  • For widths greater than 767px: Divide each row into four equally sized columns.
  • For widths smaller than 767px: Stack all the columns on top of each other.

Since the drop-down menu’s design was to remain intact, I made the following changes:

  • Replace row with menu-row
  • Replace col-sm-4 with menu-item

Now I wrote personalized css for these classes.

.menu-row{
width: 267px;
grid–template–columns: 1fr 1fr 1fr;
background-color: white;
}
.menu-item{
display: inline–block;
width: 86px;
}
  • Width: It is used to set the width of the div class, each row now has a width of 267px, with each column in it having a width of 86px.
  • Grid-template-columns: It is used to layout the structure of the template, here 1fr 1fr 1fr represents that there will be three columns in a row.
  • Display: The display is set to inline block to overwrite the default property of the div element to start in a new line.
  1. Custom css for small screens : In standard bootstrap, for screen sizes less than 767px, dropdown class has properties like transparent background, no border etc. that need to be over written. So we add a new id for the div tag as shown:

<div id=“small-drop” class=“dropdown-menu”>

/** Now we add css for it, as shown: **/
@media screen and (max-width: 767px) {
#small-drop{
position: absolute;
background-color: white;
border: 1px solid #cccccc;
right: -38px;
left: auto;
}

  • Position : absolute is used to make sure all our values are absolute and not relative to the higher div hierarchically
  • Border: The values for the border represent the following respectively: Thickness, Style and Color.
  • Auto: Here the value auto for left signifies that there is no fixed value for the left margin, it can take the default value

References:

  1. For working of grids in Bootstrap: https://www.w3schools.com/bootstrap/bootstrap_grid_examples.asp
  2. A useful article for difference between id and class: https://css-tricks.com/the-difference-between-id-and-class

 

Continue ReadingMaking Customized and Mobile Responsive Drop-down Menus in Susper using Angular
Read more about the article Making Autocomplete Box Compatible with the Search Bar using Angular in Susper

Making Autocomplete Box Compatible with the Search Bar using Angular in Susper

A major problem in Susper was that we were using the same components on different pages, with different styling properties. A major issue was that the Autocomplete box was not properly aligned in the index page and looked like this:

This was happening because the autocomplete box width was set for 634 px, a width perfect for the search bar in the results page. The index page had a search bar of width 534 px, and the autocomplete box was too large for that.
Here is how the issue was solved:

  1. Changing the suggestion box html code:

id=“sug-box” class=“suggestion-box” *ngIf=”results.length> 0”>

</div>

The code uses *ngIf which is why setting the autocomplete box width using the typescript files becomes impossible. *ngIf does not load the component into the DOM until there are results, hence we didnot have the autocomplete box in the DOM until after the query was typed in the search bar. That was why we could not set its width, hence it was decided to remove this attribute. Using the ‘hidecomponent emitter’ is a better option here (used in the typescript file).

@Output() hidecomponent: EventEmitter<any> = new EventEmitter<any>();
if (this.results.length === 0) {
this.hidecomponent.emit(1);
} else {
this.hidecomponent.emit(0);
}

See autocomplete.component.ts for the complete code.

  1. It is now required to dynamically change the id of the suggestion-box depending on the page it is on, and apply the correct CSS.

Here is the html code:

<div [id]=”getID()” class=“suggestion-box” *ngIf=“results”>

The id of the suggestion box will now depend on the value returned from the function getID(), defined as follows:

getID() {
if ( this.route.url.toString() === ‘/’) {
  return ‘index-sug-box’;
} else {
  return ‘sug-box’
}
}
  • We first check if the route url is simply ‘/’ (which implies it is in the index page).
  • If yes the id is set to index-sug-box otherwise to sug-box.

Now we can write extra CSS properties for the index-sug-box id as follows:

#index-sug-box{
width: 586px;
}

References:

  1. For basic javascript functions: https://www.w3schools.com/js/js_functions.asp
  2. To understand components in Angular: https://angular.io/api/core/Component

 

Continue ReadingMaking Autocomplete Box Compatible with the Search Bar using Angular in Susper
Read more about the article Adding Github buttons to Generated Documentation with Yaydoc

Adding Github buttons to Generated Documentation with Yaydoc

Many times repository owners would want to link to their github source code, issue tracker etc. from the documentation. This would also help to direct some users to become a potential contributor to the repository. As a step towards this feature, we added the ability to add automatically generated GitHub buttons to the top of the docs with Yaydoc.

To do so we created a custom sphinx extension which makes use of http://buttons.github.io/ which is an excellent service to embed GitHub buttons to any website. The extension takes multiple config values and using them generates the `html` which it adds to the top of the internal docutils tree using a raw node.

GITHUB_BUTTON_SPEC = {
    'watch': ('eye', 'https://github.com/{user}/{repo}/subscription'),
    'star': ('star', 'https://github.com/{user}/{repo}'),
    'fork': ('repo-forked', 'https://github.com/{user}/{repo}/fork'),
    'follow': ('', 'https://github.com/{user}'),
    'issues': ('issue-opened', 'https://github.com/{user}/{repo}/issues'),
}

def get_button_tag(user, repo, btn_type, show_count, size):
    spec = GITHUB_BUTTON_SPEC[btn_type]
    icon, href = spec[0], spec[1].format(user=user, repo=repo)
    tag_fmt = '<a class="github-button" href="{href}" data-size="{size}"'
    if icon:
        tag_fmt += ' data-icon="octicon-{icon}"'
    tag_fmt += ' data-show-count="{show_count}">{text}</a>'
    return tag_fmt.format(href=href,
                          icon=icon,
                          size=size,
                          show_count=show_count,
                          text=btn_type.title())

The above snippet shows how it takes various parameters such as the user name, name of the repository, the button type which can be one of fork, issues, watch, follow and star, whether to display counts beside the buttons and whether a large button should be used. Another method named get_button_tags is used to read the various configs and call the above method with appropriate parameters to generate each button.

The extension makes use of the doctree-resolved event emitted by sphinx to hook into the internal doctree. The following snippet shows how it is done.

def on_doctree_resolved(app, doctree, docname):
    if not app.config.github_user_name or not app.config.github_repo:
        return
    buttons = nodes.raw('', get_button_tags(app.config), format='html')
    doctree.insert(0, buttons)

Finally we add the custom javascript using the add_javascript method.

app.add_javascript('https://buttons.github.io/buttons.js')

To use this with yaydoc, users would just need to add the following to their .yaydoc.yml file.

build:
  github_button:
    buttons:
      watch: true
      star: true
      issues: true
      fork: true
      follow: true
    show_count: true
    large: true

Resources

  1.  Homepage of Github:buttons – http://buttons.github.io/
  2. Sphinx extension Tutorial – http://www.sphinx-doc.org/en/stable/extdev/tutorial.html
Continue ReadingAdding Github buttons to Generated Documentation with Yaydoc
Read more about the article UI Espresso Test Cases for Phimpme Android

UI Espresso Test Cases for Phimpme Android

Now we are heading toward a release of Phimpme soon, So we are increasing the code coverage by writing test cases for our app. What is a Test Case? Test cases are the script against which we run our code to test the features implementation. It is basically contains the output, flow and features steps of the app. To release app on multiple platform, it is highly recommended to test the app on test cases.

For example, Let’s consider if we are developing an app which has one button. So first we write a UI test case which checks whether a button displayed on the screen or not? And in response to that it show the pass and fail of a test case.

Steps to add a UI test case using Espresso

Espresso testing framework provides APIs to simulate user interactions. It has a concise API. Even, now in new Version of Android Studio, there is a feature to record Espresso Test cases. I’ll show you how to use Recorder to write test cases in below steps.

  • Setup Project Directory

Android Instrumentation tests must be placed in androidTest directory. If it is not there create a directory in app/src/androidTest/java…

  • Write Test Case

So firstly, I am writing a very simple test case, which checks whether the three Bottom navigation view items are displayed or not?

Espresso Testing framework has basically three components:

ViewMatchers

Which helps to find the correct view on which some actions can be performed E.g. onView(withId(R.id.navigation_accounts). Here I am taking the view of accounts item in Bottom Navigation View.

ViewActions

It allows to perform actions on the view we get earlier. E.g. Very basic operation used is click()

ViewAssertions

It allows to assert the current state of the view E.g. isDisplayed() is an assertion on the view we get. So a basic architecture of an Espresso Test case is

onView(ViewMatcher)       
 .perform(ViewAction)     
   .check(ViewAssertion);

We can also Use Hamcrest framework which provide extra features of checking conditions in the code.

Setup Espresso in Code

Add this in your application level build.gradle

// Android Testing Support Library's runner and rules
androidTestCompile "com.android.support.test:runner:$rootProject.ext.runnerVersion"
androidTestCompile "com.android.support.test:rules:$rootProject.ext.rulesVersion"

// Espresso UI Testing dependencies.
androidTestCompile "com.android.support.test.espresso:espresso-core:$rootProject.ext.espressoVersion"
androidTestCompile "com.android.support.test.espresso:espresso-contrib:$rootProject.ext.espressoVersion"

  • Use recorder to write the Test Case

New recorder feature is great, if you want to set up everything quickly. Go to the Run → Record Espresso Test in Android Studio.

It dumps the current User Interface hierarchy and provide the feature to assert that.

You can edit the assertions by selecting the element and apply the assertion on it.

Save and Run the test cases by right click on Name of the class. Run ‘Test Case name’

Console will show the progress of Test case. Like here it is showing passed, it means it get all the view hierarchy which is required by the Test Case.

Resources

Continue ReadingUI Espresso Test Cases for Phimpme Android
Read more about the article Display a List of Registered Repositories in User’s Dashboard

Display a List of Registered Repositories in User’s Dashboard

Yaydoc, our automatic documentation generation and deployment project was recently introduced with the dashboard for its registered users. Introducing a dashboard to Yaydoc enabled us to include a lot of new functionalities and processes associated with users and their registered repositories.

One of the main reasons of creating a dashboard for registered users was to enable them to access and edit configurations of the repositories they registered to Yaydoc. Hence, there was a need to display all of the registered repositories to users. These repositories include the public repositories owned by the users and organizations’ repositories the user has admin access to.

The organizations and users are retrieved separately in parallel using async.js. Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. The required functionality is achieved using async.parallel() method which runs the tasks of a collection of functions in parallel, without waiting until the previous function has completed.

async.parallel({
  organizations: function (callback) {
    github.retrieveOrganizations(user.token, function (error, organizations)) {
      callback(null, organizations);
    }
  },
  ownedRepositories: function (callback) {
    Repository.getRepositoriesByOwner(user.username, function (err, repositories)) {
      callback(null, repositories);
    }
  }, function (err, results) {
    res.render(‘dashboard’, {
      title: ‘Dashboard’,
      organizations: results.organizations,
      ownedRepositories: results.ownedRepositories
    });
  }
});

In order to show the organizations’ registered repositories  the github.retrieveOrganizations() function is updated, retrieving registered repositories from Yaydoc’s database. To perform the operation, we use async’s waterfall method. This method runs the tasks in series, passing the result of each function to the next function in that array. This function is needed since we are retrieving the organizations first and then for each organization, we retrieve all the repositories for each organization. To access each organization asynchronously, we use the async.forEachOf() method.

/**
 * Retrieve a list of organization the user has access to
 * Along with the organizations, also retrieve the registered repositories
 * @param accessToken: Access token of the user
 * @param callback
 */
exports.retrieveOrganizations = function (accessToken, callback) {
  async.waterfall([
    function (callback) {
      // Retrieve organizations
      ....
      organizations.push({
        id: organization.id,
        name: organization.login,
        avatar: organization.avatar_url
      });
      return callback(null, organizations);
    },
    function (organizations, callback) {
      async.forEachOf(organizations, function (value, key, callback)) {
        Repository.getRepositoriesByOwner(value.name, function (error, repositories) {
          value.repositories = repositories;
          update.push(value);
          callback()
        });
      }, function (err) {
        callback(null, update);
      }
    }
  ], function (error, result) {
    callback(error, result);
  });
};

Showing the list of repositories, we now plan to make these repositories configurable. Some of these configurations may include deleting the repository and enabling or disabling it from the build process.

Resources

  1. Async utilities for node and browser – https://caolan.github.io/async/
  2. Github’s organization API https://developer.github.com/v3/orgs/
Continue ReadingDisplay a List of Registered Repositories in User’s Dashboard
Read more about the article Electronics Experiments with PSLab

Electronics Experiments with PSLab

Numerous college level electronics experiments can be performed using Pocket Science Lab (PSLab). The Android app and the Desktop app have all the essential features needed to perform these experiments and both these apps have quite a large number of experiments built-in. Some of the common experiments involve the use of BJT (Bipolar Junction Transistor), Zener Diode, FET (Field Effect Transistor), Op-Amp ( Operational Amplifier) etc. This blog walks through the details of performing some experiments using the above commonly used elements.  

The materials required for all the experiments are minimal and includes a few things like PSLab hardware device, components like Diodes, Transistors, Op-Amps etc., connecting wires/jumpers and secondary components like resistors, capacitors etc. Most of these elements would be a part of the PSLab Accessory Kit.

It is recommended to read this blog here, go through the resources mentioned at the end and also get acquainted with construction of circuits before advancing with the experiments mentioned in this blog.

Half Wave and Full Wave Rectifiers

The Bipolar Junction Transistor (BJT) can be used as a rectifier. Rectifiers are needed in circuits to obtain a nearly constant and stable output voltage and prevent any ripples in the circuit. The rectifier can be half wave or full wave depending on whether it rectifies one or both cycles of Alternating Voltage.

The circuit for the Half and Full Wave rectifier is given as follows:

  • Construct the above circuits on a breadboard.
  • For the half wave rectifier, connect the terminals of CH1 and GND of PSLab on the input side and the terminals of CH2 and GND on the output side.
  • The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 100 Hz and magnitude to 10mV. Then go ahead and open the Oscilloscope.
  • CH1 would display the input waveform and CH2 will display the output waveform and the plots can be observed.
  • The plot obtained will have rectification in only half of the cycle. In order to obtain rectification in the complete cycle, the full wave rectifier is needed.
  • For the full wave rectifier, the procedure is the same but an additional diode is used. Use an additional channel CH3 to plot the extra input.

  • The plot obtained from the above steps would still have ripples and so a capacitor is placed in parallel to cancel this effect.
  • Place a 100uF/330uF capacitor in parallel to the resistor RL and an additional 1 ohm resistor in the circuit.

BJT Inverter

  • Transistor has a lot of functions. The most common of them is its use as an amplifier. However, transistor can be used as a switch in a circuit i.e. as an inverter.
  • The circuit for this experiment is shown below. For this experiment, it is recommended to use an external 5V DC supply like a battery. Connect the transistor and the diode initially and then connect the resistors accordingly. (Connect the terminals of diode and transistor carefully else they will be damaged).
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure.
  • The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 200 Hz and magnitude to 10mV.
  • Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo which should look like the graph shown below.

Common Mode Gain and Differential Mode Gain in Op-Amps

Gain of any amplifier can be calculated by calculating the ratio of the output and input voltage. On plotting the graph in X-Y mode, a Vo vs Vi graph is obtained. The slope of that graph gives us the gain at any particular input voltage.

  • For finding the Differential Mode gain of an Op-Amp, construct the circuit as shown below.
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure. The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • The power supply provided to the Op-Amp are set to + 12V. (If faced with any confusion, please refer to the resources mentioned at the end of the blog to learn more about Op-Amps before proceeding ahead.)
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 1000 Hz and magnitude to 0.5V. Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo.
  • For finding the Common Mode gain of the Op-Amp, remove the waveform generator input i.e W1 from R3 and attach it to R2. The rest of the steps remain the same.

Schmitt Trigger

In electronics, a Schmitt trigger is a comparator circuit with hysteresis implemented by applying positive feedback to the noninverting input of a comparator or differential amplifier. It is an active circuit which converts an analog input signal to a digital output signal.

  • Construct the circuit as shown below. Although the diagram shows a variable resistor be used, a constant value resistor would also work fine.
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure. The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • The power supply provided to the Op-Amp are set to + 12V. (If faced with any confusion, please refer to the resources mentioned at the end of the blog to learn more about Op-Amps before proceeding ahead. If done incorrectly, Op-Amps will be damaged)
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 1000 Hz and magnitude to 0.5V. Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo which should look like the graph shown below.

Resources:

  1. Read more about Half wave and Full wave rectifier and their applications – https://en.wikipedia.org/wiki/Rectifier
  2. Read more about the Bipolar Junction Transistor and its use as a switch – http://www.electronicshub.org/transistor-as-switch/
  3. Understand the common mode and differential mode of Op-Amp – https://www.allaboutcircuits.com/video-lectures/op-amps-common-differential/
  4. Find more about Schmitt Trigger and its uses – https://en.wikipedia.org/wiki/Schmitt_trigger
Continue ReadingElectronics Experiments with PSLab
Read more about the article Introduction To Kotlin in SUSI Android App

Introduction To Kotlin in SUSI Android App

Lately, we wrote some of the code of SUSI Android App in Kotlin. Kotlin is a very similar language to Java but with much more advantages than Java. It is easy to adapt and learn. There is no doubt that Kotlin is better than Java but with the announcement of Kotlin Support in Google IO’17 for Android development, Kotlin seems a decent way to write code for an Android App.

Advantages of Kotlin over Java

    1. Reduce Boilerplate Code: It helps making development of app faster as it reduces more than 20 percent of boilerplate code. Writing long statements again and again is a headache for developers. Kotlin comes to rescue in that situation.
    2. Removes Null Pointer Exception: Once a large company faced millions of dollars of loss due to null pointer exception. It causes crashes of apps more often than anything else. Thus Kotlin helps in Null checks and makes app free from Null pointer Exceptions.
    3. Interoperable with Java: Kotlin code and Java code are interoperable. Which means you can write half your code in kotlin and half in Java and it will work like a charm. You can call java methods from Kotlin code and vice versa. So, you can simply move your existing Java based app to Kotlin slowly making your app always running.
    4. Lambda and Inline functions: Yes, Kotlin also has functionalities from functional programming languages. Mainly and most widely used feature of those languages is Lambda functions.
    5. Direct Reference of Views by Id: You do not need to write findViewById(R.id.view_name) or use any other library like Butterknife for view binding. You can simply use the view by its id.
    6. No semicolon:  Last but not the least, you do not need to add a semicolon after each statement. In fact, you do not need to add semicolon at all.

Setting up Android Studio to work with Kotlin

If you have latest Android Studio Canary Version, there is already a build support for Kotlin in it. You need not do anything in that case. But if you don’t have the Canary version, you can add Kotlin Plugin in your Android Studio. Follow the below steps to do that.

  1. Install the Kotlin Plugin:

Android Studio → Preferences… →Plugins → Browse Repository → type “Kotlin” in search box → install

  1. Restart your Android Studio and Rebuild the project. Everything else is already set up in SUSI Android App but if you want to do it for your other apps, follow this link.

Implementation in SUSI Android App

So, I am not going to give unnecessary code but will point out specific things where Kotlin helped a lot to reduce unnecessary code and made the code compact.

1. Listeners:

Earlier with Java

Button signup = (Button) findViewById(R.id.sign_up);

signup.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
               startActivity(new Intent(LoginActivity.this, SignUpActivity.class));
            }
        });

Now, with Kotlin

fun signUp() {
   sign_up.setOnClickListener { startActivity(Intent(this@LoginActivity, SignUpActivity::class.java)) }
}

2. Models

With Java

public class MapData {

    private double latitude;
    private double longitude;
    private double zoom;

    public MapData(double latitude, double longitude, double zoom) {
        this.latitude = latitude;
        this.longitude = longitude;
        this.zoom = zoom;
    }

    public double getLatitude() {
        return latitude;
    }

    public void setLatitude(double latitude) {
        this.latitude = latitude;
    }

    public double getLongitude() {
        return longitude;
    }

    public void setLongitude(double longitude) {
        this.longitude = longitude;
    }

    public double getZoom() {
        return zoom;
    }

    public void setZoom(double zoom) {
        this.zoom = zoom;
    }
}

With Kotlin

class MapData (var latitude: Double, var longitude: Double, var zoom: Double)

3. Constructor

With Java

public class LoginPresenter {
    private LoginActivity loginActivity;
    public LoginPresenter(loginActivity: LoginActivity){
        this.loginActivity = loginActivity;
    }
}

With Kotlin

class LoginPresenter(loginActivity: LoginActivity) {
}

Summary

So, this blog was to give you an idea about Kotlin programming language, it’s advantages over java and information how you can set it up on your Android Studio so as to help you a little in understanding the codebase of SUSI Android App a little more.

Resources

  1. Official Kotlin Guide for Syntax Reference and further learning  https://kotlinlang.org/docs/reference/
  2. Blog by Elye on Setting up Kotlin on Android Studio https://android.jlelse.eu/setup-kotlin-for-android-studio-1bffdf1362e8
  3. Youtube Video tutorial by Derek Banas on Kotlin https://www.youtube.com/watch?v=H_oGi8uuDpA
Continue ReadingIntroduction To Kotlin in SUSI Android App
Read more about the article How to Implement Feedback System in SUSI iOS

How to Implement Feedback System in SUSI iOS

The SUSI iOS app provides responses for various queries but the response is always not accurate. To improve the response, we make use of the feedback system, which is the first step towards implementing Machine Learning on the SUSI Server. The way this works is that for every query, we present the user with an option to upvote or downvote the response and based on that a positive or negative feedback is saved on the server. In this blog, I will explain how this feedback system was implemented in the SUSI iOS app.

Steps to implement:

We start by adding the UI which is two buttons, one with a thumbs up and the other with a thumbs down image.

textBubbleView.addSubview(thumbUpIcon)
textBubbleView.addSubview(thumbDownIcon)
textBubbleView.addConstraintsWithFormat(format: "H:[v0]-4-[v1(14)]-2-[v2(14)]-8-|", views: timeLabel, thumbUpIcon, thumbDownIcon)
textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbUpIcon)
textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbDownIcon)
thumbUpIcon.isUserInteractionEnabled = true
thumbDownIcon.isUserInteractionEnabled = true

Here, we add the subviews and assign constraints so that these buttons align to the bottom right next to each other. Also, we enable the user interaction for these buttons.

We know that the user can rate the response by pressing either of the buttons added above. To do that we make an API call to the endpoint below:

BASE_URL+'/cms/rateSkill.json?'+'model='+model+'&group='+group+'&skill='+skill+’&language’+language+’&rating=’+rating

Here, the BASE_URL is the url of the server, the other three params model, group, language and skill are retrieved by parsing the skill location parameter we get with the response. The rating is positive or negative based on which button was pressed by the user. The skill param in the response looks like this:

skills:
[
"/susi_skill_data/models/general/entertainment/en/quotes.txt"
]

Let’s write the method that makes the API call and responds to the UI that it was successful.

if let accepted = response[ControllerConstants.accepted] as? Bool {
  if accepted {
    completion(true, nil)
    return
  }
  completion(false, ResponseMessages.ServerError)
  return
}

Here after receiving a response from the server, we check if the `accepted` variable is true or not. Based on that, we pass `true` or `false` to the completion handler. Below the response we actually receive by making the request.

{
session: {
identity: {
type: "host",
name: "23.105.140.146",
anonymous: true
}
},
accepted: true,
message: "Skill ratings updated"
}

Finally, let’s update the UI after the request has been successful.

if sender == thumbUpIcon {
thumbDownIcon.tintColor = UIColor(white: 0.1, alpha: 0.7)
thumbUpIcon.isUserInteractionEnabled = false
thumbDownIcon.isUserInteractionEnabled = true
feedback = "positive"
} else {
thumbUpIcon.tintColor = UIColor(white: 0.1, alpha: 0.7)
thumbDownIcon.isUserInteractionEnabled = false
thumbUpIcon.isUserInteractionEnabled = true
feedback = "negative"
}
sender.tintColor = UIColor.hexStringToUIColor(hex: "#2196F3")

Here, we check the sender (the thumbs up or down button) and based on that pass the rating (positive or negative) and update the color of the button.

Below is the app in action with the feedback system.

Resources:

Continue ReadingHow to Implement Feedback System in SUSI iOS
Read more about the article Using Mosquitto as a Message Broker for MQTT in loklak Server

Using Mosquitto as a Message Broker for MQTT in loklak Server

In loklak server, messages are collected from various sources and indexed using Elasticsearch. To know when a message of interest arrives, users can poll the search endpoint. But this method would require a lot of HTTP requests, most of them being redundant. Also, if a user would like to collect messages for a particular topic, he would need to make a lot of requests over a period of time to get enough data.

For GSoC 2017, my proposal was to introduce stream API in the loklak server so that we could save ourselves from making too many requests and also add many use cases.

Mosquitto is Eclipse’s project which acts as a message broker for the popular MQTT protocol. MQTT, based on the pub-sub model, is a lightweight and IOT friendly protocol. In this blog post, I will discuss the basic setup of Mosquitto in the loklak server.

Installation and Dependency for Mosquitto

The installation process of Mosquitto is very simple. For Ubuntu, it is available from the pre installed PPAs –

sudo apt-get install mosquitto

Once the message broker is up and running, we can use the clients to connect to it and publish/subscribe to channels. To add MQTT client as a project dependency, we can introduce following line in Gradle dependencies file – 

compile group: 'net.sf.xenqtt', name: 'xenqtt', version: '0.9.5'

[SOURCE]

After this, we can use the client libraries in the server code base.

The MQTTPublisher Class

The MQTTPublisher class in loklak would provide an interface to perform basic operations in MQTT. The implementation uses AsyncClientListener to connect to Mosquitto broker – 

AsyncClientListener listener = new AsyncClientListener() {
    // Override methods according to needs
};

[SOURCE]

The publish method for the class can be used by other components of the project to publish messages on the desired channel – 

public void publish(String channel, String message) {
    this.mqttClient.publish(new PublishMessage(channel, QoS.AT_LEAST_ONCE, message));
}

[SOURCE]

We also have methods which allow publishing of multiple messages to multiple channels in order to increase the functionality of the class.

Starting Publisher with Server

The flags which signal using of streaming service in loklak are located in conf/config.properties. These configurations are referred while initializing the Data Access Object and an MQTTPublisher is created if needed – 

String mqttAddress = getConfig("stream.mqtt.address", "tcp://127.0.0.1:1883");
streamEnabled = getConfig("stream.enabled", false);
if (streamEnabled) {
    mqttPublisher = new MQTTPublisher(mqttAddress);
}

[SOURCE]

The mqttPublisher can now be used by other components of loklak to publish messages to the channel they want.

Adding Mosquitto to Kubernetes

Since loklak has also a nice Kubernetes setup, it was very simple to introduce a new deployment for Mosquitto to it.

Changes in Dockerfile

The Dockerfile for master deployment has to be modified to discover Mosquitto broker in the Kubernetes cluster. For this purpose, corresponding flags in config.properties have to be changed to ensure that things work fine – 

sed -i.bak 's/^\(stream.enabled\).*/\1=true/' conf/config.properties && \
sed -i.bak 's/^\(stream.mqtt.address\).*/\1=mosquitto.mqtt:1883/' conf/config.properties && \

[SOURCE]

The Mosquitto broker would be available at mosquitto.mqtt:1883 because of the service that is created for it (explained in later section).

Mosquitto Deployment

The Docker image used in Kubernetes deployment of Mosquitto is taken from toke/docker-kubernetes. Two ports are exposed for the cluster but no volumes are needed – 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mosquitto
  namespace: mqtt
spec:
  ...
  template:
    ...
    spec:
      containers:
      - name: mosquitto
        image: toke/mosquitto
        ports:
        - containerPort: 9001
        - containerPort: 8883

[SOURCE]

Exposing Mosquitto to the Cluster

Now that we have the deployment running, we need to expose the required ports to the cluster so that other components may use it. The port 9001 appears as port 80 for the service and 1883 is also exposed – 

apiVersion: v1
kind: Service
metadata:
  name: mosquitto
  namespace: mqtt
  ...
spec:
  ...
  ports:
  - name: mosquitto
    port: 1883
  - name: mosquitto-web
    port: 80
    targetPort: 9001

[SOURCE]

After creating the service using this configuration, we will be able to connect our clients to Mosquitto at address mosquitto.mqtt:1883.

Conclusion

In this blog post, I discussed the process of adding Mosquitto to the loklak server project. This is the first step towards introducing the stream API for messages collected in loklak.

These changes were introduced in pull requests loklak/loklak_server#1393 and loklak/loklak_server#1398 by @singhpratyush (me).

Resources

Continue ReadingUsing Mosquitto as a Message Broker for MQTT in loklak Server