Using Picasso library in SUSI Android

SUSI is an artificial intelligence for chatbots which have the ability to reply in most intuitive way through different types of answers such as images, charts, maps and text. Hence for the image displays in the SUSI Android client we need an image loading library which can help us to cache the images in the app. There are a few options available which include Glide and Picasso. Both of these libraries are open sourced. After some research we finally came up to use Picasso as it provides more additional features in comparison to Glide.

Picasso is an image downloading library. It is an open source library. It is published and maintained by Square. It allows the developer to display an image from the external URL of the image. It provides caching of image in just a few lines of code. Previously without this library it was very difficult to download and display the image and required a lot more lines of code. But with the help of Picasso this task is reduced to just a few lines of code.

How to use Picasso?

To use Picasso in our project we must add the dependency of the library in build.gradle file.

dependencies {
  ...
  compile "com.squareup.picasso:picasso:2.4.0"
  ...
}

Let us define an imageView in which we want to load the image with the help of Picasso Library.

<ImageView
   android:layout_width="wrap_content"
   android:layout_height="wrap_content"
   android:id="@+id/imageView"
   android:layout_alignParentTop="true"
   android:layout_centerHorizontal="true">
</ImageView>


Now we are all set to download the image with the help of Picasso library in the following way:-

//Initialize ImageView
ImageView imageView = (ImageView) findViewById(R.id.imageView);

//Loading image from below url into imageView

Picasso.with(this)
  .load("IMAGE URL")
  .into(imageView);

Picasso also provide the function for setting placeholders and error images to be shown if there is any problem in the downloading of the image.

Picasso.with(this)
   .load("YOUR IMAGE URL HERE")
   .placeholder(R.drawable.ic_placeholder) 
   .error(R.drawable.ic_error_fallback)         
   .into(imageView);

Now let us see the implementation of Picasso in Susi Android

In the Susi app we are storing the link of images coming from response in the imageList.

if (imageList == null || imageList.size() == 0) {

  holder.linkPreviewImageView.setVisibility(View.GONE);

} else {

  Picasso.with(context).load(imageList.get(0))

          .fit().centerCrop()

          .into(holder.linkPreviewImageView);
  }

Here we are passing the activity context to the Picasso library. We can use additional features like fit() and centerCrop() method the way we are using in the Susi app. These methods are fit the image at the center of the imageView.

Screenshots from the Susi App

Picasso Library also provides some additional functions as well like:-

Picasso.with(this)
    .load("YOUR IMAGE URL HERE")        
    .placeholder(R.drawable.ic_placeholder)   // optional        
    .error(R.drawable.ic_error_fallback)      // optional        
    .resize(250, 200)                        // optional        
    .rotate(90)                             // optional        
    .into(imageView);

You can find more about Picasso from this link.

Continue Reading Using Picasso library in SUSI Android

Implementing DuckDuckGo Api in SUSI Android

As we know that Susi is an open source intelligent chatbot, it must be able to reply with user’s query on any topic. Therefore we have implemented DuckDuckGo API in Susi Android(https://github.com/fossasia/susi_android) which will help us to generate search result for the query asked by the user.

 

DuckDuckGo is an API which provides instant search results. This basically works as a search engine having information about various things. The most important thing about DuckDuckGo is that it is non tracking. It does not track its users and show results based on their search history. Thus the search results remain uniform across all the clients irrespective of their search history. The information inside the API is fed from more than 120 different independent sources. This is what makes it different from other search engines. The response in the form of answers include different types of links, description, categories, and definition about various stuffs.

For more details about the Api please check this link.

 

API endpoints:

http://api.duckduckgo.com/?q=DuckDuckGo&format=json

This is one of the links generated to test the api. Here we can see different parameters, the parameter q is the query parameter where we write our question/query to get the response from the API. The format here defines the format in which we want the response to be. Here in this case we are obtaining the response in the form of JSON which can be parsed to obtain the desired result in the client.

The response obtained by the following query is as follow:-

{  

  "DefinitionSource":"",

  "Heading":"DuckDuckGo",

  "ImageWidth":340,

  "RelatedTopics":[  

     {  

        "Result":"<a href=\"https://duckduckgo.com/Names_Database\">Names Database</a> - The Names Database is a partially defunct social network, owned and operated by Classmates.com, a wholly owned subsidiary of United Online. The site does not appear to be significantly updated since 2008, and has many broken links and display issues.",

        "Icon":{  

           "URL":"",

           "Height":"",

           "Width":""

        },

        "FirstURL":"https://duckduckgo.com/Names_Database",

        "Text":"Names Database - The Names Database is a partially defunct social network, owned and operated by Classmates.com, a wholly owned subsidiary of United Online. The site does not appear to be significantly updated since 2008, and has many broken links and display issues."

     }

}

 

Implementation in Susi android

In Susi Android we are using Retrofit library by Square for API calling. Retrofit is one of the best libraries present for the network calling. It helps the developer to migrate from the old way of using AsyncTask in the Android app which creates a lot of mess and ugly code.

For the implementation in Susi Android, we have made a WebSearchClient class that stores the base address for the API calling.

public class WebSearchClient {

  public static final String BASE_URL = "http://api.duckduckgo.com";

  private static Retrofit retrofit = null;



  public static Retrofit getClient() {

      if (retrofit==null) {

          retrofit = new Retrofit.Builder()

                  .baseUrl(BASE_URL)

                  .addConverterFactory(GsonConverterFactory.create())

                  .build();

      }

      return retrofit;

  }

}

To get the response we call the API with get method passing the query and format parameter in the following way.





public interface WebSearchService {



  @GET("/?format=json&pretty=1")

  Call<WebSearch> getresult(@Query("q") String query);

}



The results obtained from the server in the form of JSON is parsed in the form of objects in this way:-



public class WebSearch{



  @SerializedName("Heading")

  @Expose

  private String heading;



  @SerializedName("RelatedTopics")

  @Expose

  private List<RelatedTopics> relatedTopics;



  public WebSearch(String heading, List<RelatedTopics> relatedTopics) {

      this.heading = heading;

      this.relatedTopics = relatedTopics;

  }





public class RelatedTopics {



  @SerializedName("FirstURL")

  @Expose

  private String url;



  @SerializedName("Text")

  @Expose

  private String text;



  @SerializedName("Icon")

  @Expose

  private WebIcon icon;



  @SerializedName("Result")

  @Expose

  private String result;



  public RelatedTopics(String url, String text, WebIcon icon) {

      this.url = url;

      this.text = text;

      this.icon = icon;

  }



At the end after setting up all the above things, we call the API and store the result in the realm or local storage.

final WebSearchService apiService = WebSearchClient.getClient().create(WebSearchService.class);



Call<WebSearch> call = apiService.getresult(webquery);



call.enqueue(new Callback<WebSearch>() {

  @Override

  public void onResponse(Call<WebSearch> call, Response<WebSearch> response) {

      Log.e(TAG, response.toString());

      if (response.body() != null ) {

          realm.beginTransaction();

          RealmList<WebSearchModel> searchResults = new RealmList<>();


    }
}

This completes the calling of DuckDuckGo API and storing the results in the database.

To know more about API calling using Retrofit you can refer to this link.

Continue Reading Implementing DuckDuckGo Api in SUSI Android

Custom Views in Susi Android App

Android provides us with the ability to have different views for your App. These views help in the formation of the UI element of the application. These includes imageView, textView and layouts such as LinearLayout and FrameLayout etc. The view hierarchy of Android looks something like this.

The problem with these views is that we cannot modify them according to our own need inside the application. This is what we faced during the making of chat bubble layout for Susi Android App (https://github.com/fossasia/susi_android). We wanted to implement the chat bubble similar to Whatsapp that resizes and position the time textView according to size of the response coming from the server ie something like this as shown in the screenshot. Therefore we finally came up the solution of using Custom views inside the app that allowed us to modify the view the way we wanted.

 

So now lets us understand how we can make custom views by extending existing views

So first question that comes in our mind is why are we extending existing views if we want to make our own. The reason behind this is that extending an existing view provides us with ability to have all the features that are there in an existing view.

On top of that we can add our own functionality into it. Now see below how we can implement it.

It’s time for some actual code.

As we can see in the code below that here we made our own custom class called ValueSelector. This class is extending the existing layout which is RelativeLayout. The first constructor used in the above class which takes context as the parameter is used to create an instance of the view programmatically. The second constructor used which takes context and AttributeSet as parameters is used to inflate the view from the XML.

While the third constructor is used to define the base classes.

public class ValueSelector extends RelativeLayout {

   View rootView;
   TextView valueTextView;
   View minusButton;
   View plusButton;

   public ValueSelector(Context context) {
       super(context);
       init(context);
   }

   public ValueSelector(Context context, AttributeSet attrs) {
       super(context, attrs);
       init(context);
   }

   private void init(Context context) {
       //do setup work here
   }

The init method used here is to inflate the views and to get the reference of all the child view.

private void init(Context context) {
   rootView = inflate(context, R.layout.value_selector, this);
   valueTextView = (TextView) rootView.findViewById(R.id.valueTextView);

   minusButton = rootView.findViewById(R.id.minusButton);
   plusButton = rootView.findViewById(R.id.plusButton);

   minusButton.setOnClickListener(new View.OnClickListener() {
       @Override
       public void onClick(View v) {
           decrementValue(); //we'll define this method later
       }
   });

   plusButton.setOnClickListener(new View.OnClickListener() {
       @Override
       public void onClick(View v) {
           incrementValue(); //we'll define this method later        }
   });
}

Let’s now see the implementation of CustomViews in Susi Android.

 

public class ChatBubbleLayout extends FrameLayout {



  public ChatBubbleLayout(Context context) {

      super(context);

  }



  public ChatBubbleLayout(Context context, AttributeSet attrs) {

      super(context, attrs);

  }



  public ChatBubbleLayout(Context context, AttributeSet attrs, int defStyleAttr) {

      super(context, attrs, defStyleAttr);

  }



  @TargetApi(21)

  public ChatBubbleLayout(Context context, AttributeSet attrs, int defStyleAttr, int defStyleRes) {

      super(context, attrs, defStyleAttr, defStyleRes);

  }



  @Override

  protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {

      super.onMeasure(widthMeasureSpec, heightMeasureSpec);

  }

 

In Susi Android app we are making our own custom view for the chat bubble layout, the view is extending the FrameLayout. Here we are overriding onMeasure method. This method is used for defining the constraint of the layout relative to that of the parent view. With the help of this we can set how big or small our layout will be.

Now lets us see the design of the custom view in Susi app.

 

      You can learn more about the custom views from this link.

Continue Reading Custom Views in Susi Android App

Deploy SUSI AI to Viber messenger

Prerequisites

  1. Basic knowledge about calling API’s and fetching data or posting data to the API.
  2. Node.js language.
  3. Github
  4. Heroku

FigArchitecture for running all different messaging services.

To integrate Susi AI chat to Viber, a public account is needed, messaging to which users can chat with Susi.

We need to have a webhook url. Webhook url is a url which serves our Node.js code i.e. the code we will write to serve requests from Viber and to respond back to it.

Whenever a user messages to the SUSI AI public account, these messages come as post requests to our webhook url. The url then requests Susi API to give an answer for the (question based) message received from Viber. The answer fetched from Susi API is sent to the messenger platform’s API by the webhook url, to show it to the user on Viber.

As said we need a public account for our chat bot. The steps to be followed can be seen from here (Steps 2 and 3).

The REST API helps to make applications follow a RESTful way. In this way, the requests and response are in the form of JSON objects. Any language can be used to make an application follow a RESTful way.

In this blog, I will be using Node.js language.

The Rest API Viber, is the document to be followed for integration of a chatbot to Viber. Let’s go through each of the steps:

  1. To call Susi API and fetch an answer from it for a query (‘hi’ in this case). Let’s first visit http://api.asksusi.com/susi/chat.json?q=hi from the browser. We will get a JSON object as follows:

The answer can be found as the value of the key named expression. In this case it is “Hallo!”.

To fetch the answer through coding, we can use this code snippet in Node js:

// including request module
var request = require(‘request’);

// setting options to make a successful call to Susi API.
var options = {
    method: 'GET',
    url: 'http://api.asksusi.com/susi/chat.json',
    qs: 
    {
        timezoneOffset: '-330',
        q:'hi'
    }
};

// A request to the Susi bot
request(options, function (error, response, body) {
    if(error)
        throw new Error(error);
    //answer fetched from susi
    ans = (JSON.parse(body)).answers[0].actions[0].expression;
}

The properties required for the call are set up through a json object (i.e. options). Pass the options object to our request function as its 1st parameter. The response by the API will be stored in ‘body’ variable. We need to parse this body, to be able to access the properties of that body object. Hence, fetching the answer from Susi API.

  1. Let’s set the webhook url for our Susi public account. The folder containing our Node.js code must be pushed to a repo in github. We need to do some changes to the default package.json file in our project.

This file has a code portion:

The “test” key and its value must be replaced with “start”: “node index.js” i.e. node followed by the name of the main file which has to accept the requests from Viber and responds to it. In my case it is index.js . So the resultant code of package.json file should look like this:

Now, even on the local system you can run the node application by running “npm start” command from our project folder.

  • Push this project to github.
  • Create a new heroku app for Node js, following the steps here.
  • Then link this app to the repository where you pushed your code. For reference, follow the steps 7 and 8 here.

To set a webhook for our account, we can use this code snippet.

var headerBody = { 
                     'cache-control': 'no-cache',
                     'content-type': 'application/json',
                     'x-viber-auth-token': 'YOUR_X_VIBER_AUTH_TOKEN'
                 };

var options = {
                  method: 'POST',
                  url: 'https://chatapi.viber.com/pa/set_webhook',
                  headers: headerBody,
                  body:
                       { 
                           url: 'YOUR_WEBHOOK_URL',
                           event_types: ['delivered', 'seen', 'failed', 'subscribed', 'unsubscribed','conversation_started']
                       },
                  json: true 
              };

// request to the chat api of viber.
request(options, function(error, res, body) {
    if (error)
        throw new Error(error);
    response.write("The status message - " + body.status_message);
    response.end(); 
});

To set the webhook url, we need to post info to the viber chat api i.e. https://chatapi.viber.com/pa/set_webhook. The headers key present in our options object must have the Viber authentication key in the object passed to it. So we have passed a headerBody object to it, which contains our x-viber-auth-token. This property helps Viber to set webhook url for the account that corresponds to this passed Viber authentication token. The format of the body of options object is according to this(as stated in the doc):

{  
    "url": "https://my.host.com",  
    "event_types": ["delivered", "seen", "failed", "subscribed", "unsubscribed", "conversation_started"]  
}

We can wrap up this code in a app.get() block. So that whenever we visit our webhook url from a browser, we initiate a get request. This request as seen below, initiates a request to the chat api of Viber to set this url as a webhook url for our Susi AI public account.

app.get('/',function(req, response){
    response.writeHead(200,{'content-type': 'text/plain'});
    response.write("To chat with Susi through Viber, visit this link - chats.viber.com/chatauto and click on the 'Have a look' button\n\n");

    // setting options to request the chat api of viber.
    var options = {
                      method: 'POST',
                      url:'https://chatapi.viber.com/pa/set_webhook',
                      headers: headerBody,
                      body: 
                      {
                          url:'https://intense-crag-83953.herokuapp.com',
                          event_types: ['delivered', 'seen', 'failed', 'subscribed', 'unsubscribed', 'conversation_started']
                      },
                      json: true 
    };

    // request to the chat api of viber.
    request(options, function(error, res, body) {
        if (error) 
            throw new Error(error);
        response.write("The status message received for set Webhook request is - " + body.status_message);
        response.end();
    });
});

Now after setting up a webhook url, the messages sent to our account will come as post requests to this webhook url.

Let’s work with some of the events, for which Viber callbacks our webhook url.

First, let’s talk about the message event request.

According to Viber, this will be the body of the request:

{
    "event": "message",
    "timestamp": 1457764197627,
    "message_token": 4912661846655238145,
    "sender": {
        "id": "01234567890A=",
        "name": "John McClane",
        "avatar": "http://avatar.example.com",
        "country": "UK",
        "language": "en",
        "api_version": 1
    },
    "message": {
       "type": "text",
       "text": "a message to the service",
       "media": "http://example.com",
       "location": {
          "lat": 50.76891,
          "lon": 6.11499
       },
       "tracking_data": "tracking data"
    }
}

We check if the value of ‘event’ key is equal to ‘message’:

app.post('/', function(req, response) {
    response.writeHead(200);

    // If user sends a message in 1-on-1 chat to the susi public account
    if(req.body.event === 'message'){

If it is, then the body of if block can be populated with the code to handle messages by the user.

Assuming we have a reply by Susi API in the ans variable. We can include this code further:

// setting options to request the chat api of viber.
    var options1 = {
                       method: 'POST',
                       url: 'https://chatapi.viber.com/pa/send_message',
                       headers: headerBody,
                       body: 
                            {
                                receiver: req.body.sender.id,
                                min_api_version: 1,
                                sender: 
                                       {
                                           name: 'Susi',
                                           avatar: '' 
                                       },
                                tracking_data: 'tracking data',
                                type: 'text',
                                text: ans 
                   },
                   json: true 
   };

// request to the chat api of viber.
request(options1, function (error1, res, body1) {
    if (error1)
        throw new Error(error1);
    console.log(body1);
});

This above code can help us send a response to the user. To send a message to the user, our body object in options1 must be similar to:

{
    "receiver": "01234567890A=",
    "min_api_version": 1,
    "sender": {
       "name": "John McClane",
       "avatar": "http://avatar.example.com"
    },
    "tracking_data": "tracking data",
    "type": "text",
    "text": "a message from pa"
}

receiver key should have a value of the id of the user to which message needs to sent, which will be shown on our account interface. The user id can be easily fetched from request by ‘req.body.sender.id’ as the reply is to be sent to the same user from where we received the message.

The sender key’s value is an object which indicates the sender’s name, which in our case is Susi. Also we can pass an avatar url along with the name.

The text key must have our answer as the value i.e. the ‘ans’ variable (in this case).

The whole code to accept the request, and reply accordingly:

// If user sends a message in 1-on-1 chat to the susi public account
if(req.body.event === 'message'){
    // Susi answer to a user message
    var ans;
    // setting options to request susi bot.
    var options1 = { 
                       method: 'GET',
                       url:'http://api.asksusi.com/susi/chat.json',
                       qs: { 
                            timezoneOffset: '-330',
                            q:req.body.message.text 
                           }
    };

    // A request to the Susi bot
    request(options1, function (error1, response1, body1) {
        if (error1)
            throw new Error(error1);
        // answer fetched from susi
        ans = (JSON.parse(body1)).answers[0].actions[0].expression;
        var options = {
                          method: 'POST',
                          url:'https://chatapi.viber.com/pa/send_message',
                          headers: headerBody,
                          body: 
                               {
                               receiver: req.body.sender.id,
                               min_api_version: 1,
                               sender: 
                                      {
                                           name: 'Susi',
                                           avatar: '' 
                                      },
                               tracking_data:'tracking data',
                               type: 'text',
                               text: ans 
                      },
        json: true 
    };

    // request to the chat api of viber.
    request(options, function (error, res, body) {
        if (error)
            throw new Error(error);
        console.log(body);
    });
}

The same way we can handle “conversation started” type of event:

if(req.body.event === 'conversation_started'){
    // Welcome Message
    var request = require("request");
    var options = {
                      method: 'POST',
                      url:'https://chatapi.viber.com/pa/send_message',
                      headers: headerBody,
                      body: 
                          {
                              receiver: req.body.user.id,
                              min_api_version: 1,
                              sender: 
                                    {
                                        name: 'Susi',
                                        avatar: '' 
                                    },
                              tracking_data: 'tracking data',
                              type: 'text',
                              text:'Hi from your favourite, Susi!' 
                          },
                      json: true 
    };

    request(options, function (error, res, body) {
        if (error)
            throw new Error(error);
        console.log(body);
    });
}

This way our Susi AI chatbot can reply to messages from Viber.

The repository which contains whole project of Susi AI’s integration to Viber can be found here.

Continue Reading Deploy SUSI AI to Viber messenger

Auto Deploying accounts.susi.ai on gh-pages

While migrating web apps from susi server repository to accounts.susi.ai repository. Our team autodeployed accounts.susi.ai on gh-pages branch i.e., each time changes are made to code, it gets auto deployed on gh-pages branch. And changes are automatically live and visible on site.

To do this we have to setup travis in our repository. Travis can be easily set in github repository just by adding .travis.yml file into root directory your repo. Depending on type of repository you are dealing with, the configuration of travis changes. For accounts.susi.ai, It is a written on top of ReactJs framework. So we used the following code.

sudo: required
dist: trusty
language: node_js
node_js:
  - 6
script:
  - npm test
deploy:
  provider: script
  script: "./deploy.sh"
  env:
  global:
  - ENCRYPTION_LABEL: "<.... encryption label from previous step ....>"
  - COMMIT_AUTHOR_EMAIL: "[email protected]"
cache:
  directories:
    - node_modules
branches:
  only:
    - master

 

The above travis configuration file, After every commit checks whether the build is passing or not by running npm test command. After checking the commit, next script deploy.sh is run on the commit. The last line tells us to only execute this script for commits in master branch. deploy.sh can be placed anywhere, we jus need to modify its path in .tavis.yml file. Here is code of deploy.sh :

#!/bin/bash
set -e 

SOURCE_BRANCH="master"
TARGET_BRANCH="gh-pages"

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]; then
    echo "Skipping deploy; just doing a build."
    doCompile
    exit 0
fi
# Save some useful information
REPO=`git config remote.origin.url`
SSH_REPO=${REPO}
SHA=`git rev-parse --verify HEAD`

git clone $REPO out
cd out
git checkout $TARGET_BRANCH || git checkout --orphan $TARGET_BRANCH
cd ..
git config user.name "Travis CI"
git config user.email "[email protected]"

if git diff --quiet; then
    echo "No changes to the output on this push; exiting."
    exit 0
fi

# Commit the "changes", i.e. the new version.
# The delta will show diffs between new and old versions.
git add -A .
git commit -m "Deploy to GitHub Pages: ${SHA}"

# Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc
ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key"
ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv"
ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR}
ENCRYPTED_IV=${!ENCRYPTED_IV_VAR}
openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in ../deploy_key.enc -out ../deploy_key -d
chmod 600 ../deploy_key
eval `ssh-agent -s`
ssh-add deploy_key

# Now that we're all set up, we can push.
git push $SSH_REPO $TARGET_BRANCH

 

deploy.sh is automatically deploying master on gh pages branch. To perform this task, it needs admin authorisation of github repo. We do this authentication by encrypted keys.
To create encrypted keys we need to generate a new SSH key, SSH keys can be generated by running following command in terminal.

ssh-keygen -t rsa -b 4096 -C "[email protected]"

 

Then add this key to your project’s git repository at the github repo of project.

https://github.com/<your name>/<your repo>/settings/keys

Now, We can generate encrypted keys by running following command.

travis encrypt-file deploy_key

 

This command generates deploy_key.enc file, which shall be placed in the repo. Its location needs to be updated in the placeholders of .travis.yml and deploy.sh.

Hence, we achieved the task of auto deployment to gh-pages in accounts.susi.ai. Please ensure that account through which encrypted keys are created, always have admin access to repository for continued deployment.

Continue Reading Auto Deploying accounts.susi.ai on gh-pages

Adding Custom Scrollbar to SUSI AI Web Chat

Scrollbar represents the depth of content on your current screen. It appears when the content has overflown the depth of screen and cannot fit it anymore. We see scrollbars everywhere. By default, the scrollbar provided by the browser is not very attractive but efficient in doing its job.

We decided that as our SUSI.AI Web App is improving in both UI and functionality, let’s add a custom scrollbar to it.

Earlier we had a standard  scrollbar in our SUSI.AI webchat:

For adding a custom scrollbar to our web chat we decided to use react-custom-scrollbars npm-package.

Our reasons to choose this package were:

  • Auto Hide feature in the scrollbar, after a specific period of time, which we can modify too.
  • No requirement for extra CSS styles to style our scrollbar.
  • It is well tested and trusted by many developers in open source

To install this npm package:

npm install -S react-custom-scrollbars 

Now comes the usage part, we need to import this into our JavaScript file:

 import { Scrollbars } from 'react-custom-scrollbars';

After importing, wrap it around the data where you want to show a custom scrollbar. In our case it was messageListItems, the code snippet looked like:

<Scrollbars>
 {messageListItems}
</Scrollbars>

This made our scrollbar look much better than the default one:

Now to add Auto Hide functionality to our scrollbar we need to add some attributes to our <Scrollbars>  tag.

    1. autoHide: It allows the auto-hide feature to our scrollbar.
    2. autoHideTimeout: It allows us to set custom time of hiding delay of a scrollbar in milli-seconds.
    3. autoHideDuration: it allows us to set the duration of hiding animation in milliseconds.

After adding the above-mentioned attributes our code changes to:

<Scrollbars
 autoHide
 autoHideTimeout={1000}
 autoHideDuration={200}>
 {messageListItems}
</Scrollbars>

Resources:

A lot more of custom attributes can be found in the documentation of Malte Wessel here.

Testing Link:

Now we had a much better scrollbar for our web chat which can be tested here.

Continue Reading Adding Custom Scrollbar to SUSI AI Web Chat

How SUSI AI Searches the Web For You

SUSI is now capable of performing web search to answer your queries. When SUSI doesn’t know how to answer your queries, it performs a web search on the client side and displays all the results as horizontally swipeable tiles with each tile giving a brief description and also providing a link to the relevant source.

Lets visit SUSI WebChat and try it out.

Query : Search for Google
Response : <Web Search Results>

How does SUSI know when to perform a websearch?

It uses action types to identify if a web search is to be performed or not. The API Response is parsed to check for the action types and if a websearch action type is present, then an API call is made using the duckduckgo api with the relevant query and the results are displayed as tiles with :

  • Category : The Topic related to the given query
  • Text : The result from the websearch
  • Image : Image related to the query if present
  • Link : A url redirecting to the relevant source

Parsing the actions :

Let us look at the API response for a query.

Sample Query: search for google

Response: <API Response>

"actions": [
  {
    "type": "answer",
    "expression": "Here is a web search result:"
  },
  {
    "type": "websearch",
    "query": "google"
  }
]

Note: The API Response has been trimmed to show only the relevant content

We find a websearch type action and the query to be searched as google . So we now make a API call using duckduckgo api to get our websearch results.

API Call Format : api.duckduckgo.com/?q={query}&format=json

API Call for query google : http://api.duckduckgo.com/?q=google&format=json

And from the duckduckgo API response we generate our websearch tiles showing relevant data using the fields present in each object.

This is the sample object from duckduckgo API response under the RelatedTopics , which we use to create our websearch result tiles.

{
  "Result": "<a href=\"https:\/\/duckduckgo.com\/Google\">Google<\/a> An American multinational technology company specializing in Internet-related services and...",
  "Icon": {
    "URL": "https:\/\/duckduckgo.com\/i\/8f85c93f.png",
    "Height": "",
    "Width": ""
  },
  "FirstURL": "https:\/\/duckduckgo.com\/Google",
  "Text": "Google An American multinational technology company specializing in Internet-related services and..."
},

Let us look at the code for querying data from the API

if (actions.indexOf('websearch')>=0) {

  let actionIndex = actions.indexOf('websearch');
  let query = response.answers[0].actions[actionIndex].query;

  $.ajax({
    url: 'http://api.duckduckgo.com/?format=json&q='+query,
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,

    success: function (data) {
      receivedMessage.websearchresults = data.RelatedTopics;

      if(data.AbstractText){
        let abstractTile = {
          Text:'',
          FirstURL:'',
          Icon:{URL:''},
        }
        abstractTile.Text = data.AbstractText;
        abstractTile.FirstURL = data.AbstractURL;
        abstractTile.Icon.URL = data.Image;
        receivedMessage.websearchresults.unshift(abstractTile);
    }

    let message =  ChatMessageUtils.getSUSIMessageData(
receivedMessage, currentThreadID);

    ChatAppDispatcher.dispatch({
      type: ActionTypes.CREATE_SUSI_MESSAGE,
      message
    });
  },

    error: function(errorThrown) {
      console.log(errorThrown);
      receivedMessage.text = 'Please check your internet connection';
    }

  });

}

Here, from the actions object, we get the query needed to search the web. We then make a ajax call using that query to the duckduckgo API. If the API call succeeds then we collect the required data to create tiles as array of objects and store it as websearchresults. and dispatch the message with the websearchresults which gets reflected in the message store and when passed to the components we use it to create the result tiles.

<MuiThemeProvider>
  <Paper zDepth={0} className='tile'>
    <a rel='noopener noreferrer'
    href={tile.link} target='_blank'
    className='tile-anchor'>
    {tile.icon &&
    (<Paper className='tile-img-container'>
      <img src={tile.icon}
      className='tile-img' alt=''/>
     </Paper>
    )}
  <Paper className='tile-text'>
    <p className='tile-title'>
      <strong>
        {processText(tile.title,'websearch-rss')}
      </strong>
    </p>
    {processText(tile.description,'websearch-rss')}
  </Paper>
  }
  </a>
  </Paper>
</MuiThemeProvider>

We then display the tiles as horizontally swipeable carousel ensuring a good and interactive UX.

React-Slick module was used to implement the horizontal swiping feature.

function renderTiles(tiles){

if(tiles.length === 0){
  let noResultFound = 'NO Results Found';
  return(<center>{noResultFound}</center>);
}

let resultTiles = drawTiles(tiles);

var settings = {
  speed: 500,
  slidesToShow: 3,
  slidesToScroll: 1,
  swipeToSlide:true,
  swipe:true,
  arrows:false
};

return(
    <Slider {...settings}>
      {resultTiles}
    </Slider>
);

}

Here we are handling the corner case when there are no results to display by rendering `NO Results found`. We then have our web search results displayed as swipeable tiles with a image, title, description and link to the source.

This is how SUSI performs web search to respond to user queries ensuring that no query goes unanswered! Don’t forget to swipe left and go through all the results displayed!

Resources

Continue Reading How SUSI AI Searches the Web For You

How SUSI AI Tabulates Answers For You

SUSI is an artificial intelligence chat bot that responds to all kinds of user queries. It isn’t any regular chat bot replying in just plain text. It supports various response types which we refer to as ‘actions’. One such action is the “table” type. When the response to a user query contains a list of answers which can be grouped, it is better visualised as a table rather than plain text.

Lets visit SUSI WebChat and try it out. In our example we ask SUSI for the 2009 race statistics of British Formula 1 racing driver Lewis Hamilton.

Query: race stats of hamilton in f1 season 2009

Response: <table> (API response)

 

 

How does SUSI do that? Let us look at the skill teaching SUSI to give table responses.

# Returns race stats as a table

race summary of  * in f1 season *|race stats of  * in f1 season *
!console:
{
  "url":"http://ergast.com/api/f1/$2$/drivers/$1$/status.json",
  "path":"$.MRData.StatusTable.Status",
  "actions":[{
     "type":"table",
     "columns":{"status":"Race Status","count":"Number Of Races"}
   }]
}
eol

Here, we are telling SUSI that the data type is a table through type attribute in actions and also defining column names and under which column each value must be put using their respective keys. Using this information SUSI generates a response accordingly with the table schema and data points.

How do we know when to render a table?

We know it through the type attribute in the actions from the API response.

"actions": [{
  "type": "table",
  "columns": {
    "status": "Race Status",
    "count": "Number Of Races"
  },
  "count": -1
  }]
}],

We can see that the type is table so we now know that we have to render a table.

But what is the table schema? What do we fill it with?

There is a columns key under actions and from the value of the columns key we get a object whose key value pairs give us column names and what data to put under each column.

Here, we have two columns – Race Status and Number Of Races

And the data to put under each column is found in answers[0].data with same keys as those for each column name i.e ‘status’ and ‘count’.

Sample data object from the API response:

{
  "statusId": "2",
  "count": "1",
  "status": "Disqualified"
}

The value under ‘status’ key is ‘Disqualified’ and the column name for ‘status’ key is ‘Race Status’, so Disqualified is entered under Race Status column in the table. Similarly 1  is entered under Number Of Races column. We thus have a row of our table. We populate the table for each object in the data array using the same procedure.

let coloumns = data.answers[0].actions[index].columns;
let count = data.answers[0].actions[index].count;
let table = drawTable(coloumns,data.answers[0].data,count);

We also have a ’count’ attribute in the API response . This is used to denote how many rows to populate in the table. If count = -1 , then it means infinite or to display all the results.

function drawTable(coloumns,tableData,count){

let parseKeys;
let showColName = true;

if(coloumns.constructor === Array){
  parseKeys = coloumns;
  showColName = false;
}
else{
  parseKeys = Object.keys(coloumns);
}

let tableheader = parseKeys.map((key,i) =>{
return(<TableHeaderColumn key={i}>{coloumns[key]}</TableHeaderColumn>);
});

let rowCount = tableData.length;

if(count > -1){
  rowCount = Math.min(count,tableData.length);
}

let rows = [];

for (var j=0; j < rowCount; j++) {

  let eachrow = tableData[j];

  let rowcols = parseKeys.map((key,i) =>{
    return(
        <TableRowColumn key={i}>
          <Linkify properties={{target:'_blank'}}>
            {eachrow[key]}
          </Linkify>
        </TableRowColumn>
      );
  });

  rows.push(
      <TableRow key={j}>{rowcols}</TableRow>
  );

}

const table =
  <MuiThemeProvider>
    <Table selectable={false}>
      <TableHeader displaySelectAll={false} adjustForCheckbox={false}>
        { showColName && <TableRow>{tableheader}</TableRow>}
      </TableHeader>
      <TableBody displayRowCheckbox={false}>{rows}</TableBody>
    </Table>
  </MuiThemeProvider>

return table;

}

Here we first determine how many rows to populate using the count attribute and then parse the columns to get the column names and keys. We then loop through the data and populate each row.

This is how SUSI responds with tabulated data!

You can create your own table skill and SUSI will give the tabulated response you need. Check out this tutorial to know more about SUSI and the various other action types it supports.

Resources

Continue Reading How SUSI AI Tabulates Answers For You

How to Make SUSI AI Slack Bot

To make SUSI slack bot we will use real time messaging api of slack which will allow users to receive messages from bot in real time. To make SUSI slack bot you have to follow following steps:

Steps:

  1. First of all you have to create a team on slack in where your bot will be running. To create a team go to https://slack.com/ and create a new team.
  2. After creating sign in to your team and got to apps and integration option by clicking on left corner.
  3. Click manage on top right corner and go to custom integrations to add configuration to Bots.
  4. After adding configuration data,bot username and copying API Token now we have to write code for setting bot in slack. To set up code see below steps: 
  5. Install Node.js from the link below on your computer if you haven’t installed it already. https://nodejs.org/en/
  6. Create a folder with any name and open shell and change your current directory to the new folder you created.
  7. Type npm init in command line and enter details like name, version and entry point.
  8. Create a file with the same name that you wrote in entry point in above given step. i.e index.js and it should be in same folder you created.
  9. Type following commands in command line  npm install –save @slack/client. After slack/client is installed type npm install –save express after express is installed type npm install –save request and then npm install –save http when all the modules are installed check your package.json modules will be included within dependencies portion.
  10. Your package.json file should look like this.
    {
    "name": "slack-bot",
    "version": "1.0.0",
    "description": "SUSI Slack Bot",
    "main": "index.js",
    "dependencies": {
           "express": "^4.15.3",
           "http": "0.0.0",
           "request": "^2.81.0"
    },
    "scripts": {
           "test": "echo \"Error: no test specified\" && exit 1",
           "start": "node index.js"
    }
    }
    
  11. Copy following code into file you created i.e index.js.
    var Slack = require('@slack/client');
    var request = require('request');
    var express = require('express');
    var http = require('http');
    var app = express();
    var RtmClient = Slack.RtmClient; 
    var RTM_EVENTS = Slack.RTM_EVENTS;
    var token = process.env.APIToken;
    
    var rtm = new RtmClient(token, { logLevel: 'info' }); 
    rtm.start();
    
    //to ping heorku app after 20 minutes to keep it active
    
    setInterval(function() {
            http.get(process.env.HerokuUrl);
        }, 1200000);
    
    rtm.on(RTM_EVENTS.MESSAGE, function(message) { 
    var channel = message.channel;
    
    var options = {
           method: 'GET',
           url: 'http://api.asksusi.com/susi/chat.json',
           qs: {
               timezoneOffset: '-330',
               q: message.text
           }
       };
    
    //sending request to SUSI API for response
       request(options, function(error, response, body) {
           if (error) throw new Error(error);
           var ans = (JSON.parse(body)).answers[0].actions[0].expression;
           rtm.sendMessage(ans, channel);
       })
    });
    
    const port = process.env.PORT || 3000;
    app.listen(port, () => {
       console.log(`listening on ${port}`);
    });
     
    
    


  12. Now we have to deploy this code to heroku.
  13. Before deploying we have to make a github repository for chatbot to make github repository follow these steps:

    In command line change current directory to folder we created above and write

    git init
    git add .
    git commit -m”initial”
    git remote add origin <URL for remote repository>
    git remote -v
    git push -u origin master

    You will get URL for remote repository by making repository on your github and copying this link of your repository.

  14. To deploy your bot to heroku you need an account on Heroku and after making an account make an app.
  15. Deploy app using github deployment method.
  16. Select Automatic deployment method.
  17. Add APIToken and HerokuUrl variable to heroku app in settings options.
  18. Your SUSI Slack bot is ready enjoy chatting with it.If you want to learn more about slack API refer to https://api.slack.com
Continue Reading How to Make SUSI AI Slack Bot

Implementing Real Time Speech to Text in SUSI Android App

Android has a very interesting feature built in called Speech to Text. As the name suggests, it converts user’s speech into text. So, android provides this feature in form of a recognizer intent. Just the below 4 line code is needed to start recognizer intent to capture speech and convert into text. This was implemented earlier in SUSI Android App too.  Simple, isn’t it?

Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
       RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,
       "com.domain.app");
startActivityForResult(intent, REQ_CODE_SPEECH_INPUT);

But there is a problem here. It popups an alert dialog box to take a speech input like this :

Although this is great and working but there were many disadvantages of using this :

  1. It breaks connection between user and app.
  2. When user speaks, it first listens to complete sentence and then convert it into text. So, user was not able to see what the first word of sentence was until he says the complete sentence.
  3. There is no way for user to check if the sentence he is speaking is actually converting correctly to speech or not. He only gets to know about it when the sentence is completed.
  4. User can not cancel the conversion of Speech to text if some word is converted wrongly because the user doesn’t even get to know if the word is converted wrongly. It is kind of suspense that a whether the conversion is right or not.

So, now the main challenge was how can we improve the Speech to Text and implement it something like Google now?  So, what google now does is, it converts speech to text alongside taking input making it a real time speech to text convertor. So, basically if you speak second word of sentence, the first word is already converted and displayed. You can also cancel the conversion of rest of the sentence if the first word of sentence is converted wrongly.

I searched this problem a lot on internet, read various blogs and stackoverflow answers, read Documentation of Speech to Text until I finally came across SpeechRecognizer class and RecognitionListener. The plan was to use object of SpeechRecognizer class with RecognizerIntent and RecognitionListener to generate partial results.

So, I just added this line to the the above code snippet of recognizer intent.

intent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS,true);

And introduced SpeechRecognizer class and RecognitionListener like this

 SpeechRecognizer recognizer = SpeechRecognizer
   .createSpeechRecognizer(this.getApplicationContext());

RecognitionListener listener = new RecognitionListener() {
   @Override
   public void onResults(Bundle results) {
   }
 
   @Override
   public void onError(int error) {
   }
 
   @Override
   public void onPartialResults(Bundle partialResults) {
    //The main method required. Just keep printing results in this.
   }
};

Now combine SpeechRecognizer class with RecognizerIntent and RecognitionListener with these two lines.

recognizer.setRecognitionListener(listener);
recognizer.startListening(intent);

Everything is now set up. You can now use result in onPartialResults method and keep displaying partial results. So, if you want to speak “My name is…” , “My” will be displayed on screen by the time you speak “name” and “My name” will be displayed on screen by the time you speak “is” . Isn’t it cool?

This is how Realtime Speech to Text is implemented in SUSI Android App. If you want to implement this in your app too, just follow the above steps and you are good to go. So, the results look like these.

  

Resources

Continue Reading Implementing Real Time Speech to Text in SUSI Android App