SUSI.AI

Adding typing animation and messages in SUSI Web bot plugin

SUSI web plugin bot provides the features of SUSI as a chat window which users can plugin to their websites. They just need to copy the generated javascript code into their website and the SUSI bot plugin will be enabled. This blog explains about the process of creating messages from both the user side and the bot side, and adding typing animation in the SUSI web chat plugin. A live demo of the bot can be found at susi-chatbotplugin-demo.surge.sh.

Creating User’s message

The main javascript file of our concern is skills.susi.ai/public/susi-chatbot.js. When the user types their message and press enter, setUserResponse() function is executed:

This function adds a message box to the chat window. The message box contains the message of the user.
Result:

Adding typing animation

After the user types the message and the message box is displayed, setUserResponse() function is executed. This function sets up a message box from the bot’s side and fills it with a loading gif. The important thing to note is msgNumber variable. For each user message, this variable is incremented by one. So it keeps count of the total number of message from the user or the bot. Each message box from the bot is assigned a unique id: “susiMsg-<msgNumber>”. Thus, when the response from the SUSI server is received, the loading gif is replaced by the message from the server. The corresponding message box is identified by the above id.

This function adds a message box to the chat window containing the loading gif.
Result:

On receiving the response from the server, the following function is executed:

function setBotResponse(val,msgNumber) {
    val = val.replace(new RegExp('\r?\n','g'), '<br />');
    $("#susiMsg-"+msgNumber+" .susi-msg-content-div").text(val);
    scrollToBottomOfResults();
}

 

This function replaces the above loading gif with the server’s response.

Final result:

Resources

Continue ReadingAdding typing animation and messages in SUSI Web bot plugin

Deploying SUSI Zulip bot

Zulip is a popular Real time messaging system, which combines the immediacy of Slack with an email threading model. The SUSI Zulipbot is a custom chatbot for zulip platform which fetches the response from the SUSI Server and will have some additional zulip platform specific features too. Users can install the bot into their zulip workspaces and then interact with the bot. They can either talk to the bot in private message or talk in group channels. This blog walks through the process of deploying SUSI Zulip bot into your workspace.

Cloning python-zulip-api

Python-zulip-api is where all the bots being developed for the Zulip platform can be found. The SUSI bot can be found in zulip_bots/bots/susi. susi.py is the main file where the bot’s code resides. test_susi.py is the file where the test cases for the bot are written. To clone and run the bot locally, make sure you have python3, pip and virtualenv installed. Then follow the below steps:

  1. git clone https://github.com/zulip/python-zulip-api.git  – clone the python-zulip-api repository.
  2. cd python-zulip-api  – navigate into your cloned repository.
  3. python3 ./tools/provision  – install all requirements in a Python virtualenv.
  4. The output of provision  will end with a command of the form source …/activate; run that command to enter the new virtualenv.
  5. Finished. You should now see the name of your venv preceding your prompt, e.g. (zulip-api-py3-venv)

For more information about installing the repository, refer https://zulipchat.com/api/writing-bots

Testing the bot’s output locally

For quick testing of your bot’s output, zulip-terminal is a very useful tool. It provides you a testing environment inside your terminal. After installing the above requirements, run  zulip-terminal susi in your terminal to enable testing the bot’s output:

Enter your message: hi
Reply from the bot is printed between the dotted lines:
——-
Hello!
——-
Enter your message: tell me a joke
Reply from the bot is printed between the dotted lines:
——-
It is said that looking into Chuck Norris’ eyes will reveal your future. Unfortunately, everybody’s future is always the same: death by a roundhouse-kick to the face.
——-
Enter your message: who created you?
Reply from the bot is printed between the dotted lines:
——-
The FOSSASIA community created me
——-
Enter your message: ^C
Ok, if you’re happy with your terminal-based testing, try it out with a Zulip server.
You can refer to https://zulipchat.com/api/running-bots#running-a-bot.

Deploying the bot in a Zulip workspace

Now you can deploy the bot in your own workspace or even in chat.zulip.org workspace. Follow these steps:

  1. After logging to your workspace, go to Settings () -> Your bots -> Add a new bot. Select Generic bot for bot type, fill out the form and click on Create bot.
  2. A new bot user should appear in the Active bots panel.
  3. Download the bot’s zuliprc configuration file to your computer.
  4. Run  zulip-run-bot susi –config-file ~/zuliprc-my-bot (using the path to the zuliprc file from above step).
  5. Congrats! Your bot should be running. Talk to your bot with mentioning @susi in a group channel or directly in a private channel.

Resources

Continue ReadingDeploying SUSI Zulip bot

Adding author’s email in SUSI skill files

You can create SUSI skills by using skills.susi.ai. When you create a skill, a text file is created and stored in the server. A file can contain many skills inside it, and is thus called an “expert”. The text file contains the metadata like the author’s name, author’s url, description, etc, but it does not contains the author’s email id. Certainly, storing the author’s email along with the skill can be very useful to stuff like get skills for a particular author. But if we provide the option to the user to enter their email, it can easily be misused since they can add emails of other persons. This blog explains how the metadata in the skill file are send by the client and  interpreted by the server, and how the author’s email is added to the text file in a secured way.

Representing and storing metadata in the expert file

Let’s take a look at how the metadata is represented in the expert file. The lines containing the metadata are preceded with “::”. This lets the parser in the server to easily identify the metadata. Now, we can add a new line for the author_email. But if the user is asked to enter email themselves, they can easily add skills in the name of other authors. Thus we need to securely add this data.

::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::terms_of_use <link>

 

Extracting and adding the author’s email

When the user logs in, their email and access_token is stored in a “cookie”. The email id can be extracted by cookies.get(’emailId’) and the access token can be extracted by cookies.get(‘loggedIn’). Our approach will be to extract the email id, and append it at the top of the expert file.

This how the email is appended and the file is being send to the server in CreateSkill.js file:

let code = this.state.code;
code = '::author_email ' + cookies.get('emailId') + '\n' + code;
let form = new FormData();
form.append('model', 'general');
form.append('group', this.state.groupValue);
form.append('language', this.state.languageValue);
form.append('skill', this.state.expertValue.trim().replace(/\s/g,'_'));
form.append('image', this.state.file);
form.append('content', code);
form.append('image_name', this.state.imageUrl.replace('images/',''));
form.append('access_token', cookies.get('loggedIn'));

let settings = {
    'async': true,
    'crossDomain': true,
    'url': urls.API_URL + '/cms/createSkill.json',
    'method': 'POST',
    'processData': false,
    'contentType': false,
    'mimeType': 'multipart/form-data',
    'data': form
};

 

Making changes in the server to store the author’s email

Since we have just added the author_email in the text file itself similar to the other metadata, we don’t have to do much structural changes in the server side. We just have to add code to detect the ‘author_email’ metadata similar to the other metadata.

In SusiSkill.java file, we add author_email attribute where-ever the other meta data are already present.

And in SusiMind.java file, we add the following lines in the learn() function:

if(json.has("author_email"))
 skill.setAuthorEmail(json.getString("author_email"));

 

Thus, the author’s email is successfully stored in the server along with the other meta-data.

Resources

Continue ReadingAdding author’s email in SUSI skill files
Read more about the article Hosting SUSI.AI Web Chat on Firebase

Hosting SUSI.AI Web Chat on Firebase

“I often wonder if my smartphone or laptop could become a bit smarter… And, I could talk to it the way I talk to my best friend… Wouldn’t that be cool if I could customise my phone’s assistant and enhance its ‘personality’ whenever I want? Or maybe ‘teach it a little more than others to make its intelligence outshine other students in its class?’ Well, that’s what we aim to achieve in the name of SUSI.AI”

Currently, SUSI.AI is in its growing stage and a number of dedicated developers have been working hard to achieve an open source alternative to existing personal assistants like Google Assistant, Alexa and so on. Each one of them has been focussing on improving existing features or introducing new ones.

This blog focuses on hosting the SUSI Webchat using Firebase Hosting. It presents the proper sequence of steps that need to be followed to deploy the web chat on Firebase Hosting.

What is Firebase Hosting?

Firebase Hosting provides fast and secure static hosting for web app. It is a  production-grade web content hosting for developers. With Hosting, one can quickly and easily deploy web apps and static content to a global content-delivery network (CDN) with a single command.

Hosting gives a project a subdomain on the ‘firebaseapp.com’ domain. Using the Firebase CLI, the files can be deployed from local directories on a computer to the Hosting server. Files are served over an SSL connection from the closest edge server on the global CDN.

Steps :

  • Install the Firebase CLI

    The Firebase CLI (Command Line Interface) requires Node.js and npm, which can both be installed by following the instructions on https://nodejs.org/. Installing Node.js also installs npm.

    Note : The Firebase CLI requires Node.js version 5.10.0 or greater.

Once Node.js and npm have been installed, install the Firebase CLI via npm:

$ npm install -g firebase-tools


This installs the globally available firebase command. To update to the latest version, simply re-run the same command.

Now login to the Firebase console using 

$ firebase login

 

  • Initialize the app

Choose the Firebase app that you would like to deploy and then cd into the project directory and run the command:

$ firebase init


Once this command runs, you would be prompted to choose from the following options :

Database: Deploy Firebase Realtime Database Rules
Firestore: Deploy rules and create indexes for Firestore
Functions: Configure and deploy Cloud Functions
Hosting: Configure and deploy Firebase Hosting sites
Storage: Deploy Cloud Storage security rules

Choose Hosting (use the cursor keys to move the pointer to ‘Hosting’ and then select it by pressing Spacebar key followed by the Enter key).

Now, you would be prompted to select a project from the Firebase console that you would like to associate with the web chat. You can create a new project by logging into the Firebase console from a web browser and then follow the steps below :

1) If you don’t have an existing Firebase project, click ‘Add Project’ and enter either an existing Google Cloud Platform project name or a new project name.

2) If you already have apps in your project, click ‘Add Another App’ from the project overview page.

Running the $ firebase init command creates a firebase.json settings file in the root of your project directory.

  • Add a file

    When you initialise the app, you will be prompted for a directory to use as the public root (default is “public”). You can choose to give it some other name as well, say “build”. This is the directory in which the static content would be hosted. If you don’t already have a valid index.html file in your public root directory, one is created for you.

Note : When a new index.html file is created, it contains the default content. You need to run the following commands to customise it according to your requirements and to display the content that you want to.

$ npm install
$ npm run build


If you skip running these commands, on deploying you will see a dialog box, by default, that would direct you to the Firebase Hosting Documentation.

Test on Local Server

Now you can run the $ firebase serve command to test your web app on a local server. Once, everything looks fine, you can proceed to the next step.

  • Deploy the web app on Firebase

    To deploy the web app, simply run:
$ firebase deploy

 

Your app will be deployed to the domain
<OUR-FIREBASE-APP>.firebaseapp.com

Manage and Rollback Deploys

From the Hosting panel in the Firebase Console you can see a full history of your deploys. To rollback to a previous deploy, hover over its entry in the list, click the overflow menu icon, then click “Rollback”.
Now your app is ready to share with the world! 😀

Check out a video created by me, showing the above mentioned steps : Click here

Resources

Continue ReadingHosting SUSI.AI Web Chat on Firebase
Read more about the article Setup interactive charts for data representation

Setup interactive charts for data representation

At the end of this blog, you would be able to setup interactive charts using HighCharts and D3.js. As the charts/data-visualisation models will form the backbone of the upcoming SUSI.AI Analytics dashboard, as well as the data representational model for various useful data. For the purpose of integration with the SUSI Skills CMS project, we will be using the react-highcharts and react-tagcloud library.

There are various kinds of charts and plots that HighCharts offers. They are –

  • Line charts
  • Area charts
  • Column and Bar Charts
  • Pie Charts
  • Scatter and Bubble Charts
  • Combinations
  • Dynamic Charts
  • Gauges
  • Heat and Tree maps
  • 3D charts

Check out this link for the full list.

We would be aiming to build up the above charts for analysis of Term Frequency Trends and Trending Clouds.

For the Term Frequency Trends, we will need to setup a Basic Line Graph and for the later,  we need a World Cloud.

Setting up a Basic Line Graph

The aim is to setup a basic line graph and to accomplish that we use a react library called react-highcharts, which makes our work very easier.

Firstly, we create an object config that contains the labels and the required data, with the key values as mentioned in the API reference. The object looks like this –

const config = {
  xAxis: {
    categories: ['01/13', '01/14', '01/15', '01/16', '01/17', '01/18', '01/19', '01/20']
  },
  series: [{
    data: [750, 745, 756, 740, 760, 752, 765]
  }]
};

Secondly, we create a React Component and pass the config object as a property to the ReactHighcharts component.

Finally, we render the component in a div of the index.html file, and the following output is achieved.

The code for the component that renders the Chart is as follows:

import React from 'react';
import ReactHighcharts from 'react-highcharts';
import ReactDOM from 'react-dom';

const config = {
  xAxis: {
    categories: ['01/13', '01/14', '01/15', '01/16', '01/17', '01/18', '01/19', '01/20']
  },
  series: [{
    data: [750, 745, 756, 740, 760, 752, 765]
  }]
};

ReactDOM.render(<ReactHighcharts  config={config} />, document.getElementById('app'));

Setting up a WordCloud

Here, we wish to setup a WordCloud that would show the different words that got searched or the top trending words. We would be using the react-tagcloud library for this.

Firstly, we create an object data that contains the text along with the count/frequency of search. The object looks like this –

const data = [
  { value: "JavaScript", count: 38 },
  { value: "React", count: 30 },
  { value: "Nodejs", count: 28 },
  { value: "Express.js", count: 25 },
  { value: "HTML5", count: 33 },
  { value: "MongoDB", count: 18 },
  { value: "CSS3", count: 20 }
];

Secondly, we create a React Component and pass the data object as a property to the TagCloud component.

Finally, we render the component in a div of the index.html file, and the following output is achieved.

The code for the component that renders the Chart is as follows:

import React from 'react';
import React, { Component } from 'react';
import { TagCloud } from "react-tagcloud";
 
const data = [
  { value: "JavaScript", count: 38 },
  { value: "React", count: 30 },
  { value: "Nodejs", count: 28 },
  { value: "Express.js", count: 25 },
  { value: "HTML5", count: 33 },
  { value: "MongoDB", count: 18 },
  { value: "CSS3", count: 20 }
];
 
const SimpleCloud = () => (
  <TagCloud 
       minSize={12}
       maxSize={35}
       tags={data}
       onClick={tag => alert(`'${tag.value}' was selected!`)} />
);

ReactDOM.render(<SimpleCloud />, document.getElementById('app'));

 

These were some examples of setting up some of the data-visualization models, that would form the basic building block of the SUSI Analytics project. I hope this blogs would be a good starting point for those wanting to start with setting up charts, graphs, etc.

Resources

Continue ReadingSetup interactive charts for data representation
Read more about the article Skill Development using SUSI Skill CMS

Skill Development using SUSI Skill CMS

There are a lot of personal assistants around like Google Assistant, Apple’s Siri, Windows’ Cortana, Amazon’s Alexa, etc. What is then special about SUSI.AI which makes it stand apart from all the different assistants in the world? SUSI is different as it gives users the ability to create their own skills in a Wiki-like system. You don’t need to be a developer to be able to enhance SUSI. And, SUSI is an Open Source personal assistant which can do a lot of incredible stuff for you, made by you.

So, let’s say you want to create your own Skill and add it to the existing SUSI Skills. So, these are the steps you need to follow regarding the same –

  1. The current SUSI Skill Development Environment is based on an Etherpad. An Etherpad is a web-based collaborative real-time editor. https://dream.susi.ai/ is one such Etherpad. Open https://dream.susi.ai/ and name your dream (in lowercase letters).
  2. Define your skill in the Etherpad. The general skill format is 

::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::term_of_use <link>

#Intent
User query1|query2|query3....
Answer answer1|answer2|answer3...

 

Patterns in query can be learned easily via this tutorial.

  1. Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad.
  2. Once you’ve tested your skill, write ‘stop dreaming’ to disable dreaming for SUSI.
  3. If the testing was successful and you want your skill to be added to SUSI Skills, send a Pull Request to susi_skill_data repository providing your dream name.

How do you modify an existing skill?

SUSI Skill CMS is a web interface where you can modify the skills you’ve made. All the skills of SUSI are directly in sync with the susi_skill_data.

To edit any skill, you need to follow these steps –

  1. Login to SUSI Skill CMS website using your email and password (or Sign Up to the website if you haven’t already).
  2. Click on the skill which you want to edit and then click on the “edit” icon.
  3. You can edit all aspects of the skill in the next state. Below is a preview:

Make the changes and then click on “SAVE” button to save the skill.

What’s happening Behind The Scenes in the EDIT process?

  • SkillEditor.js is the file which is responsible for keeping a check over various validations in the Skill Editing process. There are certain validations that need to be made in the process. Those are as follows –
  • Check whether User has logged in or not

if (!cookies.get('loggedIn')) {
            notification.open({
                message: 'Not logged In',
                description: 'Please login and then try to create/edit a skill',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });
            this.setState({
                loading: false
            });
            return 0;
        }

 

  • Check whether Commit Message has been entered by User or not

if (this.state.commitMessage === null) {
            notification.open({
                message: 'Please add a commit message',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });

            this.setState({
                loading: false
            });
            return 0;
        }

 

  • Check to ensure that request is sent only if there are some differences in old values and new values

if (this.state.oldGroupValue === this.state.groupValue &&
          this.state.oldExpertValue === this.state.expertValue &&
          this.state.oldLanguageValue === this.state.languageValue &&
          !this.state.codeChanged && !this.state.image_name_changed) {
            notification.open({
                message: 'Please make some changes to save the Skill',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });
            self.setState({
                loading: false
            });
            return 0;
        }

 

  • After doing the above validations, a request is sent to the Server and the User is shown a notification accordingly, whether the Skill has been uploaded to the Server or there has been some error.

$.ajax(settings)
            .done(function (response) {
                this.setState({
                    loading: false
                });
                let data = JSON.parse(response);
                if (data.accepted === true) {
                    notification.open({
                        message: 'Accepted',
                        description: 'Your Skill has been uploaded to the server',
                        //success/>
                    });
                }
                else {
                    this.setState({
                        loading: false
                    });
                    notification.open({
                        message: 'Error Processing your Request',
                        description: String(data.message),
                        //failure />
                    });
                }
            }

 

  • If the User is notified with a Success notification, then to verify whether the Skill has been added or not, the User can go to susi_skill_data repo and see if he has a recent commit regarding the same or not.

Resources

Continue ReadingSkill Development using SUSI Skill CMS
Read more about the article Making a SUSI Skill to get details about bank from IFSC

Making a SUSI Skill to get details about bank from IFSC

We are going to make a SUSI skill that fetches information about a bank when the IFSC (Indian Financial System Code) is known. Here is a detailed explanation of how we going about doing this.

Getting started with the skill creation

API endpoint that returns the bank details

Before going to the skill development, we need to find an API that would return the bank details from the IFSC, On browsing through various open source projects. I found an apt endpoint by Razorpay. Razorpay is a payment gateway for India which allows businesses to accept, process and disburse payments with ease. The Github link  to the repository is https://github.com/razorpay/ifsc.

API endpoint –  https://ifsc.razorpay.com/<:ifsc>
Request type –  GET
Response type –  JSON

Now, head over to the SUSI Etherpad, which is the current SUSI Skill Development Environment and create a new Pad. 

Here, we need to define the skill in the Etherpad. We will now write rules/intents for the skill. An intent represents an action that fulfills a user’s spoken request.

Intents consist of 2 parts –

  • User query – It contains different patterns of query that user can ask.
  • Answer – It contains the possible answer to the user query.

The main intent that our skill focuses on is, returning the bank name and address from the IFSC code. Here is how it looks –

Name of bank with IFSC code * | Bank's name with IFSC code *
!example:Name of bank with IFSC code SBIN0007245
!expect: The name of bank is State Bank of India
!console:The name of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.BANK"
}
eol

Part-wise explanation of the intent

  • The first line contains the query pattern that the user can use while querying. You can see that a wildcard character (*) is used in the pattern. It contains the IFSC of the bank that we wish to know, and will later on use to fetch the details via the API.
  • The second line contains an example query, followed by third line that contains the expected answer.
  • Last part of the rule contains the answer that is fetched from an external API –  https://ifsc.razorpay.com/<:ifsc>  ,via the console service  provided by SUSI Skills. Here, <:ifsc> refers to the IFSC that the user wants to know about. We get it from the user query itself, and can access it by the variable name $1$ as it matches with the 1st wildcard present in the query. If there would be 2 wildcards, we could have accessed them by $1$ and $2$ respectively.
  • The console service provides us with an option to enter the url of the API that we want to hit and path of the key we want to use.

The sample response of the endpoint looks like this :

{
  "BANK": "Karnataka Bank",
  "IFSC": "KARB0000001",
  "BRANCH": "RTGS-HO",
  "ADDRESS": "REGD. & HEAD OFFICE, P.B.NO.599, MAHAVEER CIRCLE, KANKANADY, MANGALORE - 575002",
  "CONTACT": "2228222",
  "CITY": "DAKSHINA KANNADA",
  "RTGS": true,
  "DISTRICT": "MANGALORE",
  "STATE": "KARNATAKA"
}

 

  • Since, we want to extract the name of the bank, the BANK key contains our desired value and we will use $.BANK in the path of the console service. And it can be accessed by $object$ in the answer. We frame the answer using $object$ and $1$ variables, and it like the one mentioned in the expected answer. eol marks the end of the console service.
  • Similarly, the intent that gives us the address of the bank looks like this – 
Address of bank with IFSC code * | Bank's address with IFSC code *
!example:Address of bank with IFSC code SBIN0007245
!expect: The address of bank is TILAK ROAD HAKIMPARA, P.O.SILIGURI DARJEELING, WEST BENGAL ,PIN - 734401
!console:The address of bank with IFSC code $1$ is $object$
{
  "url":"https://ifsc.razorpay.com/$1$",
  "path":"$.BANK"
}
eol

Testing the skill

  • Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. We will write down dream ifsc. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad.
  • We can test the skills by asking queries and matching it with the expected answer. Once the testing is done, write stop dreaming to disable dreaming for SUSI.

  • After the testing was successful completely, we will go ahead and add it to the susi_skill_data.
  • The general skill format is – 
::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::term_of_use <link>

#Intent
User query1|query2|query3....
Answer answer1|answer2|answer3...

We will add the basic skill details and author details to the etherpad file and make it in the format as mentioned above. The final text file looks like this – 

::name IFSC to Bank Details
::author Akshat Garg
::author_url https://github.com/akshatnitd
::description It is a bank lookup skill that takes in IFSC code from the user and provides you all the necessary details for the Bank. It is valid for banks in India only
::dynamic_content Yes
::developer_privacy_policy 
::image images/download.jpeg
::terms_of_use 

Name of bank with IFSC code * | Bank's name with IFSC code *
!example:bank with IFSC code *
!expect: The name of bank is SBI
!console:The name of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.BANK"
}
eol

Address of bank with IFSC code * | Bank's address with IFSC code *
!example:Address of bank with IFSC code *
!expect: The address of bank is 
!console:The address of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.ADDRESS"
}
eol

Submitting the skill

The final part is adding the skill to the list of skills for SUSI. We can do it by 2 ways:

1st method (using the web interface)

  • Open https://susi.skills.com and login into SUSI account (or sign up, if not done).
  • Click on the create skill button.
  • Select the appropriate fields like Category, Language, Skill name, Logo.
  • Paste the text file that we had created.
  • Add comments regarding the skill and click on Save to save the skill.

2nd method (sending a PR)

  • Send a Pull Request to susi_skill_data repository providing the dream name. The PR should have the text file containing the skill.

So, this was a short blog on how we can develop a SUSI skill of our choice.

Resources

Continue ReadingMaking a SUSI Skill to get details about bank from IFSC
Read more about the article Creating Onboarding Screens for SUSI iOS

Creating Onboarding Screens for SUSI iOS

Onboarding screens are designed to introduce users to how the application works and what main functions it has, to help them understand how to use it. It can also be helpful for developers who intend to extend the current project.

When you enter in the SUSI iOS app for the first time, you see the onboarding screen displaying information about SUSI iOS features. SUSI iOS is using Material design so the UI of Onboarding screens are following the Material design.

There are four onboarding screens:

  1. Login (Showing the login features of SUSI iOS) – Login to the app using SUSI.AI account or else signup to create a new account or just skip login.
  2. Chat Interface (Showing the chat screen of SUSI iOS) – Interact with SUSI.AI asking queries. Use microphone button for voice interaction.
  3. SUSI Skill (Showing SUSI Skills features) – Browse and try your favorite SUSI.AI Skill.
  4. Chat Settings (SUSI iOS Chat Settings) – Personalize your chat settings for the better experience.

Onboarding Screens User Interface

 

There are three important components of every onboarding screen:

  1. Title – Title of the screen (Login, Chat Interface etc).
  2. Image – Showing the visual presentation of SUSI iOS features.
  3. Description – Small descriptions of features.

Onboarding screen user control:

  • Pagination – Give the ability to the user to go next and previous onboarding screen.
  • Swiping – Left and Right swipe are implemented to enable the user to go to next and previous onboarding screen.
  • Skip Button – Enable users to skip the onboarding instructions and go directly to the login screen.

Implementation of Onboarding Screens:

  • Initializing PaperOnboarding:
override func viewDidLoad() {
super.viewDidLoad()

UIApplication.shared.statusBarStyle = .lightContent
view.accessibilityIdentifier = "onboardingView"

setupPaperOnboardingView()
skipButton.isHidden = false
bottomLoginSkipButton.isHidden = true
view.bringSubview(toFront: skipButton)
view.bringSubview(toFront: bottomLoginSkipButton)
}

private func setupPaperOnboardingView() {
let onboarding = PaperOnboarding()
onboarding.delegate = self
onboarding.dataSource = self
onboarding.translatesAutoresizingMaskIntoConstraints = false
view.addSubview(onboarding)

// Add constraints
for attribute: NSLayoutAttribute in [.left, .right, .top, .bottom] {
let constraint = NSLayoutConstraint(item: onboarding,
attribute: attribute,
relatedBy: .equal,
toItem: view,
attribute: attribute,
multiplier: 1,
constant: 0)
view.addConstraint(constraint)
}
}

 

  • Adding content using dataSource methods:

    let items = [
OnboardingItemInfo(informationImage: Asset.login.image,
title: ControllerConstants.Onboarding.login,
description: ControllerConstants.Onboarding.loginDescription,
pageIcon: Asset.pageIcon.image,
color: UIColor.skillOnboardingColor(),
titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.chat.image,
title: ControllerConstants.Onboarding.chatInterface,
description: ControllerConstants.Onboarding.chatInterfaceDescription,
pageIcon: Asset.pageIcon.image,
color: UIColor.chatOnboardingColor(),
titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skill.image,
title: ControllerConstants.Onboarding.skillListing,
description: ControllerConstants.Onboarding.skillListingDescription,
pageIcon: Asset.pageIcon.image,
color: UIColor.loginOnboardingColor(),
titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skillSettings.image,
title: ControllerConstants.Onboarding.chatSettings,
description: ControllerConstants.Onboarding.chatSettingsDescription,
pageIcon: Asset.pageIcon.image,
color: UIColor.iOSBlue(),
titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont)]
extension OnboardingViewController: PaperOnboardingDelegate, PaperOnboardingDataSource {
func onboardingItemsCount() -> Int {
return items.count
}

func onboardingItem(at index: Int) -> OnboardingItemInfo {
return items[index]
}
}

     

  • Hiding/Showing Skip Buttons: 
    func onboardingWillTransitonToIndex(_ index: Int) {
skipButton.isHidden = index == 3 ? true : false
bottomLoginSkipButton.isHidden = index == 3 ? false : true
}

Resources:

Continue ReadingCreating Onboarding Screens for SUSI iOS
Read more about the article Maintaining Extension State in SUSI.AI Chrome Bot Using Chrome Storage API

Maintaining Extension State in SUSI.AI Chrome Bot Using Chrome Storage API

SUSI Chrome Bot is a browser action chrome extension which is used to communicate with SUSI AI.The browser action extension in chrome is like any other web app that you visit. It will store all your preferences like theme settings and speech synthesis settings and data till you are interacting with it, but once you close it, it forgets all of your data unless you are saving it in some database or you are using cookies for the session. We want to be able to save the chats and other preferences like theme settings the user makes when interacting with SUSI AI through Susi Chrome Bot. In this blog, we’ll explore Chrome’s chrome.storage API for storing data.

What options do we have for storing data offline?

IndexedDB: IndexedDB is a low-level API for client-side storage of data. IndexedDB allows us to store a large amount of data and works like RDBMS but IndexedDB is javascript based Object-oriented database.

localStorage API: localStorage allows us to store data in key/value pairs which is much more effective than storing data in cookies. localStorage data persists even if the user closes and reopens the browser.

Chrome.storage: Chrome provides us with chrome.storage. It provides the same storage capabilities as localStorage API with some advantages.

For susi_chromebot we will use chrome.storage because of the following advantages it has over the localstorage API:

  1. User data can be automatically synced with Chrome sync if the user is logged in.
  2. The extension’s content scripts can directly access user data without the need for a background page.
  3. A user’s extension settings can be persisted even when using incognito mode.
  4. It’s asynchronous so bulk read and write operations are faster than the serial and blocking localStorage API.
  5. User data can be stored as objects whereas the localStorage API stores data in strings.

Integrating chrome.storage to susi_chromebot for storing chat data

To use chrome.storage we first need to declare the necessary permission in the extension’s manifest file. Add “storage” in the permissions key inside the manifest file.

"permissions": [
         "storage"
       ]

 

We want to store the chat user has made with SUSI. We will use a Javascript object to store the chat data.

var storageObj = {
senderClass: "",
content: ""
};

The storageObj object has two keys namely senderClass and content. The senderClass key represents the sender of the message(user or susi) whereas the content key holds the actual content of the message.

We will use chrome.storage.get and chrome.storage.set methods to store and retrieve data.

var susimessage = newDiv.innerHTML;
storageObj.content = susimessage;
storageObj.senderClass = "susinewmessage";
chrome.storage.sync.get("message",(items) => {
if(items.message){
storageArr = items.message;
}
storageArr.push(storageObj);
chrome.storage.sync.set({"message":storageArr},() => {
console.log("saved");
});
});

 

In the above code snippet, susimessage contains the actual message content sent by the SUSI server. We then set the correct properties of the storageObj object that we declared earlier. Now we can use chrome.storage.set to save the storageObj object but that would overwrite the current data that we have inside chrome’s StorageArea. To prevent the old message data from getting overwritten, we’ll first get all the message content in our storage using chrome.storage.sync.get. Notice how we are passing the “message” string as the first perimeter to the function. This is done because we only want our message content which was saved in the StorageArea. If we pass null instead, it will return all the content inside storageArea. Once we have our messages (which will be an array of objects that we store as storageObj), we will store that into a new array storageArr. We will then push our new storageObj that contains the message and the sender into the array. Finally, we use chrome.storage.sync.set to save the message content in chrome’s StorageArea which can later be retrieved using the “message” key.

storageArr.push(storageObj);
chrome.storage.sync.set({"message":storageArr},() => {
console.log("saved");
});

We use the same procedure to save messages sent by the user.

Note: chrome.storage is not very large, so we need to be careful about what we store or we may run out of storage space. Also, we should not store confidential data in storage since the storage area is not encrypted.

Resources:

Tags:

  • FOSSASIA, codeheat, Chrome extensions, Javascript, Chrome Storage, Chrome Sync, Susi Chrome Bot, SUSI AI, Bot Development
Continue ReadingMaintaining Extension State in SUSI.AI Chrome Bot Using Chrome Storage API

Adding Map Type Response to SUSI.AI Chromebot

SUSI.AI Chromebot has almost all sorts of reply that SUSI.AI Server can generate. But it still missed the Map Type response that was generated by the SUSI.AI Server.

This blog explains how the map type response was added to the chromebot.

Brief Introduction

The original issue was planned by Manish Devgan and Mohit Sharma as an advanced task for Google Code-In 2017. The link to which can be found here: #157

For a long time the issue remained untouched and after GCI got over I assigned the issue to myself as it was a priority issue since MAP type was a major response from the SUSI.AI Server.

How was Map Type response added?

There were a lot of things to be taken in mind before starting working on this issue.

  • Changing code scheme during GCI and other PRs
  • API Response from the SUSI.AI Server
  • Understanding the new codebase that got altered during GCI-17
  • Doing it quick

I will go through all the steps in detail

Changing Code Scheme

The code was altered numerous times with the addition of a number of pull requests during GCI-17 and there were no docstrings for any functions and methods. So I had to figure them out in order to start working on the map type response.

API Response from the SUSI.AI Server

To understand the JSON that server sent, I went to SUSI.AI API and did a simple search for

“Where is Berlin?” and the response generated is given below.

( Since the JSON is very big I am only posting the relevant data for this issue )

 

    "actions": [
      {
        "type": "answer",
        "language": "en",
        "expression": "Berlin (, German: [bɛɐ̯ˈliːn] ( listen)) is the capital and the largest city of Germany as well as one of its 16 constituent states."
      },
      {
        "type": "anchor",
        "link": "https://www.openstreetmap.org/#map=13/52.52436820069531/13.41053001275776",
        "text": "Here is a map",
        "language": "en"
      },
      {
        "type": "map",
        "latitude": "52.52436820069531",
        "longitude": "13.41053001275776",
        "zoom": "13",
        "language": "en"
      }
    ]

 

Here we see and understand that “actions” is an Array of JSONs and the third part has “type” as “map”. This is the relevant information that we require for generating the map-type response.

The important variables in this context are: “latitude” and “longitude”.

Understanding the Codebase

Now I had to figure out the new pattern of adding response types to the SUSI.AI Chromebot.

After having a talk with @ms10398 I figured out the route map.

The above image shows the correct flow of Javascript Code that generated the response. After this, I was good to go and start my work.

Adding the Map-Type Response

To start with I chose “LEAFLET.JS” as the Javascript Library that will be used to create maps.

  • So I added the LEAFLET.JS to the JS folder.
  • Now changes were made to the “index.html” file

 

<link rel=”stylesheet” href=”https://unpkg.com/leaflet@1.3.1/dist/leaflet.css” />

http://”js/leaflet.js”

 

Appropriate CSS was added along with a link to leaflet.js was added.

  • Adding CSS to the “mapClass”
.mapClass{

    height : 200px;

    width : 200px;

}

 

  • Generating Maps with dynamic IDs

This part was where I applied brain, as to add the map to any div we required the div to have a proper and unique ID and so a way to generate unique IDs for div without using any external source was to be thought of.

I came with idea of using timestamp, as it will always be unique.

var timeStamp = new Date.now().toString();

 

Then I created the “composeMapReply()” function.

function composeReplyMap(response, action){

    var newDiv = messages.childNodes[messages.childElementCount];

    var mapDiv = document.createElement(“div”);

    var mapDivId = Date.now().toString();

    mapDiv.setAttribute(“id”, mapDivId);

    mapDiv.setAttribute(“class”, “mapClass”);

    newDiv.appendChild(mapDiv);

    messages.appendChild(newDiv);

    var newMap = L.map(mapDivId).setView([Number(action.latitude), Number(action.longitude)], 13);

 L.tileLayer(“https://api.tiles.mapbox.com/v4/{id}/{z}/{x}/{y}.png?access_token={accessToken}”,{

    /*

        This part contains the data for api call.

    */

}).addTo(newMap);

response.isMap = true;

response.newMap = mapDiv;

return response

    

}

 

The complete code can be found: here

At last after adding so many snippets of code we were able to generate the Map-Type response for SUSI.AI Chromebot

GIF

A gif showing the Map-Type response in action.

 

Resources

 

Continue ReadingAdding Map Type Response to SUSI.AI Chromebot