How to make SUSI AI LINE Bot

 

(The blog previously written for “How to make SUSI AI Line Bot” is no longer valid as a lot has changed since then. SUSI API, LINE developer menu etc has changed. This blog post describes the updated procedure for creating SUSI AI LINE Bot. The previous blog can be found here – https://blog.fossasia.org/how-to-make-susi-ai-line-bot/)

Susi AI is an intelligent Open Source personal assistant. SUSI AI Bots are built to enable users to chat with SUSI on different clients.

Pre-requisites –

1. In order to integrate SUSI’s API with Line bot, you will need to have a Line account first.

Download LINE Messenger app from here –

Google Play Store

Apple App Store

2. GitHub Account

Create a GitHub from here – Sign up on GitHub

3. Heroku Account

To create Heroku account, go to Heroku and click on Sign Up.

4. Node.js

Install Node.js from https://nodejs.org/en/ if it isn’t installed already on your computer.

To check if node is already installed or not, open terminal and type the command –

node -v

If you see something like this – (version can be different)

v9.4.0

Then Node.js is installed on your computer and you can follow along.

Procedure:

Now that you have created a LINE account, GitHub account, Heroku account and installed Node.js, we can move to the creating the SUSI AI bot.

You can either deploy susi_linebot repository to create SUSI AI LINE bot or you can create a new repository on your account then deploy it to Heroku. The next section will describe how to create a new repository for deploying SUSI AI Line bot. If you want to just deploy susi_linebot then skip the next section.

Creating the SUSI AI LINE bot codebase:

1. Create a folder on your computer with any name. Open terminal and change your current directory to the new folder you just created.

2. Type npm init command and enter details like name, version etc. (preferably just keep pressing enter key and let the values stay default)

3. Create a file with the same name that you wrote in entry point (index.js by default). NOTE – It should be in the same folder that you created earlier.

4. Type the following commands in command line –

npm i -s @line/bot-sdk
npm i -s express
npm i -s request

5. Open package.json file. It should look like this:

{
  "name": "susi_linebot",
  "version": "1.0.0",
  "description": "SUSI AI LINE bot",
  "main": "index.js",
  "scripts": {
    "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@line/bot-sdk": "^6.0.1",
    "express": "^4.16.3",
    "request": "^2.85.0"
  }
}

You should see “@line/bot-sdk”, “express” and “request” under dependencies. Versions can change. Make sure to add “start” under scripts or else deploying on Heroku will cause an error.

6. Copy the following code into the file that you created i.e. index.js

'use strict';
const line = require('@line/bot-sdk');
const express = require('express');
var request = require("request");

// create LINE SDK config from env variables
const config = {
   channelAccessToken: process.env.CHANNEL_ACCESS_TOKEN,
   channelSecret: process.env.CHANNEL_SECRET,
};

// create LINE SDK client
const client = new line.Client(config);

// create Express app
// about Express: https://expressjs.com/
const app = express();

// register a webhook handler with middleware
app.post('/webhook', line.middleware(config), (req, res) => {
   Promise
       .all(req.body.events.map(handleEvent))
       .then((result) => res.json(result))
       .catch((err) => {
        console.error(err);
        res.status(500).end();
      });
});

// event handler
function handleEvent(event) {
   if (event.type !== 'message' || event.message.type !== 'text') {
       // ignore non-text-message event
       return Promise.resolve(null);
   }
   var options = {
       method: 'GET',
       url: 'https://api.susi.ai/susi/chat.json',
       qs: {
           timezoneOffset: '-330',
           q: event.message.text
       }
   };
   request(options, function(error, response, body) {
       if (error) throw new Error(error);
       // answer fetched from susi
       var ans = (JSON.parse(body)).answers[0].actions[0].expression;
       // create a echoing text message
       const answer = {
           type: 'text',
           text: ans
       };
       // use reply API
       return client.replyMessage(event.replyToken, answer);
   })
}

// listen on port

const port = process.env.PORT || 3000;
app.listen(port, () => {
   console.log(`listening on ${port}`);
});

7. In order to deploy this bot on Heroku, we have to make a github repository for it. For making a github repository for the chatbot, follow these steps:

In the command line, change current directory to the folder we created above for the bot and type in the following commands.

git init
git add .
git commit -m "initial"

Now, you have to create a Github repository, follow these steps to do that –

  1. Go to https://github.com/ and login.
  2. Create a new repository. Choose any name.
  3. Get the URL for remote repository and copy it.

Again go to the command line, change current directory to the folder we created above for the bot and type in the following commands.

git remote add origin <URL for remote repository that you just copied>
git remote -v
git push -u origin master

Creating SUSI AI bot on LINE:

1. Go to LINE Developers and click on “Start using Messaging API”.

2. Login through the email ID you used for creating LINE account on LINE app. (If you didn’t enter an email address while creating LINE account then go to “Settings” on LINE app and click on “Account”. Now enter an email address.)

3. Now you have to create a new channel. For that, first of all select a provider or create a new one. You can name it anything.

4. Enter the information for Messaging API. Write all details like App name, description etc. Choose “Developer Trial” under Plan option.

5. Choose any category, subcategory, type in your email address and click Confirm.

6. Now click on your channel to open channel settings.

7. Under “Messaging settings”, click on Issue to issue a channel access token. Choose any number of hours and then issue it. Copy it and save it somewhere. We’ll need it later on.

8. Copy Channel secret given on the same page.

Deploying the Bot on Heroku:

1. Go to Heroku and login.

2. Go to dashboard and create a new app.

3. After creating an app, go to Deploy and choose “GitHub” for Deployment method.

4. Search the repository that you created on GitHub or select susi_linebot after forking it and connect to it in “App connected to GitHub”.

5. Enable Automatic deployment.

6. Now go to Settings and setup Config Variables.

Add the channel access token which you copied earlier as value of “CHANNEL_ACCESS_TOKEN” and the channel secret as value of “CHANNEL_SECRET”. After saving, click on Hide Config Vars.

Now, your bot is deployed on Heroku.

Final Steps:

Finally, go back to LINE Developers website. In settings of SUSI AI app, under “Messaging settings”, enable webhooks and add webhook URL. It will look like this:

https://<your_heroku_app_name>.herokuapp.com/webhook

Clicking on “verify” should show success as shown.

Congratulations! Your SUSI AI LINE bot is ready! Add it as a friend by scanning the QR code provided in the same settings menu and start chatting with SUSI.

References:

Setup interactive charts for data representation

At the end of this blog, you would be able to setup interactive charts using HighCharts and D3.js. As the charts/data-visualisation models will form the backbone of the upcoming SUSI.AI Analytics dashboard, as well as the data representational model for various useful data. For the purpose of integration with the SUSI Skills CMS project, we will be using the react-highcharts and react-tagcloud library.

There are various kinds of charts and plots that HighCharts offers. They are –

  • Line charts
  • Area charts
  • Column and Bar Charts
  • Pie Charts
  • Scatter and Bubble Charts
  • Combinations
  • Dynamic Charts
  • Gauges
  • Heat and Tree maps
  • 3D charts

Check out this link for the full list.

We would be aiming to build up the above charts for analysis of Term Frequency Trends and Trending Clouds.

For the Term Frequency Trends, we will need to setup a Basic Line Graph and for the later,  we need a World Cloud.

Setting up a Basic Line Graph

The aim is to setup a basic line graph and to accomplish that we use a react library called react-highcharts, which makes our work very easier.

Firstly, we create an object config that contains the labels and the required data, with the key values as mentioned in the API reference. The object looks like this –

const config = {
  xAxis: {
    categories: ['01/13', '01/14', '01/15', '01/16', '01/17', '01/18', '01/19', '01/20']
  },
  series: [{
    data: [750, 745, 756, 740, 760, 752, 765]
  }]
};

Secondly, we create a React Component and pass the config object as a property to the ReactHighcharts component.

Finally, we render the component in a div of the index.html file, and the following output is achieved.

The code for the component that renders the Chart is as follows:

import React from 'react';
import ReactHighcharts from 'react-highcharts';
import ReactDOM from 'react-dom';

const config = {
  xAxis: {
    categories: ['01/13', '01/14', '01/15', '01/16', '01/17', '01/18', '01/19', '01/20']
  },
  series: [{
    data: [750, 745, 756, 740, 760, 752, 765]
  }]
};

ReactDOM.render(<ReactHighcharts  config={config} />, document.getElementById('app'));

Setting up a WordCloud

Here, we wish to setup a WordCloud that would show the different words that got searched or the top trending words. We would be using the react-tagcloud library for this.

Firstly, we create an object data that contains the text along with the count/frequency of search. The object looks like this –

const data = [
  { value: "JavaScript", count: 38 },
  { value: "React", count: 30 },
  { value: "Nodejs", count: 28 },
  { value: "Express.js", count: 25 },
  { value: "HTML5", count: 33 },
  { value: "MongoDB", count: 18 },
  { value: "CSS3", count: 20 }
];

Secondly, we create a React Component and pass the data object as a property to the TagCloud component.

Finally, we render the component in a div of the index.html file, and the following output is achieved.

The code for the component that renders the Chart is as follows:

import React from 'react';
import React, { Component } from 'react';
import { TagCloud } from "react-tagcloud";
 
const data = [
  { value: "JavaScript", count: 38 },
  { value: "React", count: 30 },
  { value: "Nodejs", count: 28 },
  { value: "Express.js", count: 25 },
  { value: "HTML5", count: 33 },
  { value: "MongoDB", count: 18 },
  { value: "CSS3", count: 20 }
];
 
const SimpleCloud = () => (
  <TagCloud 
       minSize={12}
       maxSize={35}
       tags={data}
       onClick={tag => alert(`'${tag.value}' was selected!`)} />
);

ReactDOM.render(<SimpleCloud />, document.getElementById('app'));

 

These were some examples of setting up some of the data-visualization models, that would form the basic building block of the SUSI Analytics project. I hope this blogs would be a good starting point for those wanting to start with setting up charts, graphs, etc.

Resources

Skill Development using SUSI Skill CMS

There are a lot of personal assistants around like Google Assistant, Apple’s Siri, Windows’ Cortana, Amazon’s Alexa, etc. What is then special about SUSI.AI which makes it stand apart from all the different assistants in the world? SUSI is different as it gives users the ability to create their own skills in a Wiki-like system. You don’t need to be a developer to be able to enhance SUSI. And, SUSI is an Open Source personal assistant which can do a lot of incredible stuff for you, made by you.

So, let’s say you want to create your own Skill and add it to the existing SUSI Skills. So, these are the steps you need to follow regarding the same –

  1. The current SUSI Skill Development Environment is based on an Etherpad. An Etherpad is a web-based collaborative real-time editor. https://dream.susi.ai/ is one such Etherpad. Open https://dream.susi.ai/ and name your dream (in lowercase letters).
  2. Define your skill in the Etherpad. The general skill format is

::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::term_of_use <link>

#Intent
User query1|query2|query3....
Answer answer1|answer2|answer3...

 

Patterns in query can be learned easily via this tutorial.

  1. Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad.
  2. Once you’ve tested your skill, write ‘stop dreaming’ to disable dreaming for SUSI.
  3. If the testing was successful and you want your skill to be added to SUSI Skills, send a Pull Request to susi_skill_data repository providing your dream name.

How do you modify an existing skill?

SUSI Skill CMS is a web interface where you can modify the skills you’ve made. All the skills of SUSI are directly in sync with the susi_skill_data.

To edit any skill, you need to follow these steps –

  1. Login to SUSI Skill CMS website using your email and password (or Sign Up to the website if you haven’t already).
  2. Click on the skill which you want to edit and then click on the “edit” icon.
  3. You can edit all aspects of the skill in the next state. Below is a preview:

Make the changes and then click on “SAVE” button to save the skill.

What’s happening Behind The Scenes in the EDIT process?

  • SkillEditor.js is the file which is responsible for keeping a check over various validations in the Skill Editing process. There are certain validations that need to be made in the process. Those are as follows –
  • Check whether User has logged in or not

if (!cookies.get('loggedIn')) {
            notification.open({
                message: 'Not logged In',
                description: 'Please login and then try to create/edit a skill',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });
            this.setState({
                loading: false
            });
            return 0;
        }

 

  • Check whether Commit Message has been entered by User or not

if (this.state.commitMessage === null) {
            notification.open({
                message: 'Please add a commit message',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });

            this.setState({
                loading: false
            });
            return 0;
        }

 

  • Check to ensure that request is sent only if there are some differences in old values and new values

if (this.state.oldGroupValue === this.state.groupValue &&
          this.state.oldExpertValue === this.state.expertValue &&
          this.state.oldLanguageValue === this.state.languageValue &&
          !this.state.codeChanged && !this.state.image_name_changed) {
            notification.open({
                message: 'Please make some changes to save the Skill',
                icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
            });
            self.setState({
                loading: false
            });
            return 0;
        }

 

  • After doing the above validations, a request is sent to the Server and the User is shown a notification accordingly, whether the Skill has been uploaded to the Server or there has been some error.

$.ajax(settings)
            .done(function (response) {
                this.setState({
                    loading: false
                });
                let data = JSON.parse(response);
                if (data.accepted === true) {
                    notification.open({
                        message: 'Accepted',
                        description: 'Your Skill has been uploaded to the server',
                        //success/>
                    });
                }
                else {
                    this.setState({
                        loading: false
                    });
                    notification.open({
                        message: 'Error Processing your Request',
                        description: String(data.message),
                        //failure />
                    });
                }
            }

 

  • If the User is notified with a Success notification, then to verify whether the Skill has been added or not, the User can go to susi_skill_data repo and see if he has a recent commit regarding the same or not.

Resources

Making a SUSI Skill to get details about bank from IFSC

We are going to make a SUSI skill that fetches information about a bank when the IFSC (Indian Financial System Code) is known. Here is a detailed explanation of how we going about doing this.

Getting started with the skill creation

API endpoint that returns the bank details

Before going to the skill development, we need to find an API that would return the bank details from the IFSC, On browsing through various open source projects. I found an apt endpoint by Razorpay. Razorpay is a payment gateway for India which allows businesses to accept, process and disburse payments with ease. The Github link  to the repository is https://github.com/razorpay/ifsc.

API endpoint –  https://ifsc.razorpay.com/<:ifsc>
Request type –  GET
Response type –  JSON

Now, head over to the SUSI Etherpad, which is the current SUSI Skill Development Environment and create a new Pad. 

Here, we need to define the skill in the Etherpad. We will now write rules/intents for the skill. An intent represents an action that fulfills a user’s spoken request.

Intents consist of 2 parts –

  • User query – It contains different patterns of query that user can ask.
  • Answer – It contains the possible answer to the user query.

The main intent that our skill focuses on is, returning the bank name and address from the IFSC code. Here is how it looks –

Name of bank with IFSC code * | Bank's name with IFSC code *
!example:Name of bank with IFSC code SBIN0007245
!expect: The name of bank is State Bank of India
!console:The name of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.BANK"
}
eol

Part-wise explanation of the intent

  • The first line contains the query pattern that the user can use while querying. You can see that a wildcard character (*) is used in the pattern. It contains the IFSC of the bank that we wish to know, and will later on use to fetch the details via the API.
  • The second line contains an example query, followed by third line that contains the expected answer.
  • Last part of the rule contains the answer that is fetched from an external API –  https://ifsc.razorpay.com/<:ifsc>  ,via the console service  provided by SUSI Skills. Here, <:ifsc> refers to the IFSC that the user wants to know about. We get it from the user query itself, and can access it by the variable name $1$ as it matches with the 1st wildcard present in the query. If there would be 2 wildcards, we could have accessed them by $1$ and $2$ respectively.
  • The console service provides us with an option to enter the url of the API that we want to hit and path of the key we want to use.

The sample response of the endpoint looks like this :

{
  "BANK": "Karnataka Bank",
  "IFSC": "KARB0000001",
  "BRANCH": "RTGS-HO",
  "ADDRESS": "REGD. & HEAD OFFICE, P.B.NO.599, MAHAVEER CIRCLE, KANKANADY, MANGALORE - 575002",
  "CONTACT": "2228222",
  "CITY": "DAKSHINA KANNADA",
  "RTGS": true,
  "DISTRICT": "MANGALORE",
  "STATE": "KARNATAKA"
}

 

  • Since, we want to extract the name of the bank, the BANK key contains our desired value and we will use $.BANK in the path of the console service. And it can be accessed by $object$ in the answer. We frame the answer using $object$ and $1$ variables, and it like the one mentioned in the expected answer. eol marks the end of the console service.
  • Similarly, the intent that gives us the address of the bank looks like this –
Address of bank with IFSC code * | Bank's address with IFSC code *
!example:Address of bank with IFSC code SBIN0007245
!expect: The address of bank is TILAK ROAD HAKIMPARA, P.O.SILIGURI DARJEELING, WEST BENGAL ,PIN - 734401
!console:The address of bank with IFSC code $1$ is $object$
{
  "url":"https://ifsc.razorpay.com/$1$",
  "path":"$.BANK"
}
eol

Testing the skill

  • Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. We will write down dream ifsc. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad.
  • We can test the skills by asking queries and matching it with the expected answer. Once the testing is done, write stop dreaming to disable dreaming for SUSI.

  • After the testing was successful completely, we will go ahead and add it to the susi_skill_data.
  • The general skill format is –
::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::term_of_use <link>

#Intent
User query1|query2|query3....
Answer answer1|answer2|answer3...

We will add the basic skill details and author details to the etherpad file and make it in the format as mentioned above. The final text file looks like this –

::name IFSC to Bank Details
::author Akshat Garg
::author_url https://github.com/akshatnitd
::description It is a bank lookup skill that takes in IFSC code from the user and provides you all the necessary details for the Bank. It is valid for banks in India only
::dynamic_content Yes
::developer_privacy_policy 
::image images/download.jpeg
::terms_of_use 

Name of bank with IFSC code * | Bank's name with IFSC code *
!example:bank with IFSC code *
!expect: The name of bank is SBI
!console:The name of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.BANK"
}
eol

Address of bank with IFSC code * | Bank's address with IFSC code *
!example:Address of bank with IFSC code *
!expect: The address of bank is 
!console:The address of bank with IFSC code $1$ is $object$
{
"url":"https://ifsc.razorpay.com/$1$",
"path":"$.ADDRESS"
}
eol

Submitting the skill

The final part is adding the skill to the list of skills for SUSI. We can do it by 2 ways:

1st method (using the web interface)

  • Open https://susi.skills.com and login into SUSI account (or sign up, if not done).
  • Click on the create skill button.
  • Select the appropriate fields like Category, Language, Skill name, Logo.
  • Paste the text file that we had created.
  • Add comments regarding the skill and click on Save to save the skill.

2nd method (sending a PR)

  • Send a Pull Request to susi_skill_data repository providing the dream name. The PR should have the text file containing the skill.

So, this was a short blog on how we can develop a SUSI skill of our choice.

Resources

Creating Onboarding Screens for SUSI iOS

Onboarding screens are designed to introduce users to how the application works and what main functions it has, to help them understand how to use it. It can also be helpful for developers who intend to extend the current project.

When you enter in the SUSI iOS app for the first time, you see the onboarding screen displaying information about SUSI iOS features. SUSI iOS is using Material design so the UI of Onboarding screens are following the Material design.

There are four onboarding screens:

  1. Login (Showing the login features of SUSI iOS) – Login to the app using SUSI.AI account or else signup to create a new account or just skip login.
  2. Chat Interface (Showing the chat screen of SUSI iOS) – Interact with SUSI.AI asking queries. Use microphone button for voice interaction.
  3. SUSI Skill (Showing SUSI Skills features) – Browse and try your favorite SUSI.AI Skill.
  4. Chat Settings (SUSI iOS Chat Settings) – Personalize your chat settings for the better experience.

Onboarding Screens User Interface

 

There are three important components of every onboarding screen:

  1. Title – Title of the screen (Login, Chat Interface etc).
  2. Image – Showing the visual presentation of SUSI iOS features.
  3. Description – Small descriptions of features.

Onboarding screen user control:

  • Pagination – Give the ability to the user to go next and previous onboarding screen.
  • Swiping – Left and Right swipe are implemented to enable the user to go to next and previous onboarding screen.
  • Skip Button – Enable users to skip the onboarding instructions and go directly to the login screen.

Implementation of Onboarding Screens:

  • Initializing PaperOnboarding:
override func viewDidLoad() {
super.viewDidLoad()

UIApplication.shared.statusBarStyle = .lightContent
view.accessibilityIdentifier = "onboardingView"

setupPaperOnboardingView()
skipButton.isHidden = false
bottomLoginSkipButton.isHidden = true
view.bringSubview(toFront: skipButton)
view.bringSubview(toFront: bottomLoginSkipButton)
}

private func setupPaperOnboardingView() {
let onboarding = PaperOnboarding()
onboarding.delegate = self
onboarding.dataSource = self
onboarding.translatesAutoresizingMaskIntoConstraints = false
view.addSubview(onboarding)

// Add constraints
for attribute: NSLayoutAttribute in [.left, .right, .top, .bottom] {
let constraint = NSLayoutConstraint(item: onboarding,
attribute: attribute,
relatedBy: .equal,
toItem: view,
attribute: attribute,
multiplier: 1,
constant: 0)
view.addConstraint(constraint)
}
}

 

  • Adding content using dataSource methods:

    let items = [
    OnboardingItemInfo(informationImage: Asset.login.image,
    title: ControllerConstants.Onboarding.login,
    description: ControllerConstants.Onboarding.loginDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.skillOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.chat.image,
    title: ControllerConstants.Onboarding.chatInterface,
    description: ControllerConstants.Onboarding.chatInterfaceDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.chatOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skill.image,
    title: ControllerConstants.Onboarding.skillListing,
    description: ControllerConstants.Onboarding.skillListingDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.loginOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skillSettings.image,
    title: ControllerConstants.Onboarding.chatSettings,
    description: ControllerConstants.Onboarding.chatSettingsDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.iOSBlue(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont)]
    extension OnboardingViewController: PaperOnboardingDelegate, PaperOnboardingDataSource {
    func onboardingItemsCount() -> Int {
    return items.count
    }
    
    func onboardingItem(at index: Int) -> OnboardingItemInfo {
    return items[index]
    }
    }
    

     

  • Hiding/Showing Skip Buttons:
    func onboardingWillTransitonToIndex(_ index: Int) {
    skipButton.isHidden = index == 3 ? true : false
    bottomLoginSkipButton.isHidden = index == 3 ? false : true
    }
    

Resources:

FOSSASIA Internship Program 2018

Are you interested to participate in the development of Open Source projects in a summer internship? Build up your developer profile with FOSSASIA and spend your summer coding on an open source project.  Contribute to SUSI.AIOpen EventBadgeyayYaydoc, Meilix or PSLab and join us at a workshop week and Jugaadfest in India. Please find the details below and submit your application to our form. Be sure to check out FOSSASIA’s program guidelines.

1. Program Details

  • Sign up on our dedicated form at fossasia.org/internship (Interns need to become members of the org and sign up on its social channels)
  • Internships are 3 months with monthly evaluations
  • plus preparation onboarding after acceptance
  • Eligible are contributors above 18 years of age. Any contributor is eligible including students, professionals, university staff etc. Prefered are contributors who have participated in the community previously.
  • Benefits of the program include Shirts, Swag, certificates. All participants who pass the final evaluation will be eligible to participate in a workshop week and Jugaadfest in September 2018 in Hyderabad. Travel grants and accommodation will be provided.
  • The program is intended as a full-time program. However, if contributors would like to participate who have a day job, they can still join and pass the program if they fulfill all program requirements. All contributors who pass the program will be able to receive funding for workshops and Jugaadfest participation.

2. Timeline

  • Application period ongoing until May 12
  • Acceptance ongoing until May 12
  • Start of pre-period:  May
  • Start of Internship: 1st June
  • Evaluation 1: July
  • Evaluation 2: August
  • Evaluation 3: September
  • End of Internship:  September, 2018
  • Issuing of Certificates: September 2018
  • FOSSASIA Workshop Week /Jugaadfest: September/October

3. Deliverables

  • Daily scrum email to project mailing list answering three questions: What did I do yesterday? What is my plan for today? Is there anything preventing me from achieving my goals, e.g. blockers?
  • Work according to pull requests and issues (submit code on Github and match it with issues)
  • Daily code submissions (software, hardware)
  • Documentation: Text, YouTube videos
  • 1 technical blog post a month with details on solving a problem in a FOSSASIA project (Monthly – 1: by Monday of second week)
  • Design items (in open formats, e.g. XCF, SVG, EPS)

4. Participating Projects

5. Best Practices

Please follow best practices as defined here: https://blog.fossasia.org/open-source-developer-guide-and-best-practices-at-fossasia/

6. Participant Benefits/Support

Participants will receive Swag, certificates and travel support to the FOSSASIA Workshop week and Jugaadfest.

  • Evaluation 1: July, 2018: Successful Participants receive a FOSSASIA Tshirt (sent out together with bag in evaluation 2)
  • Evaluation 2: August: Successful Participants receive a beautiful FOSSASIA bag
  • Evaluation 3: September: Successful Participants receive the following support to participate in the FOSSASIA India Workshop Week and Jugaadfest:
    • 100 SGD travel support from within India and 200 SGD support if coming from outside India
    • One week accommodation in Hyderabad (organized by FOSSASIA)
    • Catering during workshops

Maintaining Extension State in SUSI.AI Chrome Bot Using Chrome Storage API

SUSI Chrome Bot is a browser action chrome extension which is used to communicate with SUSI AI.The browser action extension in chrome is like any other web app that you visit. It will store all your preferences like theme settings and speech synthesis settings and data till you are interacting with it, but once you close it, it forgets all of your data unless you are saving it in some database or you are using cookies for the session. We want to be able to save the chats and other preferences like theme settings the user makes when interacting with SUSI AI through Susi Chrome Bot. In this blog, we’ll explore Chrome’s chrome.storage API for storing data.

What options do we have for storing data offline?

IndexedDB: IndexedDB is a low-level API for client-side storage of data. IndexedDB allows us to store a large amount of data and works like RDBMS but IndexedDB is javascript based Object-oriented database.

localStorage API: localStorage allows us to store data in key/value pairs which is much more effective than storing data in cookies. localStorage data persists even if the user closes and reopens the browser.

Chrome.storage: Chrome provides us with chrome.storage. It provides the same storage capabilities as localStorage API with some advantages.

For susi_chromebot we will use chrome.storage because of the following advantages it has over the localstorage API:

  1. User data can be automatically synced with Chrome sync if the user is logged in.
  2. The extension’s content scripts can directly access user data without the need for a background page.
  3. A user’s extension settings can be persisted even when using incognito mode.
  4. It’s asynchronous so bulk read and write operations are faster than the serial and blocking localStorage API.
  5. User data can be stored as objects whereas the localStorage API stores data in strings.

Integrating chrome.storage to susi_chromebot for storing chat data

To use chrome.storage we first need to declare the necessary permission in the extension’s manifest file. Add “storage” in the permissions key inside the manifest file.

"permissions": [
         "storage"
       ]

 

We want to store the chat user has made with SUSI. We will use a Javascript object to store the chat data.

var storageObj = {
senderClass: "",
content: ""
};

The storageObj object has two keys namely senderClass and content. The senderClass key represents the sender of the message(user or susi) whereas the content key holds the actual content of the message.

We will use chrome.storage.get and chrome.storage.set methods to store and retrieve data.

var susimessage = newDiv.innerHTML;
storageObj.content = susimessage;
storageObj.senderClass = "susinewmessage";
chrome.storage.sync.get("message",(items) => {
if(items.message){
storageArr = items.message;
}
storageArr.push(storageObj);
chrome.storage.sync.set({"message":storageArr},() => {
console.log("saved");
});
});

 

In the above code snippet, susimessage contains the actual message content sent by the SUSI server. We then set the correct properties of the storageObj object that we declared earlier. Now we can use chrome.storage.set to save the storageObj object but that would overwrite the current data that we have inside chrome’s StorageArea. To prevent the old message data from getting overwritten, we’ll first get all the message content in our storage using chrome.storage.sync.get. Notice how we are passing the “message” string as the first perimeter to the function. This is done because we only want our message content which was saved in the StorageArea. If we pass null instead, it will return all the content inside storageArea. Once we have our messages (which will be an array of objects that we store as storageObj), we will store that into a new array storageArr. We will then push our new storageObj that contains the message and the sender into the array. Finally, we use chrome.storage.sync.set to save the message content in chrome’s StorageArea which can later be retrieved using the “message” key.

storageArr.push(storageObj);
chrome.storage.sync.set({"message":storageArr},() => {
console.log("saved");
});

We use the same procedure to save messages sent by the user.

Note: chrome.storage is not very large, so we need to be careful about what we store or we may run out of storage space. Also, we should not store confidential data in storage since the storage area is not encrypted.

Resources:

Tags:

  • FOSSASIA, codeheat, Chrome extensions, Javascript, Chrome Storage, Chrome Sync, Susi Chrome Bot, SUSI AI, Bot Development

Adding Map Type Response to SUSI.AI Chromebot

SUSI.AI Chromebot has almost all sorts of reply that SUSI.AI Server can generate. But it still missed the Map Type response that was generated by the SUSI.AI Server.

This blog explains how the map type response was added to the chromebot.

Brief Introduction

The original issue was planned by Manish Devgan and Mohit Sharma as an advanced task for Google Code-In 2017. The link to which can be found here: #157

For a long time the issue remained untouched and after GCI got over I assigned the issue to myself as it was a priority issue since MAP type was a major response from the SUSI.AI Server.

How was Map Type response added?

There were a lot of things to be taken in mind before starting working on this issue.

  • Changing code scheme during GCI and other PRs
  • API Response from the SUSI.AI Server
  • Understanding the new codebase that got altered during GCI-17
  • Doing it quick

I will go through all the steps in detail

Changing Code Scheme

The code was altered numerous times with the addition of a number of pull requests during GCI-17 and there were no docstrings for any functions and methods. So I had to figure them out in order to start working on the map type response.

API Response from the SUSI.AI Server

To understand the JSON that server sent, I went to SUSI.AI API and did a simple search for

“Where is Berlin?” and the response generated is given below.

( Since the JSON is very big I am only posting the relevant data for this issue )

 

    "actions": [
      {
        "type": "answer",
        "language": "en",
        "expression": "Berlin (, German: [bɛɐ̯ˈliːn] ( listen)) is the capital and the largest city of Germany as well as one of its 16 constituent states."
      },
      {
        "type": "anchor",
        "link": "https://www.openstreetmap.org/#map=13/52.52436820069531/13.41053001275776",
        "text": "Here is a map",
        "language": "en"
      },
      {
        "type": "map",
        "latitude": "52.52436820069531",
        "longitude": "13.41053001275776",
        "zoom": "13",
        "language": "en"
      }
    ]

 

Here we see and understand that “actions” is an Array of JSONs and the third part has “type” as “map”. This is the relevant information that we require for generating the map-type response.

The important variables in this context are: “latitude” and “longitude”.

Understanding the Codebase

Now I had to figure out the new pattern of adding response types to the SUSI.AI Chromebot.

After having a talk with @ms10398 I figured out the route map.

The above image shows the correct flow of Javascript Code that generated the response. After this, I was good to go and start my work.

Adding the Map-Type Response

To start with I chose “LEAFLET.JS” as the Javascript Library that will be used to create maps.

  • So I added the LEAFLET.JS to the JS folder.
  • Now changes were made to the “index.html” file

 

<link rel=”stylesheet” href=”https://unpkg.com/[email protected]/dist/leaflet.css” />

http://”js/leaflet.js”

 

Appropriate CSS was added along with a link to leaflet.js was added.

  • Adding CSS to the “mapClass
.mapClass{

    height : 200px;

    width : 200px;

}

 

  • Generating Maps with dynamic IDs

This part was where I applied brain, as to add the map to any div we required the div to have a proper and unique ID and so a way to generate unique IDs for div without using any external source was to be thought of.

I came with idea of using timestamp, as it will always be unique.

var timeStamp = new Date.now().toString();

 

Then I created the “composeMapReply()” function.

function composeReplyMap(response, action){

    var newDiv = messages.childNodes[messages.childElementCount];

    var mapDiv = document.createElement(“div”);

    var mapDivId = Date.now().toString();

    mapDiv.setAttribute(“id”, mapDivId);

    mapDiv.setAttribute(“class”, “mapClass”);

    newDiv.appendChild(mapDiv);

    messages.appendChild(newDiv);

    var newMap = L.map(mapDivId).setView([Number(action.latitude), Number(action.longitude)], 13);

 L.tileLayer(“https://api.tiles.mapbox.com/v4/{id}/{z}/{x}/{y}.png?access_token={accessToken}”,{

    /*

        This part contains the data for api call.

    */

}).addTo(newMap);

response.isMap = true;

response.newMap = mapDiv;

return response

    

}

 

The complete code can be found: here

At last after adding so many snippets of code we were able to generate the Map-Type response for SUSI.AI Chromebot

GIF

A gif showing the Map-Type response in action.

 

Resources

 

Toggling Voice On/Off in SUSI Chromebot

SUSI Chromebot has a lot of features that make it one of the best projects of FOSSASIA.

Recently Voice/Speech was added to SUSI Chromebot. But there was no option that controlled the fact that whether speech output is needed or not.

The latest addition to SUSI Chromebot is Toggling the Voice of SUSI On or Off.

How was it achieved?

Toggling Voice for SUSI required adding a button and a snippet of Javascript code to the main JS file. The code will take care of the fact whether the voice is to be toggled on or off.

I started off by adding a button to the main HTML file.

<a href=”javascript: void(0)” id=”speak” style=”color: white”><i class=”material-icons” id=”speak-icon”>volume_up</i></a>

The above snippet of HTML code adds a voice button to the top bar of chromebot.

Then there was the major part where the javascript code was to be added to add the functionality to the button.

var shouldSpeak = true;

I started off by creating a variable called as “shouldSpeak” which will determine whether or not SUSI should use the Chrome’s API to speak.

Then I changed the “speakOut()” function and added another parameter to it.

function speakOut(msg,speak=false){

if(speak){

var voiceMsg = new SpeechSynthesisUtterance(msg);

window.speechSynthesis.speak(voiceMsg);

}

}

The above code made sure that susi was only allowed to speak when and only “speak” variable was set to true.

Then “eventListeners” were added to buttons and other things to link the functionality.

document.getElementById(‘speak’).addEventListener(‘click’,changeSpeak);

It adds the events of click to “speak” and associates it with the function “changeSpeak”.

Now the function “changeSpeak” is created as follows. It toggles the on/off mechanism of voice in SUSI Chromebot.

function changeSpeak(){

shouldSpeak = !shouldSpeak;

var SpeakIcon = document.getElementById(‘speak-icon’);

if(!shouldSpeak){

SpeakIcon.innerText = “volume_off”;

}

else{

SpeakIcon.innerText = “volume_up”;

}

console.log(‘Should be speaking? ’ + shouldSpeak);

}

Everytime the user clicks on the icon to toggle on/off voice the icon must also change and this functionality was taken care of by the above piece of code.

Resources

 

Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners

SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop.

Any electron app essentially comprises of the following components

    • Main Process (Managing windows and other interactions with the operating system)
    • Renderer Process (Manage the view inside the BrowserWindow)

Steps to setup development environment

      • Clone the repo locally.
$ git clone https://github.com/fossasia/susi_desktop.git
$ cd susi_desktop
      • Install the dependencies listed in package.json file.
$ npm install
      • Start the app using the start script.
$ npm start

Structure of the project

The project was restructured to ensure that the working environment of the Main and Renderer processes are separate which makes the codebase easier to read and debug, this is how the current project is structured.

The root directory of the project contains another directory ‘app’ which contains our electron application. Then we have a package.json which contains the information about the project and the modules required for building the project and then there are other github helper files.

Inside the app directory-

  • Main – Files for managing the main process of the app
  • Renderer – Files for managing the renderer process of the app
  • Resources – Icons for the app and the tray/media files
  • Webview Tag

    Display external web content in an isolated frame and process, this is used to load chat.susi.ai in a BrowserWindow as

    <webview src="https://chat.susi.ai/"></webview>
    

    Adding event listeners to the app

    Various electron APIs were used to give a native feel to the application.

  • Send focus to the window WebContents on focussing the app window.
  • win.on('focus', () => {
    	win.webContents.send('focus');
    });
    
  • Display the window only once the DOM has completely loaded.
  • const page = mainWindow.webContents;
    ...
    page.on('dom-ready', () => {
    	mainWindow.show();
    });
    
  • Display the window on ‘ready-to-show’ event
  • win.once('ready-to-show', () => {
    	win.show();
    });
    

    Resources

    1. A quick article to understand electron’s main and renderer process by Cameron Nokes at Medium link
    2. Official documentation about the webview tag at https://electron.atom.io/docs/api/webview-tag/
    3. Read more about electron processes at https://electronjs.org/docs/glossary#process
    4. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.