Protected Skills for SUSI.AI Bot Wizard

  • Post author:
  • Post category:FOSSASIA

The first version of SUSI AI web bot plugin is working like a protected skill. A SUSI.AI skill can be created by someone with minimum base user role USER, or in other words, anyone who is logged in.  Anyone can see this skill or edit it. This is not the case with a protected skill or bot. If a skill is protected it becomes a personal bot and then it can only be used by the SUSI AI web client (chatbot) created by that user. Also, only the person who created this skill will be able to edit it or delete it. This skill won’t be listed with all the other public skills on skills.susi.ai.

How is a skill made private?

To make a skill private, a new parameter is added to SusiSkill.java file. This is a boolean called protected. If this parameter is true (Yes) then the skill is protected else if the parameter is false (No) then the skill is not protected. To add protected parameter, we add the following code to SusiSkill.java:

private Boolean protectedSkill;
boolean protectedSkill = false;

You can see that protectedSkill is a boolean with initial value false.
We need to read the value of this parameter in skill code. This is done by the following code:

if (line.startsWith("::protected"&& (thenpos = line.indexOf(' ')) > 0) {
  if (line.substring(thenpos+ 1).trim().equalsIgnoreCase("yes")) protectedSkill=true;
  json.put("protected",protectedSkill);
}

As you can see that the value of protected is read from the skill code. If it’s ‘Yes’ then protectedSkill is set as true and if it’s ‘No’ then protectedSkill is set as false.
If no protected parameter is given in the skill code, its value remains false.

How to add only protected skills from bot wizard?

Now that protected parameter has been added to the server, we need to add this parameter in the skill code but it should be ‘Yes’ if the user is creating a bot and ‘No’ if user is creating a skill using skill creator. In order to do this, we simply check if the user is creating a bot wizard. This can be done by passing a props in the CreateSkill.js file when the skill creator is being used to create a protected skill. Next, we can determine whether the user is using bot wizard or not simply by an if else statement. The following code will demonstrate it:

if (this.props.botBuilder) {
 code = '::protected Yes\n' + code;
else {
 code = '::protected No\n' + code;
}

References:

Continue Reading Protected Skills for SUSI.AI Bot Wizard

Display Skill Usage of the Past Week in a Line Chart

Skill usage statistics in SUSI.AI is an important aspect which greatly helps the skill developers know what is more engaging for the users and the users know which skills are more popular than the others and are being used widely. So data like this should be interactively available on the clients. An API is developed at the server to retrieve the skill usage details, we can use these details to render attractive components for a better visual understanding of how the skill is performing and get statistics like on which days the skill has been active in particular.

 

About the API

Endpoint : /cms/getSkillUsage.json

Parameters

  • model
  • group
  • language
  • skill

After consuming these params the API will return the number of times a skill is called along with the date on which it is called. We use that data as an input for the line chart component that we want to render.

Creating a Skill Usage card component

Import the required packages from corresponding libraries. We are using recharts as the library for the charts support and thus we import several components required for the line chart at the top of the Skill Usage component.

// Packages
import React, { Component } from 'react';
import PropTypes from 'prop-types';
import { LineChart, Line, XAxis, YAxis, Tooltip, Legend } from 'recharts';
import { Paper } from 'material-ui';

 

Create a SkillUsageCard component which is enclosed in a Paper component from material ui library to give it a card like look and then we render a Line chart inside it using appropriate data props which are received from the skill page component, we set a height and a width for the chart, Add an X-Axis, Y-Axis, Tooltip which shows up on hovering over points on the graph, legends to describe the line on the chart and then finally a line with several styling and other props.

class SkillUsageCard extends Component {
 render() {
   return(
     <Paper className="margin-b-md margin-t-md">
       <h1 className='title'>
           Skill Usage
       </h1>
       <div className="skill-usage-graph">
         <LineChart width={600} height={300} data={this.props.skill_usage}
               margin={{top: 5, right: 30, left: 20, bottom: 5}}>
           <XAxis dataKey="date" padding={{right: 20}} />
           <YAxis/>
           <Tooltip wrapperStyle={{height: '60px'}}/>
           <Legend />
           <Line name='Skill usage count' type="monotone" dataKey="count" stroke="#82ca9d" activeDot={{r: 8}}/>
         </LineChart>
       </div>
     </Paper>
   )
 }
}

 

Add prop validation at the end of the file to validate the coming props from the skill page component to validate that correct props are being received.

SkillUsageCard.propTypes = {
   skill_usage: PropTypes.array
}

 

Export the component so it can be used in other components.

export default SkillUsageCard;

 

Fetch the data for the component from the API in the skill page component where the skill usage component will be rendered. First set the API url and then make an AJAX call to that URL and once the data is received from the server pass that received data to a saveSkillUsage function which does the simple task of saving the data to the state and passing the saved data as a prop to the skill usage component. In case the call fails we log the error to the console.

let skillRatingUrl = `${urls.API_URL}/cms/getSkillRating.json`;
skillUsageUrl = skillUsageUrl + '?model=' + modelValue + '&group=' + this.groupValue + '&language=' + this.languageValue + '&skill=' + this.name;
// Fetch skill usage of the visited skill
$.ajax({
  url: skillUsageUrl,
  dataType: 'json',
  crossDomain: true,
  success: function (data) {
      self.saveSkillUsage(data.skill_usage)
  },
  error: function(e) {
       console.log(e)
  }
});

 

Save the skill usage details in the component state to render the skill usage component.

saveSkillUsage = (skill_usage = []) => {
  this.setState({
     skill_usage
 })
}

 

Send the received data as props to the Skill Usage component and render it.

<SkillUsageCard skill_usage={this.state.skill_usage} /> 

 

So I hope after reading this blog you have a more clearer insight into how the skill usage details are implemented in the CMS.

Resources –

  • Jerry J. Muzsik, Creating and deploying a react app using recharts URL.
  • Recharts, Documentation, URL.
Continue Reading Display Skill Usage of the Past Week in a Line Chart

“STOP” action in SUSI Android App

Generally whenever there was a long query asked from SUSI through speech, it would respond to the user with a speech output, similarly the output is given through speech whenever the user clicks on the “Try it” button on the skill details activity.

The long answers for e.g. asking SUSI “How to cook biryani ?” gives a very long response which when conveyed through the speech output takes a long amount of time. Also most of the time users don’t want to listen to such long answers, but at the same time there is no option or feature to stop SUSI. The user either needs to switch over to another activity or has to close the app.

So, to solve this problem such that neither the user has to shut down the app nor the user has to switch over to another activity, the “STOP” action was added in SUSI.

The “STOP” action was integrated in the server and is of following type :

 

“actions”: [{“type”: “stop”}],

How to define STOP response ?

To integrate this action type in the app, a separate response type was added and checked for. In the file ParseSusiResponseHandler.kt the action type stop was added as :

Constant.STOP -> try {
  stop = susiResponse.answers[0].actions[1].type
} catch (e: Exception) {

}

So, now whenever the query of type “stop” was entered by the user it would be caught by this block and further processing could be done. The stop variable takes the value from the JSON response fetched and the value extracted is the one stored against the type key in the second field of the actions JSON in the first field of the answers JSON.

What to be done when executing STOP?

After the STOP action was caught the task to do was to define the behaviour of the app when this response was caught. So, first thing that should happen when STOP is received is that the TextToSpeech Engine should close so that SUSI no longer can speak the sentence but defining top is more than just closing the TTS engine. STOP signifies that any activity that is being continued right now should be stopped.

So, Using the android lifecycle methods I added a function in the IChatView interface to define what actions should take place in the stopping process.

override fun stopMic() {
  onPause()
  registerReceiver(networkStateReceiver, IntentFilter(ConnectivityManager.CONNECTIVITY_ACTION))

  window.setSoftInputMode(
          WindowManager.LayoutParams.SOFT_INPUT_STATE_ALWAYS_HIDDEN
  )

  if (recordingThread != null)
      chatPresenter.startHotwordDetection()

  if (etMessage.text.toString().isNotEmpty()) {
      btnSpeak.setImageResource(R.drawable.ic_send_fab)
      etMessage.setText(“”)
      chatPresenter.micCheck(false)
  }

  chatPresenter.checkPreferences()
}

The above function sends the activity to the onPause() lifecycle method to put the activity in a pausing state so that all that is taking place right now in the activity stops and this definitely serves our purpose. But after pausing the activity, there was a further need to perform some functionality that had to done on the start of the activity and therefore the code for that was also added in this function.

How to catch STOP?

The below code was added in ChatPresenter.kt file in which if the actionType from the psh object of type ParseSusiResponseHelper is “STOP” then the view function stopMic() is called which was defined above.

val psh = ParseSusiResponseHelper()
psh.parseSusiResponse(susiResponse, i, utilModel.getString(R.string.error_occurred_try_again))

var setMessage = psh.answer
if (psh.actionType == Constant.ANSWER && (PrefManager.checkSpeechOutputPref() && check || PrefManager.checkSpeechAlwaysPref())) {
  setMessage = psh.answer

  var speechReply = setMessage
  if (psh.isHavingLink) {
      speechReply = setMessage.substring(0, setMessage.indexOf(“http”))
  }
  chatView?.voiceReply(speechReply, susiResponse.answers[0].actions[i].language)
} else if (psh.actionType == Constant.STOP) {
  setMessage = psh.stop
  chatView?.stopMic()
}

Final Output

References

  1. Stop  json response from susi server : https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=susi+stop
  2. Android life cycle methods – Google: https://developer.android.com/guide/components/activities/activity-lifecycle
  3. Interaction between view and presenters : https://medium.com/@cervonefrancesco/model-view-presenter-android-guidelines-94970b430ddf
Continue Reading “STOP” action in SUSI Android App

Implementing Skill Listing in SUSI iOS

Skills are basically a set of rules which respond to the user’s queries through any client app. All the skills are defined in the SUSI Skill Data repo where the user’s queries are matched with the already present skills and the server responds accordingly. Apps like Alexa, Google Assistant have an interface to view skills with a set of sample queries that can be used therefore, we are adding the same skill display UI in the SUSI iOS app.

Implementation

All the skills are arranged into categories or groups, so we first need to fetch all those groups followed by fetching skills from each group. For listing all the groups, we use the endpoint below:

http://api.susi.ai/cms/getGroups.json

Which returns all the groups in the groups object like below:

{
  "session": {"identity": {
    "type": "host",
    "name": "67.214.191.117",
    "anonymous": true
  }},
  "accepted": true,
  "groups": [
    "Social",
    "News",
    "Food and Drink",
    "Travel and Transportation",
    "Connected Car",
    "Movies and TV",
    "Problem Solving",
    "Knowledge",
    "Business and Finance",
    "Productivity",
    "Games, Trivia and Accessories",
    "Lifestyle",
    "Health and Fitness",
    "Music and Audio",
    "Shopping",
    "Communication",
    "Novelty and Humour",
    "Utilities",
    "Sports", 
    "Weather"
  ],
  "message": "Success: Fetched group list"
}

After the groups have been fetched, we need to get the skills for each group. Here, we use the endpoint below:

http://api.susi.ai/cms/getSkillList.json?group=GROUP_NAME

Since, we have a number of groups, we need to make the above API call as many times as the group count is. A sample call would look like:

http://api.susi.ai/cms/getSkillList.json?group=News

which would fetch all the skills in the News group.

{
  "accepted": true,
  "model": "general",
  "group": "News",
  "language": "en",
  "skills": {"news": {
    "image": "images/news.png",
    "author_url": "https://github.com/AliAyub007",
    "examples": [
      "News",
      "latest news",
      "most viewed articles in science today",
      "most viewed articles in science in the last week",
      "most viewed articles in science in the last month",
      "most shared articles in science today",
      "can you tell me last week's most shared articles in science",
      "do you know most shared articles in arts in the last month",
      "most emailed articles today in arts",
      "most emailed articles in science in the last week",
      "can you tell me most emailed articles in science in the last month",
      "articles in science",
      "show me articles",
      "most emailed articles",
      "most shared articles",
      "tell me news in tech world"
    ],
    "developer_privacy_policy": null,
    "author": "Ali Ayub Khan",
    "skill_name": "NEWS",
    "dynamic_content": true,
    "terms_of_use": null,
    "descriptions": "A skill to give news.",
    "skill_rating": null
  }},
  "message": "Success: Fetched skill list",
  "session": {"identity": {
    "type": "host",
    "name": "23.94.137.239",
    "anonymous": true
  }}
}

Each such json object gives us the following values:

  • Model
  • Group
  • Language
  • Image Path
  • Author’s URL
  • Author’s Name
  • A list of sample queries
  • Skill Name
  • Licence and terms of use
  • Rating
  • Description

Implementation in SUSI iOS

The UI for listing skills is a little complex as it consists of a UITableView where each UITableViewCell consists of a UILabel(group name) and a UICollectionVIew (horizontal scroll).

Let’s see the step by step process to implement the skill listing.

  1. First, we need to fetch all the groups available using the endpoint above.
// get all groups
func getAllGroups() {
  Client.sharedInstance.getAllGroups { (groups, success, message) in
    DispatchQueue.main.async {
      if success {
        self.groups = groups
        self.tableView.reloadData()
      } else {
        print(message ?? "error")
      }
    }
  }
}
  1. After the response from the above call is obtained, the groups object containing all the groups is returned to the controller which the group name to set the UILabel.
var groupName: String? {
  didSet {
    backgroundColor = Color.grey.lighten3
    groupNameLabel.text = groupName
  }
}
  1. After the label is set, each cell makes another API call to populate the collection view, since these calls are concurrent, the collection view populates as soon as the response is fetched.
  2. The response from the above API call is used to create an array of Skill Model objects which is used to parse the response and to effectively use the data to display.
if let skills = response[Client.SkillListing.skills] as? [String : AnyObject],
  let model = response[Client.SkillListing.model] as? String,
  let group = response[Client.SkillListing.group] as? String,
  let language = response[Client.SkillListing.language] as? String,
  skills.count > 0 {
    let skillData = Skill.getAllSkill(skills, model, group, language)
    completion(skillData, true, nil)
    return
}
static func getAllSkill(_ skills: [String : AnyObject], _ model: String, _ group: String, _ language: String) -> [Skill] {
  var skillData = [Skill]()
  for skill in skills {
    let newSkill = Skill(dictionary: skill.value as! [String : AnyObject])
    newSkill.imagePath = getImagePath(model, group, language, newSkill.imagePath)
    skillData.append(newSkill)
  }
  return skillData
}
  1. At last, the skill object for the collection view is used to populate it as below:
var skill: Skill? {
  didSet {
    if let skill = skill {
      if let url = URL(string: skill.imagePath) {
        imageView.kf.setImage(with: url)
      }
      exampleQueryLabel.text = "\(skill.examples.first?.debugDescription ?? "")"
      skillNameLabel.text = skill.skillName
      skillDescription.text = skill.skillDescription
    }
  }
}

Here, we make use of the didSet method to populate the collection view. We have used Kingfisher to display the images.

That’s all for the scope of this tutorial. We learned how to fetch the skill groups followed by fetching the skills and displaying the skill data using the skill model object.

Below is the final UI we see after implementation.

Resources

Continue Reading Implementing Skill Listing in SUSI iOS

How to make a SUSI chat bot skill for Cortana

Cortana is assistant from Microsoft just like Siri from Apple. We can make a skill for Cortana i.e Creating your own bot with in cortana so that you can use your bot through cortana by activating it using invocation name. To create SUSI cortana skill we will use SUSI API and Microsoft bot framework. First of all we will have to make a bot on microsoft bot framework and to do so follow this tutorial. Change code in this tutorial with code given below.

var restify = require('restify');
var builder = require('botbuilder');
var request = require('request');
var http = require('http');

// Setup Restify Server
var server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 8080, function() {
   console.log('listening to ', server.name, server.url);
});
// Create chat bot
var connector = new builder.ChatConnector({
   appId: process.env.appId,
   appPassword: process.env.appPassword
});

setInterval(function() {
       http.get(process.env.HerokuURL);
   }, 1200000);

var bot = new builder.UniversalBot(connector);
server.post('/api/messages', connector.listen());

//getting response from SUSI API upon receiving messages from User
bot.dialog('/', function(session) {
   var msg = session.message.text;
   var options = {
       method: 'GET',
       url: 'http://api.asksusi.com/susi/chat.json',
       qs: {
           timezoneOffset: '-330',
           q: session.message.text
       }
   };
//sending request to SUSI API for response
   request(options, function(error, response, body) {
       if (error) throw new Error(error);
       var ans = (JSON.parse(body)).answers[0].actions[0].expression;
       //responding back to user
       session.say(ans,ans);

   })
});

After making bot we have to configure this bot for Cortana and to do so select cortana from list of channel on your bot from https://dev.botframework.com and add following detail shown in figure below.

Invocation name is name which you will call to use your bot. There are many ways to in invoke your skill like Run <invocation name> , Launch <invocation name>. Here is the complete list of invocation commands. How SUSI skill in cortana actually works is when user invoke SUSI skill cortana acts as middleware to send and receive responses from skill to user. After configuring your bot for Cortana test it with cortana with same hotmail account which you have used for bot framework by calling invocation name. Here is the demo video for testing SUSI Skill https://youtu.be/40yX16hxcls.

You have complete creating a skill for cortana now if you want to publish it to world select manage cortana dashboard from your bots in https://dev.botframework.com and publish it to first by filling form.

If you want to learn more about bot framework refer to https://docs.microsoft.com/en-us/Bot-Framework/index

References:
SUSI Cortana Repository: https://github.com/fossasia/susi_cortana_skill
Tutorial for bot: https://blog.fossasia.org/susi-ai-bots-with-microsofts-bot-framework/

Continue Reading How to make a SUSI chat bot skill for Cortana

How to add the Google Books API to SUSI AI

SUSI.AI is a Open Source personal assistant. You can also add new skills to SUSI easily. In this blog post I’m going to add Google’s Books API to SUSI as a skill. A complete tutorial on SUSI.AI skills is n the repository. Check out Tutorial Level 11: Call an external API here and you will understand how can we integrate an external API with SUSI AI.

To start adding book skills to SUSI.AI , first go to this URL http://dream.susi.ai/  > give a name in the text field and press OK.

 

Copy and paste above code to the newly opened etherpad.

Go to this url http://chat.susi.ai to test new skill.

Type “dream blogpost” on chat and press enter. Now we can use the skills we  add to the etherpad.

To understand  Google’s book API use this url.Your request url should be like this:

[code]https://www.googleapis.com/books/v1/volumes?q=BOOKNAME&amp;amp;amp;amp;amp;amp;amp;amp;key=yourAPIKey[/code]

 

you should replace APIKey with your API key.

To get started you first need to get an API key.

Go to this url > click GET A KEY button which is in right top > and select “Create a new project”

Add name to a project and click “CREATE AND ENABLE API” button

Copy your API key and replace the API Key part of request URL.

Paste request url on your browser address bar and replace BOOKNAME part with “flower” and go to the URL. It will give this JSON.

We need to get the full name of books which is in items array to that we have to go through this hierarchy
items array >first item>volumeInfo >title
Go to the etherpad we made before and paste the following code.


is there any book called * ?
!console:did you mean "$title$" ? Here is a link to read more: $infoLink$
{
"url":"https://www.googleapis.com/books/v1/volumes?q=$1$&key=AIzaSyCt3Wop5gN3S5H0r1CKZlXIgaM908oVDls",
"path":"$.items[0].volumeInfo"
}
eol

first line of the code “is there any book called *?” is the question user ask. *  is the variant part  of question. that part can be used in the code by $1$ , if there more variants we can add multiple asterisk marks and refer by using corresponding number Ex: $1$,$2$,$3$
  • In this code  “path” : “$.items[0].volumeInfo”
  • $  represents full JSON result.
  • items[0] for get first element
  • .volumeInfo is to refer  volumeInfo object
!console:did you mean “$title$” ?  Here is a link to read more: $infoLink$
this line produce the output.
  • $title$ this one is for refer the “title” part of data that comes from “path”
  • $infoLink$ this one gives link to more details

Now go to the chat UI and type again “dream blogpost”. And after it shows “dreaming enabled” type in”is there any book called world war?”. It will result in the following.

This  is a simple way to add any service to SUSI as a skill.

Continue Reading How to add the Google Books API to SUSI AI