How to Change a Password and Forgot Password Feature in the SUSI.AI Server

The accounting system of SUSI.AI provides its users the option to change the password of their accounts. This features gives us two options, either we can change our password by entering the older password or if the user forgot the password, they are provided a link on their email address through which we can change our password. Using either option, the user has to authenticate themselves before they can actually change their passwords. If the user has the current password, it is considered as a parameter of authentication. In other case the user has to check their email account for the link which also confirms the authenticity of user. In this post we will discuss how both options works on SUSI.

Continue ReadingHow to Change a Password and Forgot Password Feature in the SUSI.AI Server

Youtube search as a Console Service Endpoint in SUSI.AI

SUSI.AI now has the ability to search and play any song or video directly in the webclient and the speaker. When a user asks for a query regarding playing a song, the clients sends a search request to the server. In this post I will discuss about the server side implementation of the Youtube search. Every time a request is made by any client, the client sends a query request to the server in the form of a json object. For more on the working on the webclient side can be seen here.

The endpoint for youtube search is  http://api.susi.ai/susi/console.json

Continue ReadingYoutube search as a Console Service Endpoint in SUSI.AI

Delete User Account Service using LoginService.java API in SUSI.AI

SUSI.AI has an api to handle all account related services called accounts.susi.ai. This API provides services like change password and delete account. the main goal of accounts.susi.ai is to centralise the accounts’ related settings from web, android and iOS clients into a single place like other accounting services. In this post we will discuss one of these features which is the delete user account from SUSI.AI.

The api accounts.susi.ai has a react file DeleteAccount.react.js. This file handle the delete account service and interacts with the LoginServce.java api, which is the core file for authentication. I will discuss in details how these two interact with each other and with other files.

Continue ReadingDelete User Account Service using LoginService.java API in SUSI.AI

Drafts of SUSI Chatbots

While creating a SUSI bot using bot wizard, a user has the option to save a draft of the bot at any step. Then he/she can continue building the bot from there. We can do four operations on a draft: Storing, fetching, editing and deleting.

Storing a draft:

For the storage of drafts, a GET request is made to the storeDraft.json API. The code of chatbot is passed in ‘object’ parameter. This code must be in JSON format. A sample API call looks like this:

https://api.susi.ai/cms/storeDraft.json?object={"test":"123"}

The JSON passed in ‘object’ has the following format:

{
   “group”: /*group_here*/
   “language”: /*language_here*/
   “name”: /*bot_name_here*/
   “buildCode”: /*code_in_build_tab_here*/
   “designCode”: /*code_in_design_tab_here*/
   “configCode”: /*code_in_config_tab_here*/
}

While storing the draft, we can either specify an ID ourselves as a parameter named ‘id’ in the API call url or we can get a randomly generated id from the server itself.
The hex codes in designCode includes ‘#’ which can not be passed in the url. Hence, we remove it while saving a draft and then add it back while editing it.

Fetching a draft:

For fetching drafts, a GET request is made to the readDraft.json API. We can either simply call the API without any other parameters which will return all the chatbots of the user. The API call for this looks like this:

https://api.susi.ai/cms/readDraft.json

We can fetch a particular draft by specifying the ID of draft in the API call like this:

https://api.susi.ai/cms/readDraft.json?id=abcdef12

Editing a draft:

The drafts are displayed on the botbuilder page. From there a user can open a draft simply by clicking on it. This uses the ID of that draft to fetch its details by making a GET request to readDraft.json API as specified above.
For applying the details fetched to bot wizard, we pass the draft ID in url of bot wizard as “/botbuilder/botwizard?draftID=abcdef12”.  Then we fetch this ID using “this.getQueryStringValue(‘draftID’)” and this is followed by fetching all details of bot.
After fetching details of the chatbot, all the details are updated in the state of Botwizard component and hence user can continue making SUSI bot.

Deleting a draft:

For the deletion of drafts, a GET request is made to the deleteDraft.json API. We need to specify the id of the draft that we want to delete in the ‘id’ parameter. So, the API call looks like this:

https://api.susi.ai/cms/deleteDraft.json?id=abcdef12

Resources

Continue ReadingDrafts of SUSI Chatbots

Fetching Bots from SUSI.AI Server

Public skills of SUSI.AI are stored in susi_skill_cms repository. A private skill is different from a public skill. It can be viewed, edited and deleted only by the user of who created it. The purpose of private skill is that this acts as a chatbot. All the information of the bot like design, configuration etc is stored within the private skill itself. In order to show the lists of chatbots a user has created to him/her, we need to fetch these lists from the server. The API for fetching private skills is in ListSkillService.java servlet. This servlet is used for both the purposes for fetching public skills and private skills.

How are bots fetched?

All the private skills are stored in data/chatbot/chatbot.json location in the SUSI server. Hence, we can fetch these skills from here.
The API call to fetch private skills is:

https://api.susi.ai/cms/getSkillList.json?private=1&access_token=accessTokenHere

The API call has two parameters:

  1. private = 1: This parameter tells the servlet that we’re asking for private skills because same API is used for fetching public skills as well.
  2. access_token : We have to pass the access-token in order to authenticate the user and access the chatbots of the specific user because the chatbots are stored according to user id of the users in chatbot.json. To get the user id, we need access-token.

For fetching the chatbots from chatbot.json, we firstly authenticate the user. Then we check if the user has any chatbots or not. If the user has chatbots then we loop through chatbot.json to find the chatbots of user by using user id. After getting chatbots details, we simply present it in json structure. You can easily understand this through the following code snippets. For authenticating:

String userId = null;
String privateSkill = call.get("private", null);
if (call.get("access_token", null) != null) {
    ClientCredential credential = new ClientCredential(ClientCredential.Type.access_token, call.get("access_token", null));
    Authentication authentication = DAO.getAuthentication(credential);
    // check if access_token is valid
    if (authentication.getIdentity() != null) {
        ClientIdentity identity = authentication.getIdentity();
        userId = identity.getUuid();
    }
}

You can see that the parameter private is stored in privateSkill. We check if privateSkill is not null followed by checking if the user id provided is valid or not. When both the checks are successful, we create a JSON object in which we store all the bots. To do this, we loop through the chatbot.json file to find the user id provided in API call. If the user id doesn’t exist in chatbot.json file then a message “User has no chatbots.” is displayed. If the user id is found then we again loop through all the chatbots and put their name as the value of key “name”,  language as the value of key “language” and group as the value of key “group”. All these key value pairs for the chatbots are pushed in an array and finally that array is added to the JSON object. This JSON is then received as response of the API call. The following code will demonstrate this:

if(privateSkill != null) {
    if(userId != null) {
        JsonTray chatbot = DAO.chatbot;
        JSONObject result = new JSONObject();
        JSONObject userObject = new JSONObject();
        JSONArray botDetailsArray = new JSONArray();
        JSONArray chatbotArray = new JSONArray();
        for(String user_id : chatbot.keys())
        {
            if(user_id.equals(userId)) {
                userObject = chatbot.getJSONObject(user_id);
                Iterator chatbotDetails = userObject.keys();
                List<String> chatbotDetailsKeysList = new ArrayList<String>();
                while(chatbotDetails.hasNext()) {
                    String key = (String) chatbotDetails.next();
                    chatbotDetailsKeysList.add(key);
                }
                for(String chatbot_name : chatbotDetailsKeysList)
                {
                    chatbotArray = userObject.getJSONArray(chatbot_name);
                    for(int i=0; i<chatbotArray.length(); i++) {
                        String name = chatbotArray.getJSONObject(i).get("name").toString();
                        String language = chatbotArray.getJSONObject(i).get("language").toString();
                        String group = chatbotArray.getJSONObject(i).get("group").toString();
                        JSONObject botDetails = new JSONObject();
                        botDetails.put("name", name);
                        botDetails.put("language", language);
                        botDetails.put("group", group);
                        botDetailsArray.put(botDetails);
                        result.put("chatbots", botDetailsArray);
                    }
                }
            }
        }
        if(result.length()==0) {
            result.put("accepted", false);
            result.put("message", "User has no chatbots.");
            return new ServiceResponse(result);
        }
        result.put("accepted", true);
        result.put("message", "All chatbots of user fetched.");
        return new ServiceResponse(result);
    }
}

So, if chatbot.json contains:

{"c9b58e182ce6466e413d5acafae906ad": {"chatbots": [
 {
   "name": "testing111",
   "language": "en",
   "group": "Knowledge"
 },
 {
   "name": "DNS_Bot",
   "language": "en",
   "group": "Knowledge"
 }
]}}

Then, the JSON received at http://api.susi.ai/cms/getSkillList.json?private=1&access_token=JwTlO8gdCJUns569hzC3ujdhzbiF6I is:

{
  "session": {"identity": {
    "type": "email",
    "name": "test@whatever.com",
    "anonymous": false
  }},
  "chatbots": [
    {
      "name": "testing111",
      "language": "en",
      "group": "Knowledge"
    },
    {
      "name": "DNS_Bot",
      "language": "en",
      "group": "Knowledge"
    }
  ],
  "accepted": true,
  "message": "All chatbots of user fetched."
}

References:

Continue ReadingFetching Bots from SUSI.AI Server

How User preferences data is stored in Chat.susi.ai using Accounting Model

Like any other accounting services SUSI.AI also provides a lot of account preferences. Users can select their language, timezone, themes, speech settings etc. This data helps users to customize their experience when using SUSI.AI.

In the web client these user preferences are fetch from the server by UserPreferencesStore.js and the user identity is fetched by UserIdentityStore.js. These settings are then exported to the settings.react.js file. This file is responsible for the settings page and takes care of all user settings. Whenever a user changes a setting, it identifies the changes and show an option to save these changes. These changes are then updated on the server using the accounting model of SUSI.AI. Let’s take a look at each file discussed above in detail.

Continue ReadingHow User preferences data is stored in Chat.susi.ai using Accounting Model

Play Youtube Videos in SUSI.AI Android app

SUSI.AI Android app has many response functionalities ranging from giving simple ANSWER type responses to complex TABLE and MAP type responses. Although, even after all these useful response types there were some missing action types all related to media. SUSI.AI app was not capable of playing any kind of music or video.So, to do this  the largest source of videos in the world was thought and the functionality to play youtube videos in the app was added.

Since, the app now has two build flavors corresponding to the FDroid version and PlayStore version respectively it had to be considered that while adding the feature to play youtube videos any proprietary software was not included with the FDroid version. Google provides a youtube API that can be used to play videos inside the app only instead of passing an intent and playing the videos in the youtube app.

Steps to integrate youtube api in SUSI.AI

The first step was to create an environment variable that stores the API key, so that each developer that wants to test this functionality can just create an API key from the google API console and put it in the build.gradle file in the line below

def YOUTUBE_API_KEY = System.getenv(‘YOUTUBE_API_KEY’) ?: ‘”YOUR_API_KEY”‘    

In the build.gradle file the buildConfigfield parameter names API_KEY was created so that it can used whenever we need the API_KEY in the code. The buildConfigfield was declared for both the release and debug build types as :

release {
  minifyEnabled false
 proguardFiles getDefaultProguardFile(‘proguard-android.txt’), ‘proguard-rules.pro’
 buildConfigField “String”, ‘YOUTUBE_API_KEY’, YOUTUBE_API_KEY
}

debug {
  buildConfigField “String”, ‘YOUTUBE_API_KEY’, YOUTUBE_API_KEY
}

The second step is to catch the audio and video action type in the ParseSusiResponseHelper.kt file which was done by adding two constants “video_play” and “audio_play” in the Constant class. These actions were easily caught in the app as :

Constant.VIDEOPLAY -> try {
  identifier = susiResponse.answers[0].actions[i].identifier
} catch (e: Exception) {
  Timber.e(e)
}

Constant.AUDIOPLAY -> try {
  identifier = susiResponse.answers[0].actions[i].identifier
} catch (e: Exception) {
  Timber.e(e)
}

The third step involves making a ViewHolder class for the audio and video play actions. A simple layout was made for this viewholder which can display the thumbnail of the video and has a play button on the thumbnail, on clicking which plays the youtube video. Also, in the ChatFeedRecyclerAdapter we need to specify the action as Audio play and video play and then load the specific viewholder for the youtube videos whenever the response from the server for this particular action is fetched. The YoutubeViewHolder.java file describes the class that displays the thumbnail of the youtube video whenever a response arrives.

public void setPlayerView(ChatMessage model) {
  this.model = model;

  if (model != null) {
      try {
          videoId = model.getIdentifier();
          String img_url = “http://img.youtube.com/vi/” + videoId + “/0.jpg”;

          Picasso.with(itemView.getContext()).load(img_url).
                  placeholder(R.drawable.ic_play_circle_filled_white_24dp)
                  .into(playerView);
      } catch (Exception e) {
          Timber.e(e);
      }

  }

The above method shows how the thumbnail is set for a particular youtube video.

The fourth step is to pass the response from the server in the ChatPresenter to play the video asked by the user. This is  achieved by calling a function declared in the IChatView so that the youtube video will be played after fetching the response from the server :

if (psh.actionType == Constant.VIDEOPLAY || psh.actionType == Constant.AUDIOPLAY) {
  // Play youtube video
  identifier = psh.identifier
  chatView?.playVideo(identifier)
}

The fifth step is a bit complicated, as we know the app contains two flavors, one for the FDroid version that contains all the Open Source libraries and software and the second is the PlayStore version which may keep the proprietary software. Since, the youtube api is nothing but a piece of proprietary software we cannot include its implementation in the FDroid version and so different methods were devised to play the youtube videos in both versions. Since,  we use flavors and only want that Youtube API is compiled and included in the playstore version the youtube dependency was updated as :

playStoreImplementation files(‘libs/YouTubeAndroidPlayerApi.jar’)

This ensures that the library is only included in the playStore version. But on this step another problem is encountered, as now the code in both the versions will be different an encapsulation method was devised which enabled the app to keep separate code in both flavors.

To keep different code for both variants, in the src folder directories named fdroid and playStore were created and package name was added and then finally a java folder was created in each directory in which the separate code was kept for each flavor. An interface in the main directory was created which was used as a means to provide encapsulation so that it was implemented differently in each of the flavors to provide different functionality. The interface was created as :

interface IYoutubeVid {
  fun playYoutubeVid(videoId: String)
}

An object of the interface was made initialised in the ChatActivity and separate classes

named YoutubeVid.kt was made in each flavor which implemented the interface IYoutubeVid. So, now what happened was that depending on the build variant the YoutubeVid class of the particular flavor was called and the video would play according the sources that are available in that flavor.

In ChatActivity the following implementation was followed :

Declaration :

lateinit var youtubeVid: IYoutubeVid

Initialisation :

youtubeVid = YoutubeVid(this)

Call to the overridden function :

override fun playVideo(videoId: String) {
  Timber.d(videoId)
  youtubeVid.playYoutubeVid(videoId)
}

Final Output

References

  1. Youtube Android player API : https://developers.google.com/youtube/android/player/
  2. Building apps with product flavors – Tomoaki Imai https://medium.com/@tomoima525/building-multiple-regions-apps-with-product-flavors-good-and-bad-fe33e3c31060
  3. Kotlin style guide : https://android.github.io/kotlin-guides/style.html

 

Continue ReadingPlay Youtube Videos in SUSI.AI Android app

Protected Skills for SUSI.AI Bot Wizard

The first version of SUSI AI web bot plugin is working like a protected skill. A SUSI.AI skill can be created by someone with minimum base user role USER, or in other words, anyone who is logged in.  Anyone can see this skill or edit it. This is not the case with a protected skill or bot. If a skill is protected it becomes a personal bot and then it can only be used by the SUSI AI web client (chatbot) created by that user. Also, only the person who created this skill will be able to edit it or delete it. This skill won’t be listed with all the other public skills on skills.susi.ai.

How is a skill made private?

To make a skill private, a new parameter is added to SusiSkill.java file. This is a boolean called protected. If this parameter is true (Yes) then the skill is protected else if the parameter is false (No) then the skill is not protected. To add protected parameter, we add the following code to SusiSkill.java:

private Boolean protectedSkill;
boolean protectedSkill = false;

You can see that protectedSkill is a boolean with initial value false.
We need to read the value of this parameter in skill code. This is done by the following code:

if (line.startsWith("::protected"&& (thenpos = line.indexOf(' ')) > 0) {
  if (line.substring(thenpos+ 1).trim().equalsIgnoreCase("yes")) protectedSkill=true;
  json.put("protected",protectedSkill);
}

As you can see that the value of protected is read from the skill code. If it’s ‘Yes’ then protectedSkill is set as true and if it’s ‘No’ then protectedSkill is set as false.
If no protected parameter is given in the skill code, its value remains false.

How to add only protected skills from bot wizard?

Now that protected parameter has been added to the server, we need to add this parameter in the skill code but it should be ‘Yes’ if the user is creating a bot and ‘No’ if user is creating a skill using skill creator. In order to do this, we simply check if the user is creating a bot wizard. This can be done by passing a props in the CreateSkill.js file when the skill creator is being used to create a protected skill. Next, we can determine whether the user is using bot wizard or not simply by an if else statement. The following code will demonstrate it:

if (this.props.botBuilder) {
 code = '::protected Yes\n' + code;
else {
 code = '::protected No\n' + code;
}

References:

Continue ReadingProtected Skills for SUSI.AI Bot Wizard

Integrating Gravatar and Anonymizing Email Address in Feedback Section

SUSI skills are having a very nice feedback system that allows the user to rate skills from 1-star to 5-star and showing ratings in skills screens. SUSI also allow the user to post feedback about skills and display them. You can check out how posting feedback implemented here and how displaying feedback feature implemented here. To enhance the user experience, we are adding user gravatar in the feedback section and to respect user privacy, we are anonymizing the user email displayed in the feedback section. In this post, we will see how these features implemented in SUSI iOS.

Integrating Gravatar –

We are showing gravatar of the user before feedback. Gravatar is a service for providing globally-unique avatars. We are using user email address to get the gravatar. The most basic gravatar image request URL looks like this:

https://www.gravatar.com/avatar/HASH

where HASH is replaced with the calculated hash for the specific email address we are requesting. We are using the MD5 hash function to hash the user’s email address.

The MD5 hashing algorithm is a one-way cryptographic function that accepts a message of any length as input and returns as output a fixed-length digest value to be used for authenticating the original message.

In SUSI iOS, we have MD5Digest.swift file that gives the hash value of email string. We are using the following method to set gravatar:

if let userEmail = feedback?.email {
setGravatar(from: userEmail)
}
func setGravatar(from emailString: String) {
let baseGravatarURL = "https://www.gravatar.com/avatar/"
let emailMD5 = emailString.utf8.md5.rawValue
let imageString = baseGravatarURL + emailMD5 + ".jpg"
let imageURL = URL(string: imageString)
gravatarImageView.kf.setImage(with: imageURL)
}

Anonymizing User’s Email Address –

Before the implementation of this feature, the user’s full email address was displayed in the feedback section and see all review screen. To respect the privacy of the user, we are now only showing user email until the `@` sign.

In Feedback object, we have the email address string that we modify to show until `@` sign by following way:

if let userEmail = feedback?.email, let emailIndex = userEmail.range(of: "@")?.upperBound {
userEmailLabel.text = String(userEmail.prefix(upTo: emailIndex)) + "..."
}

 

Final Output –

Resources –

  1. Post feedback for SUSI Skills in SUSI iOS
  2. Displaying Skills Feedback on SUSI iOS
  3. What is MD5?
Continue ReadingIntegrating Gravatar and Anonymizing Email Address in Feedback Section

Fetching responses from SUSI.AI Server for Botbuilder Build Views

In SUSI.AI, we use skill editor for creating and editing public/private skills. The editor we use is Ace editor. The skill is written in a code format documented here. This works fine for a developer but for a user with little experience in coding, this can be confusing. Hence, for providing more clarity as to what the skill does, I built conversation view and tree view along with code view for the skill editor.

Conversation view shows the skill in form of actual conversation between the user and the bot while tree views shows the same conversation in form of a tree. Earlier these views were implemented by converting the code view into an object containing user queries and SUSI responses.

While this works for simple skills, it obviously won’t work for a complex skill in which the responses are fetched from an API. Hence we needed live responses from SUSI server.

This is done similar to how preview works. We pass the whole skill in an instant parameter in the chat.json API along with the user query. This gives us the response from SUSI in form of a JSON.

API Call:

We send a GET request to the following URL:

https://api.susi.ai/susi/chat.json?q=userQuery&instant=wholeSkill

This contains two parameters:

  • q: The user query is passed in this parameter.
  • instant: The whole skill code (present in the code view) is passed in this parameter.

The response is a JSON providing response from SUSI.

Getting user queries:

We can not rely on user to provide the user queries in conversation view and tree view because the user has already provided it in the code view. Hence, we fetch the user queries from code view. This is simply done by dissecting the code and putting all the lines which don’t start with ::, !, #, {, } and “ in an array. Then we split the entries of this array wherever a vertical bar (|) is found. This provides us an array containing all the user queries. It’ll be clear from the following function:

fetchUserInputs = () => {
  let code = this.state.code;
  let userInputs = [];
  let userQueries = [];
  var lines = code.split('\n');
  for (let i = 0; i < lines.length; i++) {
    let line = lines[i];
    if (
      line &&
      !line.startsWith('::') &&
      !line.startsWith('!') &&
      !line.startsWith('#') &&
      !line.startsWith('{') &&
      !line.startsWith('}') &&
      !line.startsWith('"')
    ) {
      let user_query = line;
      while (true) {
        i++;
        if (i >= lines.length) {
          break;
        }
        line = lines[i];
        if (
          line &&
          !line.startsWith('::') &&
          !line.startsWith('!') &&
          !line.startsWith('#')
        ) {
          break;
        }
      }
      userQueries.push(user_query);
    }
  }
  for (let i = 0; i < userQueries.length; i++) {
    let queries = userQueries[i];
    let queryArray = queries.trim().split('|');
    for (let j = 0; j < queryArray.length; j++) {
      userInputs.push(queryArray[j]);
    }
  }
  this.setState({ userInputs }, () => this.getResponses(0));
};

Getting response to a single query at a time:

Now, we have an array containing all the user queries but we can not simply run a loop through the array and then get responses for each query because the AJAX call that we’re making to fetch response is asynchronous. Hence, this will result in multiple AJAX calls in a very short period of time. This will cause a failure in fetching responses and conversation view won’t work. We definitely don’t want that.

To solve this problem, we get response for a single query at a time and make the next AJAX call only when the response for the current call is received. You can see in the code snippet provided in last section, after updating state of userInputs, we’re calling getResponses as a callback and passing 0. This 0 is the index of array which will be incremented on every successful AJAX call. The following code snippet will demonstrate this:

$.ajax({
  type: 'GET',
  url: url,
  contentType: 'application/json',
  dataType: 'json',
  success: function(data) {
  let answer;
  if (data.answers[0]) {
    // Putting response in an object along with user query
    if (responseNumber + 1 === userInputs.length) { // Stopping when responses are fetched for all user queries.
      this.setState({ loaded: true });
    }
    this.setState({ responseData }, () => // updating the response data
      this.getResponses(++responseNumber),  // Incrementing the index and calling getResnponses again as a callback when response data state is updated.
    );
  }.bind(this),
  error: function(err) {
    console.log(err);
  }.bind(this),
});

The code snippets I provided are used in conversation view. Same algorithm is used in tree view as well.

References:

Continue ReadingFetching responses from SUSI.AI Server for Botbuilder Build Views