Making API for Fetching Private Skill Bot Settings from Server

SUSI Server needs to provide an API which would return the private skill’s metadata. This metadata will include the chatbot’s design and configuration settings. The skill’s meta data is being stored in the chatbot.json file in the server. This API will be called from the websites where the chatbot has been deployed. The skill’s metadata will be used for customizing the design of the chatbot. This blog explains how the API for fetching a private skill bot’s metadata is made.

Understanding the API

The API used to fetch public skill’s metadata is /cms/getSkillMetadata.json, with the following parameters:

  • Model
  • Group
  • Language
  • Skill

These parameters help to uniquely identify the skill.

The same API is used to fetch private skill’s metadata too. But the parameters changes to:

  • userid
  • Group
  • Language
  • Skill

The userid also helps to secure the identity of the user. Unlike email id, the user can be easily identified using the userid. Also it prevents from accessing other user’s chatbots without knowing their userid.

Understanding the data stored

In the chatbot.json file, the skill’s metadata are stored in the following format:

{"17a70987d09c33e6f56fe05dca6e3d27": {"Knowledge": {"en": {"test2new1": {
  "design": {
    "botIconColor": "#000000",
    "userMessageTextColor": "#ffffff",
    "botMessageBoxBackground": "#f8f8f8",
    "userMessageBoxBackground": "#0077e5",
    "botMessageTextColor": "#455a64",
    "bodyBackground": "#ffffff"
  "configure": {
    "sites_enabled": ",",
    "sites_disabled": ""
  "timestamp": "2018-07-20 01:04:39.571"


Thus, each entry is stored as a key-value pair. This makes the retrieval of record very efficient.

Making API to fetch private skill bot’s settings

In file, we detect if the client is sending the “userid” parameter or not. For fetching public skill’s metadata, the client will send the “model” parameter and for fetching private skill bot’ settings, the client will send the “userid” parameter. To fetch the private skill bot’s settings, we need to extract it from the chatbot.json file. We fetch the entire object for the particular skill and return it:

// fetch private skill (chatbot) meta data
JsonTray chatbot = DAO.chatbot;
JSONObject json = new JSONObject(true);
JSONObject userObject = chatbot.getJSONObject(userid);
JSONObject groupObject = userObject.getJSONObject(group);
JSONObject languageObject = groupObject.getJSONObject(language);
JSONObject skillObject = languageObject.getJSONObject(skillname);
json.put("skill_metadata", skillObject);
json.put("accepted", true);
json.put("message", "Success: Fetched Skill's Metadata");
return new ServiceResponse(json);



Example API request:

This gives the following output:

  "skill_metadata": {
    "design": {
      "botIconColor": "#000000",
      "userMessageTextColor": "#ffffff",
      "botMessageBoxBackground": "#f8f8f8",
      "userMessageBoxBackground": "#0077e5",
      "botMessageTextColor": "#455a64",
      "bodyBackground": "#ffffff"
    "configure": {
      "sites_enabled": ",",
      "sites_disabled": ""
    "timestamp": "2018-07-20 01:39:55.205"
  "accepted": true,
  "message": "Success: Fetched Skill's Metadata",
  "session": {"identity": {
    "type": "host",
    "name": "",
    "anonymous": true



Continue Reading Making API for Fetching Private Skill Bot Settings from Server

Fetching Private Skill from SUSI Server

SUSI Server needs to provide an API which would return the private skill’s file content. The private skill is used for the botbuilder project. Since the skill is private, the fetching process is little different from that of the public skills. The client has to provide the user’s access_token and a parameter private to access the private skill instead of the public skill. This blog explains how the private skill is being fetched from the SUSI Server.

Understanding the API

The API used to fetch public skill is /cms/getSkill.json, with the following parameters:

  • Model
  • Group
  • Language
  • Skill

These parameters helps to uniquely identify the skill.

The same API is used to fetch private skill too. But the parameters changes to:

  • Access_token
  • Private
  • Group
  • Language
  • Skill

The access token is used to authenticate the user, since the user should only be able to fetch their own private skill. The access token is also used to extract the user’s UUID which will be useful for locating the private skill.

String userId = null;
if (call.get("access_token") != null) { // access tokens can be used by api calls, somehow the stateless equivalent of sessions for browsers
ClientCredential credential = new ClientCredential(ClientCredential.Type.access_token, call.get("access_token"));
Authentication authentication = DAO.getAuthentication(credential);
// check if access_token is valid
if (authentication.getIdentity() != null) {
ClientIdentity identity = authentication.getIdentity();
userId = identity.getUuid();


Fetching the Private Skill from private skill repository

Now that we have all the necessary parameters, we can fetch the private skill from the susi_private_skill_data repository. The susi_private_skill_data folder has the following structure:

Thus to locate the skill, we will need the above parameters which we got from client and the user’s UUID. The following code checks if the client is requesting for private skill and changes the directory path accordingly.

// if client sends private=1 then it is a private skill
String privateSkill = call.get("private", null);
File private_skill_dir = null;
if(privateSkill != null){
private_skill_dir = new File(DAO.private_skill_watch_dir,userId);
String model_name = call.get("model", "general");
File model = new File(DAO.model_watch_dir, model_name);
if(privateSkill != null){
model = private_skill_dir;




Continue Reading Fetching Private Skill from SUSI Server

Store User’s Chatbots in SUSI Server

Users can create their own private skill which corresponds to their susi bot in SUSI Server. We store these private skills inside the susi_private_skill_data directory. We also store the information of the chatbots of each user in chatbot.json file inside the local storage of the server. It contains the list of chatbots created by each user against their UUIDs. This makes it easier and more organised to retrieve the chatbots of the users and their associated settings. This blog explains how the chatbots are being saved in the chatbot.json file in the SUSI server.

Receiving Private Skill from client

The client can create a private skill by accessing the /cms/createSkill.txt API. Along with the other parameters used to create a skill, it also has to send private=1 so that the server can recognise that this is a private skill. The private skills are stored in the folder susi_private_skill_data. The API is made by the file.

protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
    resp.setHeader("Access-Control-Allow-Origin", "*"); // enable CORS
    String userEmail = null;
    JSONObject json = new JSONObject();
    Part imagePart = req.getPart("image");
    if (req.getParameter("access_token") != null) {
        if (imagePart == null) {
            json.put("accepted", false);
            json.put("message", "Image not given");


Getting skill parameters and Saving skill locally

The client sends various parameters related to the skill. Such as the group name, language, skill name, image, etc. Also to identify the skill as a private skill, it needs to send private=1 parameter. If it is a private skill, then we call the function storePrivateSkillBot().

if(privateSkill != null){
    this.storePrivateSkillBot(userId, skill_name, group_name, language_name);
    try (Git git = DAO.getPrivateGit()) {
        // commit the changes
        DAO.pushCommit(git, "Created " + skill_name, userEmail);
        json.put("accepted", true);

    } catch (IOException | GitAPIException e) {
        json.put("message", "error: " + e.getMessage());



Creating the chatbot.json file and saving user’s bot

Inside the function storePrivateSkillBot(), we create the json file in the local storage and save the user’s bot along with their user ids. If the bot with the same name already exists, then we update it or else create a new one.

private static void storePrivateSkillBot(String userId, String skillName, String group, String language) {
    JsonTray chatbot = DAO.chatbot;
    JSONArray userBots = new JSONArray();
    JSONObject botObject = new JSONObject();
    JSONObject userChatBotsObject = new JSONObject();
    if (chatbot.has(userId)) {
        userBots = chatbot.getJSONObject(userId).getJSONArray("chatbots");
        // save a new bot


In chatbot.json file, on creating private skills, the following json gets stored:



Continue Reading Store User’s Chatbots in SUSI Server

Creating API to Retrieve Images in SUSI.AI Server

SUSI Server needed to have an API where users can upload and store images. This images will be useful in setting custom theme for the botbuilder or for User can upload image from their systems instead of pasting the link of an image stored in cloud. Also we need an API to retrieve the stored image given its full path. This blog explains how the SUSI Server returns the image stored upon requesting.

Understanding where the image is stored

The images uploaded to the SUSI Server is stored in its local storage. The local storage is where all the json files and other data are stored. We store the images inside “image_uploads” folder. In the get image API, the client will provide the full path of the image i.e the user UUID and the image’s full name. Then we need to fetch this image from the local storage and return it.

Retrieving the image from local storage

The Retrieving image API has the endpoint “/cms/getImage.png”. The servlet file is “”. The client has to send the full path of the image in the parameter “image”. A sample GET request is “ icon.png”.

We need to retrieve the given image from the correct path.

Query post = RemoteAccess.evaluate(request);
String image_path = post.get("image","");
File imageFile = new File(DAO.data_dir  + File.separator + "image_uploads" + File.separator+ image_path);
if (!imageFile.exists()) {response.sendError(503, "image does not exist"); return;}
String file = image_path.substring(image_path.indexOf("/")+1);


Writing the image in Byte Stream

Now that we have the image file, we need to write that image in a Byte array output stream. We read the pixels data from the image using “FileInputStream” method.

ByteArrayOutputStream data = new ByteArrayOutputStream();
byte[] b = new byte[2048];
InputStream is = new BufferedInputStream(new FileInputStream(imageFile));
int c;
try {
while ((c = >  0) {data.write(b, 0, c);}
} catch (IOException e) {}


Returning the image to the client

Now that we have the image data in the Byte array, we can send that data to the client. Before that we have to set the response type. We set the response type accordingly if the image is png, gif, or jpg. For other types, we the response type as “octet-stream”. Finally we attach the image in the response object and send it.

if (file.endsWith(".png") || (file.length() == 0 && request.getServletPath().endsWith(".png"))) post.setResponse(response, "image/png");
else if (file.endsWith(".gif") || (file.length() == 0 && request.getServletPath().endsWith(".gif"))) post.setResponse(response, "image/gif");
else if (file.endsWith(".jpg") || file.endsWith(".jpeg") || (file.length() == 0 && request.getServletPath().endsWith(".jpg"))) post.setResponse(response, "image/jpeg");
else post.setResponse(response, "application/octet-stream");

ServletOutputStream sos = response.getOutputStream();





Continue Reading Creating API to Retrieve Images in SUSI.AI Server

Creating API to Upload Images in SUSI.AI Server

SUSI Server needed to have an API where users can upload and store images. This images will be useful in setting custom theme for the botbuilder or for User can upload image from their systems instead of pasting the link of an image stored in cloud. This blog explains how the SUSI Server stores the images in its local storage.

Creating Upload Image Service is the file which creates the API to upload image. After creating the UploadImageService class in this file, we need to include the class in to enable the API:

// add services
  services = new Class[]{

Restricting access rights and giving API endpoint name

The Upload Image API should be available only to logged in users and not to anonymous users. Therefore we need to set the base user role to “USER”.  Also we need to set the name of the API endpoint. Lets set it to “/cms/uploadImage.json”.

public UserRole getMinimalUserRole() {
return UserRole.USER;

public String getAPIPath() {
return "/cms/uploadImage.json";


Creating API to accept the Post request

We need to accept the image uploaded by the client in a post request. The post request will contain the following parameters:

  • access_token – used to verify identity of the user and get their UUID
  • image_name – the name of the image
  • image (the image file) – the actual image file

To accept the post request on this route, we define the following function:

protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
resp.setHeader("Access-Control-Allow-Origin", "*"); // enable CORS
JSONObject json = new JSONObject();
Part imagePart = req.getPart("image");
if (req.getParameter("access_token") != null) {
if (imagePart == null) {
json.put("accepted", false);
json.put("message", "Image not given");
} else {
// save image


Getting the correct storage location

We are saving the image in the local storage of the server, i.e inside the “data” folder. Inside it, we store inside the “image_uploads” folder. The image path and name has three parts:

  • User’s UUID as the sub folder’s name
  • System time in milliseconds as the first part of the image’s name
  • The image’s name given by the user as the second part of the image’s name

String image_name = req.getParameter("image_name");
ClientCredential credential = new ClientCredential(ClientCredential.Type.access_token, req.getParameter("access_token"));
Authentication authentication = DAO.getAuthentication(credential);
ClientIdentity identity = authentication.getIdentity();
String userId = identity.getUuid();
String imagePath = DAO.data_dir  + File.separator + "image_uploads" + File.separator + userId;


Saving the image

After we have formed the location of the image to be stored, we need to actually write that file to the local storage. Here is the code snippet which does that:

// Reading content for image
Image image =;
BufferedImage bi = this.toBufferedImage(image);
// Checks if images directory exists or not. If not then create one
if (!Files.exists(Paths.get(imagePath))) new File(imagePath).mkdirs();
String new_image_name = System.currentTimeMillis() + "_" + image_name;
File p = new File(imagePath + File.separator + new_image_name);
if (p.exists()) p.delete();
ImageIO.write(bi, "jpg", new File(imagePath + File.separator + new_image_name));


Thus the image gets stored in the local storage inside of the folder named by user’s UUID

Here, the number 963e84467b92c0916b27d157b1d45328 is the user UUID of the user and 1529692996805 is generated by getting the System time in milliseconds. This helps to differentiate between images of same name uploaded by the same user.


Continue Reading Creating API to Upload Images in SUSI.AI Server

Add More Languages to a Skill in SUSI.AI

The SUSI SKill CMS provides skills in multiple languages. Often there are similar skills in different languages. For example, there is a News skill in English and Samachar skill in Hindi. Then why not link them together and mark one as the translation of the other. This will help the user to reach and explore the desired skill in an efficient way. Moreover, it may be easier to type ‘News’ than ‘समाचार’ and find the required skill through translations. So here it has been explained how to link two SUSI skills as translations.

Server side implementation

Create a skillSupportedLanguages.json file to store the related skills together as translations and make a JSONTray object for that in src/ai/susi/ file. The JSON file contains the language name and the skill name in that language, wrapped in an array.

public static JsonTray skillSupportedLanguages;

Path skillSupportedLanguages_per = skill_status_dir.resolve("skillSupportedLanguages.json");
Path skillSupportedLanguages_vol = skill_status_dir.resolve("skillSupportedLanguages_session.json");
skillSupportedLanguages = new JsonTray(skillSupportedLanguages_per.toFile(), skillSupportedLanguages_vol.toFile(), 1000000);

Now create an API that accepts the skill details and translation details and stores them in the JSON file. Create class for the API.

Endpoint: /cms/updateSupportedLanguages.json

Minimum user role: Anonymous


  • Model
  • Group
  • Language (language of the skill for which translation is to be added)
  • Skill (name of the skill for which translation is to be added)
  • New language (translation language of the skill)
  • New skill name (name of the skill in translated language)

When a new translation is added check if it already exists in the translation group stored in the skillSupportedLanguages.json. Use the DAO object and loop over the array, check is the language name and the language name already exists. If yes then simply return.

if (!alreadyExixts) {
    groupName.put(createSupportedLanguagesArray(language_name, skill_name, new_language_name, new_skill_name));

Otherwise, create a new object containing the new language name and the skill name in that language and add it to the translation group.

public JSONArray createSupportedLanguagesArray(String language_name, String skill_name, String new_language_name, String new_skill_name) {
    JSONArray supportedLanguages =  new JSONArray();

    JSONObject languageObject = new JSONObject();
    languageObject.put("language", language_name);
    languageObject.put("name", skill_name);

    JSONObject newLanguageObject = new JSONObject();
    newLanguageObject.put("language", new_language_name);
    newLanguageObject.put("name", new_skill_name);

    return supportedLanguages;

Add this API to

// Add translation to the skill



Continue Reading Add More Languages to a Skill in SUSI.AI

Adding Support for Playing Audio in SUSI iOS App

SUSI.AI supports various actions like the answer, map, table, video play and many more. You can play youtube videos in the chat screen. It also supports for playing audio in the chat screen. In this post, we will see that how playing audio feature implemented in SUSI iOS.

Getting audio action from server side –

In the chat screen, when we ask SUSI to play audio, we get the audio source from the server side. For example, if we ask SUSI “open the pod bay door”, we get the following action object:

"actions": [
"type": "audio_play",
"identifier_type": "youtube",
"identifier": "7qnd-hdmgfk"
"language": "en",
"type": "answer",
"expression": "I'm sorry, Dave. I'm afraid I can't do that."

In the above action object, we can see that we get two actions, audio_play and answer. In audio_play action, we are getting an identifier type which tells us about the source of audio. Identifier type can be youtube or local or any other source. When the identifier is youtube, we play audio from youtube stream. In identifier, we get the audio file path. In case of youtube identifier type, we get youtube video ID and play from youtube stream. In answer action type, we get the expression which we display in chat screen after thumbnail.

Implementing Audio Support in App –

We use Google’s youtube Iframe API to stream audio from youtube videos. We have a VideoPlayerView that handle all the iFrame API methods and player events with help of YTPlayer HTML file.

Presenting the YouTubePlayerCell –

If the action type is audio_play, we are presenting the cell in chat screen using cellForItemAt method of UICollectionView.

if message.actionType == ActionType.audio_play.rawValue {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.youtubePlayerCell, for: indexPath) as? YouTubePlayerCell {
cell.message = message
cell.delegate = self
return cell

Setting size for cell –

Using sizeForItemAt method of UICollectionView to set the size.

if message.actionType == ActionType.audio_play.rawValue {
return CGSize(width: view.frame.width, height: 158)

In YouTubePlayerCell, we fetch thumbnail and display in the cell with a play button. On clicking the play button, we open the player and stream music.

Final Output –

Resources –

  1. Apple’s Documentations on sizeForItemAt
  2. SUSI API Sample for Audio Play Action
  3. YouTube iFrame API for iOS
  4. Apple’s Documentations on cellForItemAt
Continue Reading Adding Support for Playing Audio in SUSI iOS App

Integrating Gravatar and Anonymizing Email Address in Feedback Section

SUSI skills are having a very nice feedback system that allows the user to rate skills from 1-star to 5-star and showing ratings in skills screens. SUSI also allow the user to post feedback about skills and display them. You can check out how posting feedback implemented here and how displaying feedback feature implemented here. To enhance the user experience, we are adding user gravatar in the feedback section and to respect user privacy, we are anonymizing the user email displayed in the feedback section. In this post, we will see how these features implemented in SUSI iOS.

Integrating Gravatar –

We are showing gravatar of the user before feedback. Gravatar is a service for providing globally-unique avatars. We are using user email address to get the gravatar. The most basic gravatar image request URL looks like this:

where HASH is replaced with the calculated hash for the specific email address we are requesting. We are using the MD5 hash function to hash the user’s email address.

The MD5 hashing algorithm is a one-way cryptographic function that accepts a message of any length as input and returns as output a fixed-length digest value to be used for authenticating the original message.

In SUSI iOS, we have MD5Digest.swift file that gives the hash value of email string. We are using the following method to set gravatar:

if let userEmail = feedback?.email {
setGravatar(from: userEmail)
func setGravatar(from emailString: String) {
let baseGravatarURL = ""
let emailMD5 = emailString.utf8.md5.rawValue
let imageString = baseGravatarURL + emailMD5 + ".jpg"
let imageURL = URL(string: imageString)
gravatarImageView.kf.setImage(with: imageURL)

Anonymizing User’s Email Address –

Before the implementation of this feature, the user’s full email address was displayed in the feedback section and see all review screen. To respect the privacy of the user, we are now only showing user email until the `@` sign.

In Feedback object, we have the email address string that we modify to show until `@` sign by following way:

if let userEmail = feedback?.email, let emailIndex = userEmail.range(of: "@")?.upperBound {
userEmailLabel.text = String(userEmail.prefix(upTo: emailIndex)) + "..."


Final Output –

Resources –

  1. Post feedback for SUSI Skills in SUSI iOS
  2. Displaying Skills Feedback on SUSI iOS
  3. What is MD5?
Continue Reading Integrating Gravatar and Anonymizing Email Address in Feedback Section

Use Timber for Logging in SUSI.AI Android App

As per the official GitHub repository of Timber : “Timber is a logger with a small, extensible API which provides utility on top of Android’s normal Log class”. It is a flexible logging library for Android which makes logging a lot more convenient. In this blog you will learn how to use Timber in SUSI.AI Android app.

To begin, add Timber to your build.gradle and refresh your gradle dependencies.

implementation 'com.jakewharton.timber:timber:4.7.0’

Two easy steps to use Timber:

  1. Install any Tree instances that you want in the onCreate() of your application class.
  2. Call Timber’s static methods everywhere throughout the app.

You can add the following code to your application class :

   public void onCreate() {

       if (BuildConfig.DEBUG) {
           Timber.plant(new Timber.DebugTree() {
               //Add the line number to the tag
               protected String createStackElementTag(StackTraceElement element) {
                   return super.createStackElementTag(element) + ": " + element.getLineNumber();
       } else {
           //Release mode
           Timber.plant(new ReleaseLogTree());

   private static class ReleaseLogTree extends Timber.Tree {

       protected void log(int priority, String tag, @NonNull String message,  
                                     Throwable throwable) {
           if (priority == Log.DEBUG || priority == Log.VERBOSE || priority == Log.INFO) {                           

           if (priority == Log.ERROR) {
               if (throwable == null) {
               } else {
                   Timber.e(throwable, message);

Timber ships with a ‘Debug Tree’ that provides all the basic facilities that you are very used to in the common Android.log framework. Timber has all the logging levels that are used in the normal Android logging framework which are as follows:

        • Log.e: Use this tag in places like inside a catch statement where you are aware that an error has occurred and therefore you’re logging an error.
        • Log.w: Use this to log stuff you didn’t expect to happen but isn’t necessarily an error.
        • Log.i: Use this to post useful information to the log. For instance, a message that you have successfully connected to a server.
        • Log.d: Use this for debugging purposes. For instance, if you want to print out a bunch of messages so that you can log the exact flow of your program or if you want to keep a log of variable values.
        • Log.v: Use this if, for some reason, you need to log every little thing in a particular part of your app.
        • Use this when you encounter a terrible failure.


    • You can use Timber.e, Timber.w, Timber.i, Timber.d and Timber.v respectively for the logging levels mentioned above. However, there is a small difference. In Android logging framework the exception is passed as the last parameter. But, when using Timber, you need to provide the exception as the first parameter. Interestingly, while using Timber you don’t have to provide a TAG in logging calls because it automatically does that for you. It uses the file name, where you are logging from, as the TAG. To log an exception you can simply write:
    • Timber.e(exception, "Message");
    • Or a null message if you want:
    • Timber.e(exception, null);
    • A simple message can be logged sent via Crashlytics too:
    • Timber.e("An error message");

Did you notice? There is no TAG parameter. It is automatically assigned as the caller class’ name. Now, you can use Timber for logging in SUSI Android app.



Continue Reading Use Timber for Logging in SUSI.AI Android App

Connecting to a Raspberry Pi through a SSH connection Wirelessly

The tech stack of the SUSI.AI smart speaker project is mainly Python/Bash scripts. Every smart speaker has an essential feature that allows the user’s mobile device to connect and give instructions to the speaker wirelessly. To make this connection possible, we are trying to implement this using an SSH connection.

Why SSH?

SSH(a.k.a Secure Shell) is a cryptographic connection which allows secure transfer of data even over an unsecured connection.SSH connection even allows TCP as well as X11 forwarding which are an added bonus.

Step 1: Initial Setup

  • Both the raspberry Pi with raspbian installed and the mobile device should be on a same wireless network
  • One should have an SSH viewer like JuiceSSH(Android) and iTerminal(IOS) installed on their mobile devices
  • Now we must enable SSH on our raspberry Pi

Step 2: Enabling SSH on Raspberry PI

  • To enable SSH on your Pi , follow the steps mentioned below:
Menu > Preferences > Raspberry Pi Configuration.

Choose the interfaces tab and enable SSH

Step 3:Setting Up the client


  • Login to your raspberry pi as the root user (pi by default)
  • Type the following command to know the broadcasting ip address
[email protected]:hostname -I


  • Now , open the client on your mobile device and add the configurations

By default the username of the system is ‘pi’ and the password is ‘raspberry’

Step 4: Changing the default SSH password

Since the default password of every RaspberryPi is the same. So , the pi can be accessed by any device that has access to the local network which is not a secure way of accessing the device

  • In the SSH window type ‘passwd’
  • Type the current password
  • Type the new password
  • Re-enter the new password

Now you will be able to login to your raspberry through an SSH connection



Fossasia, gsoc, gsoc’18, susi,, hardware,susi_linux

Continue Reading Connecting to a Raspberry Pi through a SSH connection Wirelessly