Server Side Implement of Usage Analytics of a Skill in SUSI.AI

Which skills are being used most frequently in SUSI.AI? And which are rarely used? Such statistics are needed to give proper responses from the most used skills.

So when the user sends a query to SUSI, it searches for the best-suited skill to respond to that query. The usage count of that skill is incremented accordingly with the date.

Storage of Skill Usage Data

  1. Create a skillUsage.json file to store the stats and make a JSONTray object for that in DAO.java file.
  2. Modify the SusiArguement.java file to check which skill is being currently used for the response and write the skill usage stats to skillUsage.json.The function updateUsageData takes the skillPath in parameter and splits it from slash character. Thus the model name, group name, language name and skill name can be obtained. Then it searches for the date in the JSON file. If it exists then its count value is increased by 1, else a new object is inserted in the JSON file that contains the date and the corresponding usage count.

public void updateUsageData(String skillPath) {
    Boolean dateExists = false;
    for (int i = 0; i<usageData.length(); i++) {
        JSONObject dayUsage = usageData.getJSONObject(i);
        if (dayUsage.get("date").equals(today)){
            dayUsage.put("count", dayUsage.getInt("count")+1+"");
            usageData.put(i,dayUsage);
            dateExists = true;
            break;
        }
    }
}   

API to access the Skill Usage Data

  1. Create GetSkillUsageService.java file to return the usage stats stored in skillUsage.json. It runs at the endponit – /cms/getSkillUage.json and mining role required to access this API is anonymous. It simply reads the data stored in skillUsage.json file and returns it.
JsonTray skillRating = DAO.skillUsage;
JSONObject  languageName = groupName.getJSONObject(language_name);
if (languageName.has(skill_name)) {
    JSONArray skillUsage = languageName.getJSONArray(skill_name);
    result.put("skill_name", skill_name);
    result.put("skill_usage", skillUsage);
    result.put("accepted", true);
    result.put("message", "Skill usage fetched");
    return new ServiceResponse(result);
}
  1. Add the API file to skillUsage.json
services = new Class[]{
	...
	//Skill usage data
	GetSkillUsageService.class
	...
}

So now we know which skills are being used at what rate and make the required changes like server scaling, adding more accuracy to the feature etc. We can expand these stats to country wise or device wise usage distribution.

Resources

 

Continue ReadingServer Side Implement of Usage Analytics of a Skill in SUSI.AI

Limit the Chatbots to Run on Specified Domains

SUSI botbuilder enables users to make their own private skill and deploy a chatbot widget in their websites. Users can copy paste a javascript code into their website’s source code to activate the bot. But what if someone copies that code from your website and put it in their own website? You won’t want the chat bot to work for such users in some cases. Thus we have a feature through which the bot creator can restrict the usage of the chatbot to only certain domains. The chat bot will not work from other domains.

Understanding the APIs Used

In working of the chatbot, there are mainly two APIs used from the server which play a mainstream role. The first API is the cms/getSkillMetaData.json API. It is used to get the design and configurations of the chatbot. The second API is the susi/chat.json API. It is used to get responses from the server applying the private skill. By restricting the chatbot usage we try to restrict the usage of these two APIs. Also, on the client side we display the chatbot only if the server sends a valid response indicating that the chatbot is legitimate. However, this can be circumvented if the person modifies the javascript of the chatbot. Hence, we need to secure the above two APIs. We check the domain from where the request is coming by checking the referer field in the request’s header.

Securing the APIs

In each of the above two APIs, we check if the bot owner has checked “allow bot only on own site”. If no, then the APIs can be accessed from any domain we need not check. If selected yes, then we need to check if the current site’s domain is allowed in the allowed sites list. For this, we extract the current domain from the request’s referer field. The allowed sites list is fetched from the configure object of that skill.

public static boolean allowDomainForChatbot(JSONObject configureObject, String referer) {
    Boolean allowed_site = true;
    if (configureObject.getBoolean("allow_bot_only_on_own_sites") && configureObject.has("allowed_sites") && configureObject.getString("allowed_sites").length() > 0) {
        allowed_site = false;
        if (referer != null && referer.length() > 0) {
            String[] sites = configureObject.getString("allowed_sites").split(",");
            for (int i = 0; i < sites.length; i++) {
                String site = sites[i].trim();
                int referer_index = referer.indexOf("://");
                String host = referer;
                if (referer.indexOf('/',referer_index+3) > -1) {
                    host = referer.substring(0,referer.indexOf('/',referer_index+3));
                }
                if (host.equalsIgnoreCase(site)) {
                    allowed_site = true;
                    break;
                }
            }
        }
    }
    return allowed_site; 
}

 

Result

Not allowed from other domains:
(For getSkillMetaData.json API)

Allowed on approved domains:
(For getSkillMetaData.json API)

Resources

 

Continue ReadingLimit the Chatbots to Run on Specified Domains

Applying Private Skill as Web Bots in SUSI Chat

Along with public skills, we now have private skills as web bots! Users can create their own private skills which can be used only by them and in the chatbots deployed by them. The SUSI Server accepts parameters to identify a valid private skill and applies that private skill for a particular chat. It then executes the query and sends the response to the client. This blog explains how private skill is applied in the SUSI Chat.

Understanding the API

The API to receive response from SUSI is /susi/chat.json. For applying only the public skills, we can send a request like /susi/chat.json?q=hello. Here only one parameter “q” is involved in which we send the query. However, for a private skill, the following parameters are involved:

  • privateskill – when the client sends this parameter, it indicates to use a private skill.
  • userid – the userid of the user who has created the private skill
  • group – the group name of the private skill
  • language – the language of the private skill
  • skill – the skill name of the private skill

Thus, the four parameters userid, group, language, skill serves to uniquely identify a private skill.

Fetching the private skill

After the client sends the appropriate parameters to apply a private skill, the server must actually apply the private skill. This is done in a similar manner how persona and dream skills are applied. The First step is the fetch the private skill from the susi_private_skill_data folder.

// read the private skill
File private_skill_dir = new File(DAO.private_skill_watch_dir,userId);
File group_file = new File(private_skill_dir, group_name);
File language_file = new File(group_file, language);
skillfile = SusiSkill.getSkillFileInLanguage(language_file, skill_name, false);
String text = new String(Files.readAllBytes(skillfile.toPath()), StandardCharsets.UTF_8);

Applying the private skill

After we have fetched the private skill, the next step is to apply it. To do this, we create a SusiMind object and add it to the general minds variable. Thus the skill will be applied to the particular chat with the highest priority, since it will be the first skill to be added to the minds variable. Later, the public skills can be added to the minds variable, thus their priority will be lower than the private skill.

String text = new String(Files.readAllBytes(skillfile.toPath()), StandardCharsets.UTF_8);
// fill an empty mind with the private skill
SusiMind awakeMind = new SusiMind(DAO.susi_chatlog_dir, DAO.susi_skilllog_dir); // we need the memory directory here to get a share on the memory of previous dialoges, otherwise we cannot test call-back questions
JSONObject rules = SusiSkill.readLoTSkill(new BufferedReader(new InputStreamReader(new ByteArrayInputStream(text.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8)), SusiLanguage.unknown, dream);
awakeMind.learn(rules, skillfile);
SusiSkill.ID skillid = new SusiSkill.ID(skillfile);
SusiSkill activeskill = awakeMind.getSkillMetadata().get(skillid);
awakeMind.setActiveSkill(activeskill);
// we are awake!
minds.add(awakeMind);

 

Result

Example API: http://localhost:4000/susi/chat.json?q=hi&privateskill=1&userid=17a70987d09c33e6f56fe05dca6e3d27&group=Knowledge&language=en&skill=knowprides

The skill exists in the correct location:

The skill file content is:

Thus, on sending the query “tell me” with the other parameters, we get the correct reply i.e “yes sure” from the server:

Resources

 

 

Continue ReadingApplying Private Skill as Web Bots in SUSI Chat

Making API for Fetching Private Skill Bot Settings from Server

SUSI Server needs to provide an API which would return the private skill’s metadata. This metadata will include the chatbot’s design and configuration settings. The skill’s meta data is being stored in the chatbot.json file in the server. This API will be called from the websites where the chatbot has been deployed. The skill’s metadata will be used for customizing the design of the chatbot. This blog explains how the API for fetching a private skill bot’s metadata is made.

Understanding the API

The API used to fetch public skill’s metadata is /cms/getSkillMetadata.json, with the following parameters:

  • Model
  • Group
  • Language
  • Skill

These parameters help to uniquely identify the skill.

The same API is used to fetch private skill’s metadata too. But the parameters changes to:

  • userid
  • Group
  • Language
  • Skill

The userid also helps to secure the identity of the user. Unlike email id, the user can be easily identified using the userid. Also it prevents from accessing other user’s chatbots without knowing their userid.

Understanding the data stored

In the chatbot.json file, the skill’s metadata are stored in the following format:

{"17a70987d09c33e6f56fe05dca6e3d27": {"Knowledge": {"en": {"test2new1": {
  "design": {
    "botIconColor": "#000000",
    "userMessageTextColor": "#ffffff",
    "botMessageBoxBackground": "#f8f8f8",
    "userMessageBoxBackground": "#0077e5",
    "botMessageTextColor": "#455a64",
    "bodyBackground": "#ffffff"
  },
  "configure": {
    "sites_enabled": "website1.com, website2.com",
    "sites_disabled": "website3.com"
  },
  "timestamp": "2018-07-20 01:04:39.571"
}}}}}

 

Thus, each entry is stored as a key-value pair. This makes the retrieval of record very efficient.

Making API to fetch private skill bot’s settings

In GetSkillMetadataService.java file, we detect if the client is sending the “userid” parameter or not. For fetching public skill’s metadata, the client will send the “model” parameter and for fetching private skill bot’ settings, the client will send the “userid” parameter. To fetch the private skill bot’s settings, we need to extract it from the chatbot.json file. We fetch the entire object for the particular skill and return it:

// fetch private skill (chatbot) meta data
JsonTray chatbot = DAO.chatbot;
JSONObject json = new JSONObject(true);
JSONObject userObject = chatbot.getJSONObject(userid);
JSONObject groupObject = userObject.getJSONObject(group);
JSONObject languageObject = groupObject.getJSONObject(language);
JSONObject skillObject = languageObject.getJSONObject(skillname);
json.put("skill_metadata", skillObject);
json.put("accepted", true);
json.put("message", "Success: Fetched Skill's Metadata");
return new ServiceResponse(json);

 

Result

Example API request: http://127.0.0.1:4000/cms/getSkillMetadata.json?userid=17a70987d09c33e6f56fe05dca6e3d27&group=Knowledge&language=en&skill=testnew

This gives the following output:

{
  "skill_metadata": {
    "design": {
      "botIconColor": "#000000",
      "userMessageTextColor": "#ffffff",
      "botMessageBoxBackground": "#f8f8f8",
      "userMessageBoxBackground": "#0077e5",
      "botMessageTextColor": "#455a64",
      "bodyBackground": "#ffffff"
    },
    "configure": {
      "sites_enabled": "website1.com, website2.com",
      "sites_disabled": "website3.com"
    },
    "timestamp": "2018-07-20 01:39:55.205"
  },
  "accepted": true,
  "message": "Success: Fetched Skill's Metadata",
  "session": {"identity": {
    "type": "host",
    "name": "127.0.0.1_af2c1fe3",
    "anonymous": true
  }}
}

 

Resources

Continue ReadingMaking API for Fetching Private Skill Bot Settings from Server
Read more about the article Adding Option to Choose Room for SUSI Smart Speaker in iOS App
SAMSUNG CAMERA PICTURES

Adding Option to Choose Room for SUSI Smart Speaker in iOS App

SUSI Smart Speaker is an open source smart speaker that supports many features. The user can use Android or iOS to connect their device with SUSI Smart Speaker. During initial installation, it is asking from use to enter the Room name. Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them. You can find useful instructions for the initial connection between the iOS device and SUSI Smart Speaker here. It this post, we will see how the adding rooms feature implemented for SUSI iOS.

When the user enters into the Device Activity screen, we check if the iOS device is connected to SUSI.AI Wi-Fi hotspot or not. If the device is connected to SUSI Smart Speaker, it shows the Wi-Fi displayed SSID in Device Activity Screen. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field. The user can enter Room location and by clicking the Next button, proceed further with the setup.

In the popup, there is also an option for choosing rooms, where the list of most common room names is displayed and the user can choose room name from the list.

Presenting Room Picker View Controller –

func presentRoomsPicker() {
let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let roomVC = storyboard.instantiateViewController(withIdentifier: "RoomPickerVC") as? RoomPickerController {
roomVC.deviceActivityVC = self
let roomNVC = AppNavigationController(rootViewController: roomVC)
self.present(roomNVC, animated: true)
}
}

Room Picker View Controller is UITableViewController that display the rooms names in table cells. Some of the most common rooms names displayed are:

let rooms: [String] = ["Bedroom", "Kitchen", "Family Room", "Entryway", "Living Room", "Front Yard", "Guest Room", "Dining Room", "Computer Room", "Downstairs", "Front Porch", "Garage", "Hallway", "Driveway"]

 

Presenting Room Cell –

We are using cellForRowAt method to present the cell.

override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCell(withIdentifier: "roomsCell", for: indexPath)
cell.textLabel?.text = rooms[indexPath.row]
cell.imageView?.image = ControllerConstants.Images.roomsIcon
return cell
}

 

Selecting the room from cell –

When the user clicks on the cell, first willSelectRowAt method use to display the right icon in the accessory view that shows which cell is selected.

if let oldIndex = tableView.indexPathForSelectedRow {
tableView.cellForRow(at: oldIndex)?.accessoryType = .none
}
tableView.cellForRow(at: indexPath)?.accessoryType = .checkmark

We are storing the selected room in the following variable and selecting it by using didSelectRowAt method of UITableView.

var selectedRoom: String?
override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
selectedRoom = rooms[indexPath.row]
}

In Room Picker Screen, the user has two option, Cancel and Done. If the user clicks the Cancel, we dismiss the Room Picker screen and display the popup with the empty room location text field and with Choose Room option. If the user clicks the Done button, we store the picked room in UserDefaults shared instance and dismiss Room Picker screen with a different popup which has already filled room location in the text field and without choose room option in the popup as in the image below. By clicking the next, the user proceeds with the further setup.

Resources –

  1. Apple’s Documentations on UserDefaults.
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. Apple’s Documentations on tableView(_:cellForRowAt:)
  4. Apple’s Documentations on tableView(_:willSelectRowAt:)
  5. Apple’s Documentations on tableView(_:didSelectRowAt:)
Continue ReadingAdding Option to Choose Room for SUSI Smart Speaker in iOS App

Fetching Private Skill from SUSI Server

SUSI Server needs to provide an API which would return the private skill’s file content. The private skill is used for the botbuilder project. Since the skill is private, the fetching process is little different from that of the public skills. The client has to provide the user’s access_token and a parameter private to access the private skill instead of the public skill. This blog explains how the private skill is being fetched from the SUSI Server.

Understanding the API

The API used to fetch public skill is /cms/getSkill.json, with the following parameters:

  • Model
  • Group
  • Language
  • Skill

These parameters helps to uniquely identify the skill.

The same API is used to fetch private skill too. But the parameters changes to:

  • Access_token
  • Private
  • Group
  • Language
  • Skill

The access token is used to authenticate the user, since the user should only be able to fetch their own private skill. The access token is also used to extract the user’s UUID which will be useful for locating the private skill.

String userId = null;
if (call.get("access_token") != null) { // access tokens can be used by api calls, somehow the stateless equivalent of sessions for browsers
ClientCredential credential = new ClientCredential(ClientCredential.Type.access_token, call.get("access_token"));
Authentication authentication = DAO.getAuthentication(credential);
// check if access_token is valid
if (authentication.getIdentity() != null) {
ClientIdentity identity = authentication.getIdentity();
userId = identity.getUuid();
}
}

 

Fetching the Private Skill from private skill repository

Now that we have all the necessary parameters, we can fetch the private skill from the susi_private_skill_data repository. The susi_private_skill_data folder has the following structure:

Thus to locate the skill, we will need the above parameters which we got from client and the user’s UUID. The following code checks if the client is requesting for private skill and changes the directory path accordingly.

// if client sends private=1 then it is a private skill
String privateSkill = call.get("private", null);
File private_skill_dir = null;
if(privateSkill != null){
private_skill_dir = new File(DAO.private_skill_watch_dir,userId);
}
String model_name = call.get("model", "general");
File model = new File(DAO.model_watch_dir, model_name);
if(privateSkill != null){
model = private_skill_dir;
}

 

Result:

Resources

Continue ReadingFetching Private Skill from SUSI Server

Store User’s Chatbots in SUSI Server

Users can create their own private skill which corresponds to their susi bot in SUSI Server. We store these private skills inside the susi_private_skill_data directory. We also store the information of the chatbots of each user in chatbot.json file inside the local storage of the server. It contains the list of chatbots created by each user against their UUIDs. This makes it easier and more organised to retrieve the chatbots of the users and their associated settings. This blog explains how the chatbots are being saved in the chatbot.json file in the SUSI server.

Receiving Private Skill from client

The client can create a private skill by accessing the /cms/createSkill.txt API. Along with the other parameters used to create a skill, it also has to send private=1 so that the server can recognise that this is a private skill. The private skills are stored in the folder susi_private_skill_data. The API is made by the CreateSkillService.java file.

protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
    resp.setHeader("Access-Control-Allow-Origin", "*"); // enable CORS
    String userEmail = null;
    JSONObject json = new JSONObject();
    Part imagePart = req.getPart("image");
    if (req.getParameter("access_token") != null) {
        if (imagePart == null) {
            json.put("accepted", false);
            json.put("message", "Image not given");
        } 

 

Getting skill parameters and Saving skill locally

The client sends various parameters related to the skill. Such as the group name, language, skill name, image, etc. Also to identify the skill as a private skill, it needs to send private=1 parameter. If it is a private skill, then we call the function storePrivateSkillBot().

if(privateSkill != null){
    this.storePrivateSkillBot(userId, skill_name, group_name, language_name);
    try (Git git = DAO.getPrivateGit()) {
        git.add().addFilepattern(".").call();
        // commit the changes
        DAO.pushCommit(git, "Created " + skill_name, userEmail);
        json.put("accepted", true);

    } catch (IOException | GitAPIException e) {
        e.printStackTrace();
        json.put("message", "error: " + e.getMessage());

    }
}

 

Creating the chatbot.json file and saving user’s bot

Inside the function storePrivateSkillBot(), we create the json file in the local storage and save the user’s bot along with their user ids. If the bot with the same name already exists, then we update it or else create a new one.

private static void storePrivateSkillBot(String userId, String skillName, String group, String language) {
    JsonTray chatbot = DAO.chatbot;
    JSONArray userBots = new JSONArray();
    JSONObject botObject = new JSONObject();
    JSONObject userChatBotsObject = new JSONObject();
    if (chatbot.has(userId)) {
        userBots = chatbot.getJSONObject(userId).getJSONArray("chatbots");
    }
        // save a new bot
    botObject.put("name",skillName);
    botObject.put("group",group);
    botObject.put("language",language);
    userBots.put(botObject);
    userChatBotsObject.put("chatbots",userBots);
    chatbot.put(userId,userChatBotsObject,true);
} 

 

In chatbot.json file, on creating private skills, the following json gets stored:

Resources

 

Continue ReadingStore User’s Chatbots in SUSI Server

Adding Functionality to Switch between List and Grid View of the Skill Cards on SUSI.AI CMS

In this blog post, we are going to understand the implementation of the functionality that enables the user to switch between the List View and the Grid View UI for the skill cards that is displayed on various routes of the SUSI Skill CMS Web-App. Let us go through the implementation in the blog –

Working of the feature

Going through the implementation

  • The UI for implementing the switching of views was achieved via the use of RadioButtonGroup component of the Material-UI library for React.
  • The type of view currently being shown was stored in the component state of the BrowseSkill component as viewType, whose default value is set to list, indicating that the skills are firstly shown in a List View.

.
.

export default class BrowseSkill extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
        .
        …..
        viewType: 'list',
        ….
    };
  }
….
}

 

  • The RadioButtonGroup component has 2 child components, for each view. The child component that is to be used is RadioButton.
  • The props passed in the RadioButtonGroup are –
    • name : It is the name given to the component.
    • defaultSelected : It is the default view type.
    • style : It contains the style object of the UI.
    • valueSelected : It is set to the state variable assigned for storing view type.
    • onChange : It is the handler which executes, when the radio buttons are clicked.
  • The style for the desktop view and mobile view is different depending on the screen size and is follows –

//Mobile view
Style {
    right: 12,
    position: 'absolute',
    top: 216,
    display: 'flex',
}

//Desktop view
style={
    display: 'flex',
    marginTop: 34
}

 

  • The props passed in the RadioButton are –
    • value : The value stored in the state, that is responsible for the view type. The values for List and Grid view are list and grid respectively.
    • label : The label for the RadioButton.
    • labelStyle : The style object for the label.
    • checkedIcon : The icon used in the checked state.
    • uncheckedIcon : The icon used in the unchecked state.

UI of the Radio Buttons

  • The onClick handler of the radio buttons is –

handleViewChange = (event, value) => {
    this.setState({ viewType: value });
};

 

  • The code snippet for the UI implementation, written inside the render function is as follows :

….
<RadioButtonGroup
  name="view_type"
  defaultSelected="list"
  style={
    window.innerWidth < 430
      ? {
          right: 12,
          position: 'absolute',
          top: 216,
          display: 'flex',
        }
      : { display: 'flex', marginTop: 34 }
  }
  valueSelected={this.state.viewType}
  onChange={this.handleViewChange}
>
  <RadioButton
    value="list"
    label="List view"
    labelStyle={{ display: 'none' }}
    style={{ width: 'fit-content' }}
    checkedIcon={
      <ActionViewStream style={{ fill: '#4285f4' }} />
    }
    uncheckedIcon={<ActionViewStream />}
  />
  <RadioButton
    value="grid"
    label="Grid view"
    labelStyle={{ display: 'none' }}
    style={{ width: 'fit-content' }}
    checkedIcon={
      <ActionViewModule style={{ fill: '#4285f4' }} />
    }
    uncheckedIcon={<ActionViewModule />}
  />
</RadioButtonGroup>
….

 

I hope the implementation of the switching between Views would be clear after going through the blog and proved to be helpful for your understanding.

References

Continue ReadingAdding Functionality to Switch between List and Grid View of the Skill Cards on SUSI.AI CMS

Add More Languages to a Skill in SUSI.AI

The SUSI SKill CMS provides skills in multiple languages. Often there are similar skills in different languages. For example, there is a News skill in English and Samachar skill in Hindi. Then why not link them together and mark one as the translation of the other. This will help the user to reach and explore the desired skill in an efficient way. Moreover, it may be easier to type ‘News’ than ‘समाचार’ and find the required skill through translations. So here it has been explained how to link two SUSI skills as translations.

Server side implementation

Create a skillSupportedLanguages.json file to store the related skills together as translations and make a JSONTray object for that in src/ai/susi/DAO.java file. The JSON file contains the language name and the skill name in that language, wrapped in an array.

public static JsonTray skillSupportedLanguages;

Path skillSupportedLanguages_per = skill_status_dir.resolve("skillSupportedLanguages.json");
Path skillSupportedLanguages_vol = skill_status_dir.resolve("skillSupportedLanguages_session.json");
skillSupportedLanguages = new JsonTray(skillSupportedLanguages_per.toFile(), skillSupportedLanguages_vol.toFile(), 1000000);
OS.protectPath(skillSupportedLanguages_per);
OS.protectPath(skillSupportedLanguages_vol);

Now create an API that accepts the skill details and translation details and stores them in the JSON file. Create UpdateSupportedLanguages.java class for the API.

Endpoint: /cms/updateSupportedLanguages.json

Minimum user role: Anonymous

Params:

  • Model
  • Group
  • Language (language of the skill for which translation is to be added)
  • Skill (name of the skill for which translation is to be added)
  • New language (translation language of the skill)
  • New skill name (name of the skill in translated language)

When a new translation is added check if it already exists in the translation group stored in the skillSupportedLanguages.json. Use the DAO object and loop over the array, check is the language name and the language name already exists. If yes then simply return.

if (!alreadyExixts) {
    groupName.put(createSupportedLanguagesArray(language_name, skill_name, new_language_name, new_skill_name));
}

Otherwise, create a new object containing the new language name and the skill name in that language and add it to the translation group.

public JSONArray createSupportedLanguagesArray(String language_name, String skill_name, String new_language_name, String new_skill_name) {
    JSONArray supportedLanguages =  new JSONArray();

    JSONObject languageObject = new JSONObject();
    languageObject.put("language", language_name);
    languageObject.put("name", skill_name);
    supportedLanguages.put(languageObject);

    JSONObject newLanguageObject = new JSONObject();
    newLanguageObject.put("language", new_language_name);
    newLanguageObject.put("name", new_skill_name);
    supportedLanguages.put(newLanguageObject);

    return supportedLanguages;
}

Add this API to SusiServer.java

// Add translation to the skill
UpdateSupportedLanguages.class

Resources

 

Continue ReadingAdd More Languages to a Skill in SUSI.AI

Adding Support for Playing Audio in SUSI iOS App

SUSI.AI supports various actions like the answer, map, table, video play and many more. You can play youtube videos in the chat screen. It also supports for playing audio in the chat screen. In this post, we will see that how playing audio feature implemented in SUSI iOS.

Getting audio action from server side –

In the chat screen, when we ask SUSI to play audio, we get the audio source from the server side. For example, if we ask SUSI “open the pod bay door”, we get the following action object:

"actions": [
{
"type": "audio_play",
"identifier_type": "youtube",
"identifier": "7qnd-hdmgfk"
},
{
"language": "en",
"type": "answer",
"expression": "I'm sorry, Dave. I'm afraid I can't do that."
}
]

In the above action object, we can see that we get two actions, audio_play and answer. In audio_play action, we are getting an identifier type which tells us about the source of audio. Identifier type can be youtube or local or any other source. When the identifier is youtube, we play audio from youtube stream. In identifier, we get the audio file path. In case of youtube identifier type, we get youtube video ID and play from youtube stream. In answer action type, we get the expression which we display in chat screen after thumbnail.

Implementing Audio Support in App –

We use Google’s youtube Iframe API to stream audio from youtube videos. We have a VideoPlayerView that handle all the iFrame API methods and player events with help of YTPlayer HTML file.

Presenting the YouTubePlayerCell –

If the action type is audio_play, we are presenting the cell in chat screen using cellForItemAt method of UICollectionView.

if message.actionType == ActionType.audio_play.rawValue {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.youtubePlayerCell, for: indexPath) as? YouTubePlayerCell {
cell.message = message
cell.delegate = self
return cell
}
}

Setting size for cell –

Using sizeForItemAt method of UICollectionView to set the size.

if message.actionType == ActionType.audio_play.rawValue {
return CGSize(width: view.frame.width, height: 158)
}

In YouTubePlayerCell, we fetch thumbnail and display in the cell with a play button. On clicking the play button, we open the player and stream music.

Final Output –

Resources –

  1. Apple’s Documentations on sizeForItemAt
  2. SUSI API Sample for Audio Play Action
  3. YouTube iFrame API for iOS
  4. Apple’s Documentations on cellForItemAt
Continue ReadingAdding Support for Playing Audio in SUSI iOS App