Displaying Private Skills and Drafts on SUSI.AI

The ListPrivateSkillService and ListPrivateDraftSkillService endpoint was implemented on SUSI.AI Server for SUSI.AI Admins to view the bots and drafts created by users respectively. This allows admins to monitor the bots and drafts created by users, and delete the ones which violate the guidelines. Also admins can see the sites where the bot is being used.

The endpoint of both ListPrivateSkillService and ListPrivateDraftSkillService is of GET type. Both of them have a compulsory access_token parameter but ListPrivateSkillService has an extra optional search parameter.

  • access_token(necessary): It is the access_token of the logged in user. It means this endpoint cannot be accessed in anonymous mode. 
  • search: It fetches a bot with the searched name.

The minimum user role is set to OPERATOR.

API Development

ListPrivateSkillService

For creating a list, we need to access each property of botDetailsObject, in the following manner:

Key → Group  → Language → Bot Name  → BotList

The below code iterates over the uuid of all the users having a bot, then over different groupNames,languageNames, and finally over the botNames. If search parameter is passed then it searches for the bot_name in the language object. Each botDetails object consists of bot name, language, group and key i.e uuid of the user which is then added to the botList array.

       JsonTray chatbot = DAO.chatbot;
       JSONObject botDetailsObject = chatbot.toJSON();
       JSONObject keysObject = new JSONObject();
       JSONObject groupObject = new JSONObject();
       JSONObject languageObject = new JSONObject();
       List botList = new ArrayList();
       JSONObject result = new JSONObject();

       Iterator Key = botDetailsObject.keys();
       List keysList = new ArrayList();

       while (Key.hasNext()) {
           String key = (String) Key.next();
           keysList.add(key);
       }

       for (String key_name : keysList) {
           keysObject = botDetailsObject.getJSONObject(key_name);
           Iterator groupNames = keysObject.keys();
           List groupnameKeysList = new ArrayList();

           while (groupNames.hasNext()) {
               String key = (String) groupNames.next();
               groupnameKeysList.add(key);
           }

           for (String group_name : groupnameKeysList) {
               groupObject = keysObject.getJSONObject(group_name);
               Iterator languageNames = groupObject.keys();
               List languagenamesKeysList = new ArrayList();

               while (languageNames.hasNext()) {
                   String key = (String) languageNames.next();
                   languagenamesKeysList.add(key);
               }

               for (String language_name : languagenamesKeysList) {
                   languageObject = groupObject.getJSONObject(language_name);

If search parameter is passed, then search for a bot with the given name and add the bot to the botList if it exists. It will return all bots which have bot name as the searched name.

                   if (call.get("search", null) != null) {
                       String bot_name = call.get("search", null);
                       if(languageObject.has(bot_name)){
                           JSONObject botDetails = languageObject.getJSONObject(bot_name);
                           botDetails.put("name", bot_name);
                           botDetails.put("language", language_name);
                           botDetails.put("group", group_name);
                           botDetails.put("key", key_name);
                           botList.add(botDetails);
                       }
                   }

If search parameter is not passed, then it will return all the bots created by the users.

                    else {
                       Iterator botNames = languageObject.keys();
                       List botnamesKeysList = new ArrayList();

                       while (botNames.hasNext()) {
                           String key = (String) botNames.next();
                           botnamesKeysList.add(key);
                       }

                       for (String bot_name : botnamesKeysList) {
                           JSONObject botDetails = languageObject.getJSONObject(bot_name);
                           botDetails.put("name", bot_name);
                           botDetails.put("language", language_name);
                           botDetails.put("group", group_name);
                           botDetails.put("key", key_name);
                           botList.add(botDetails);
                       }
                   }
               }
           }
       }

List of all bots, botList is return as server response.

ListPrivateDraftSkillService

For creating a list we need to iterate over each user and check whether the user has a draft bot. We get all the authorized clients from DAO.getAuthorizedClients(). We then iterate over each client and get their identity and authorization. We get the drafts of the client from DAO.readDrafts(userAuthorization.getIdentity()). We then iterate over each draft and add it to the drafts object. Each draft object consists of date created,date modified, object which contains draft bot information such as name,language,etc provided by the user while saving the draft, email Id and uuid of the user.

       JSONObject result = new JSONObject();
       List draftBotList = new ArrayList();
       Collection authorized = DAO.getAuthorizedClients();

       for (Client client : authorized) {
         String email = client.toString().substring(6);
         JSONObject json = client.toJSON();
         ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, client.getName());
         Authorization userAuthorization = DAO.getAuthorization(identity);
         Map map = DAO.readDrafts(userAuthorization.getIdentity());
         JSONObject drafts = new JSONObject();

         for (Map.Entry entry: map.entrySet()) {
           JSONObject val = new JSONObject();
           val.put("object", entry.getValue().getObject());
           val.put("created", DateParser.iso8601Format.format(entry.getValue().getCreated()));
           val.put("modified", DateParser.iso8601Format.format(entry.getValue().getModified()));
           drafts.put(entry.getKey(), val);
         }
         Iterator keys = drafts.keySet().iterator();
         while(keys.hasNext()) {
           String key = (String)keys.next();
           if (drafts.get(key) instanceof JSONObject) {
             JSONObject draft = new JSONObject(drafts.get(key).toString());
             draft.put("id", key);
             draft.put("email", email);
             draftBotList.add(draft);
           }
         }
       }
       result.put("draftBots", draftBotList);

List of all drafts, draftBotList is returned as server response.

In conclusion, the admins can now see the bots and drafts created by the user and monitor where they are being used.

Resources

Continue ReadingDisplaying Private Skills and Drafts on SUSI.AI

Creating a Media Daemon for SUSI Smart Speaker

A daemon in reference of operating systems is a computer program that runs as a background process rather than under direct control of the user. Various daemons are being used in SUSI smart speaker.

The following features have been created

  • Update Daemon
  • Media Discovery Daemon
  • Factory Reset Daemon

In this blog, we’ll be discussing the implementation of the Media Discovery Daemon

Media Discovery Daemon:

The SUSI Smart speaker will have an essential feature which will allow the Users to play music from their USB devices. Hence , a media daemon will be running which will detect a USB connection and then scan it’s contents checking for all the mp3 files and then create custom SUSI skills to allow SUSI Smart Speaker to play music from your USB device.

 

The Media Daemon was implemented in the following steps

1.UDEV Rules

We had to figure out a way to run our daemon as soon as the user inserted the USB storage and stop the daemon as soon as the USB storage was removed

 

So, we used UDEV rules to trigger the Media Daemon.

 

ACTION==“add”, KERNEL==“sd?”, SUBSYSTEM==“block”, ENV{ID_BUS}==“usb”, RUN=“/home/pi/SUSI.AI/susi_linux/media_daemon/autostart.sh”ACTION==“remove, KERNEL==“sd?”, SUBSYSTEM==“block”, ENV{ID_BUS}==“usb”, RUN=“/home/pi/SUSI.AI/susi_linux/media_daemon/autostop.sh”

The Udev rules trigger a script called ‘autostart.sh’  on USB detection and a script called ‘autostop.sh’ on USB removal.

2. Custom Skill Creation

As the USB connection is now detected ,a script is triggered which checks the presence of a  local SUSI server in the repo. If a local server instance is detected,a python script is triggered which parses through the USB mount point and checks for the list of mp3 files present in the storage device and then create a custom skill file in the local server instance.

 

media_daemon_folder = os.path.dirname(os.path.abspath(__file__))
base_folder = os.path.dirname(media_daemon_folder)
server_skill_folder = os.path.join(base_folder, ‘susi_server/susi_server/data/generic_skills/media_discovery’)
server_settings_folder = os.path.join(base_folder, ‘susi_server/susi_server/data/settings’)

def make_skill(): # pylint-enable
   name_of_usb = get_mount_points()
   print(type(name_of_usb))
   print(name_of_usb[0])
   x = name_of_usb[0]
   os.chdir(‘{}’.format(x[1]))
   USB = name_of_usb[0]
   mp3_files = glob(“*.mp3”)
   f = open( media_daemon_folder +‘/custom_skill.txt’,‘w’)
   music_path = list()
   for mp in mp3_files:
       music_path.append(“{}”.format(USB[1]) + “/{}”.format(mp))

   song_list = ” “.join(music_path)
   skills = [‘play audio’,‘!console:Playing audio from your usb device’,‘{“actions”:[‘,‘{“type”:”audio_play”, “identifier_type”:”url”, “identifier”:”file://’+str(song_list) +‘”}’,‘]}’,‘eol’]
   for skill in skills:
       f.write(skill + ‘\n’)
   f.close()
   shutil.move( media_daemon_folder + ‘custom_skill.txt’, server_skill_folder)
   f2 = open(server_settings_folder + ‘customized_config.properties’,‘a’)
   f2.write(‘local.mode = true’)
   f2.close()

def get_usb_devices():
   sdb_devices = map(os.path.realpath, glob(‘/sys/block/sd*’))
   usb_devices = (dev for dev in sdb_devices
       if ‘usb’ in dev.split(‘/’)[5])
   return dict((os.path.basename(dev), dev) for dev in usb_devices)

def get_mount_points(devices=None):
   devices = devices or get_usb_devices() # if devices are None: get_usb_devices
   output = check_output([‘mount’]).splitlines() #nosec #pylint-disable type: ignore
   output = [tmp.decode(‘UTF-8’) for tmp in output ] # pytlint-enable
   def is_usb(path):
       return any(dev in path for dev in devices)
   usb_info = (line for line in output if is_usb(line.split()[0]))
   return [(info.split()[0], info.split()[2]) for info in usb_info] 

 

Now a custom skill file will be created in the local server instance by the name of `custom_skill.txt` and the user can play audio from USB by speaking the command ‘play audio’

 

3. Preparing for the Next USB insertion

Now if the User wants to update his/her music library or wants to use another USB storage device. The USB will be removed and hence the custom skill file is also deleted from the script ‘autstop.sh’ which is triggered via the UDEV rules

#! /bin/bash

SCRIPT_PATH=$(realpath $0)
DIR_PATH=$(dirname $SCRIPT_PATH)

cd $DIR_PATH/../susi_server/susi_server/data/generic_skills/media_discovery/

sudo rm custom_skill.txt  

 

This is how the Media Discovery Daemon works in SUSI Smart Speaker

 

References

Tags

gsoc, gsoc’18 , fossasia, susi.ai, smart speaker, media daemon, susi skills

Continue ReadingCreating a Media Daemon for SUSI Smart Speaker

Displaying Skills Feedback on SUSI.AI Android App

SUSI.AI has a feedback system where the user can post feedback for a skill using Android, iOS, and web clients. In skill details screen, the feedback posted by different users is displayed. This blog shows how the feedback from different users can be displayed in the skill details screen under feedback section.

Three of the items from the feedback list are displayed in the skill details screen. To see the entire list of feedback, the user can tap the ‘See All Reviews’ option at the bottom of the list.

The API endpoint that has been used to get skill feedback from the server is https://api.susi.ai/cms/getSkillFeedback.json

The following query params are attached to the above URL to get the specific feedback list :

  • Model
  • Group
  • Language
  • Skill Name

The list received is an array of `Feedback` objects, which hold three values :

  • Feedback String (feedback) – Feedback string posted by a user
  • Email (email) – Email address of the user who posted the feedback
  • Time Stamp – Time of posting feedback

To display feedback, use the RecyclerView. There can be three possible cases:

  • Case – 1: Size of the feedback list is greater than three
    In this case, set the size of the list to three explicitly in the FeedbackAdapter so that only three view holders are inflated. Inflate the fourth view holder with “See All Reviews” text view and make it clickable if the size of the received feedback list is greater than three.
    Also, when the user taps “See All Reviews”, launch an explicit intent to open the Feedback Activity. Set the AllReviewsAdapter for this activity. The size of the list will not be altered here because this activity must show all feedback.
  • Case – 2: Size of the feedback list is less than or equal to three
    In this case simply display the feedback list in the SkillDetailsFragment and there is no need to launch any intent here. Also, “See All Reviews” will not be displayed here.

    Case – 3: Size of the feedback list is zero
    In this case simply display a message that says no feedback has been submitted yet.Here is an example of how a “See All Reviews” screen looks like :

Implementation

First of all, define an XML layout for a feedback item and then create a data class for storing the query params.

data class FetchFeedbackQuery(
       val model: String,
       val group: String,
       val language: String,
       val skill: String
)


Now, make the GET request using Retrofit from the model (M in MVP).

override fun fetchFeedback(query: FetchFeedbackQuery, listener: ISkillDetailsModel.OnFetchFeedbackFinishedListener) {

   fetchFeedbackResponseCall = ClientBuilder.fetchFeedbackCall(query)

   fetchFeedbackResponseCall.enqueue(object : Callback<GetSkillFeedbackResponse> {
       override fun onResponse(call: Call<GetSkillFeedbackResponse>, response: Response<GetSkillFeedbackResponse>) {
           listener.onFetchFeedbackModelSuccess(response)
       }

       override fun onFailure(call: Call<GetSkillFeedbackResponse>, t: Throwable) {
           Timber.e(t)
           listener.onFetchFeedbackError(t)
       }
   })
}

override fun cancelFetchFeedback() {
   fetchFeedbackResponseCall.cancel()
}


The feedback list received in the JSON response can now be used to display the user reviews with the help of custom adapters, keeping in mind the three cases already discussed above.

Resources

Continue ReadingDisplaying Skills Feedback on SUSI.AI Android App

Showing skills based on different metrics in SUSI Android App using Nested RecyclerViews

SUSI.AI Android app had an existing skills listing page, which displayed skills under different categories. As a result, there were a number of API calls at almost the same time, which led to slowing down of the app. Thus, the UI of the Skill Listing page has been changed so as to reduce the number of API calls and also to make this page more useful to the user.

API Information

For getting a list of SUSI skills based on various metrics, the endpoint used is /cms/getSkillMetricsData.json

This will give you top ten skills for each metric. Some of the metrics include skill ratings, feedback count, etc. Sample response for top skills based on rating :

"rating": [
  {
    "model": "general",
    "group": "Knowledge",
    "language": "en",
    "developer_privacy_policy": null,
    "descriptions": "A skill to tell atomic mass and elements of periodic table",
    "image": "images/atomic.png",
    "author": "Chetan Kaushik",
    "author_url": "https://github.com/dynamitechetan",
    "author_email": null,
    "skill_name": "Atomic",
    "protected": false,
    "reviewed": false,
    "editable": true,
    "staffPick": false,
    "terms_of_use": null,
    "dynamic_content": true,
    "examples": ["search for atomic mass of radium"],
    "skill_rating": {
      "bookmark_count": 0,
      "stars": {
        "one_star": 0,
        "four_star": 3,
        "five_star": 8,
        "total_star": 11,
        "three_star": 0,
        "avg_star": 4.73,
        "two_star": 0
      },
      "feedback_count": 3
    },
    "usage_count": 0,
    "skill_tag": "atomic",
    "supported_languages": [{
      "name": "atomic",
      "language": "en"
    }],
    "creationTime": "2018-07-25T15:12:25Z",
    "lastAccessTime": "2018-07-30T18:50:41Z",
    "lastModifiedTime": "2018-07-25T15:12:25Z"
  },
  .
  .

]

 

Note : The above response shows only one of the ten objects. There will be ten such skill metadata objects inside the “rating” array. It contains all the details about skills.

Implementation in SUSI.AI Android App

Skill Listing UI of SUSI SKill CMS

Skill Listing UI of SUSI Android App

The UI of skills listing in SUSI Android app displays skills for each metric in a horizontal recyclerview, nested in a vertical recyclerview. Thus, for implementing horizontal recyclerview inside vertical recyclerview, you need two viewholders and two adapters (one each for a recyclerview). Let us go through the implementation.

  • Make a query object consisting of the model and language query parameters that shall be passed in the request to the server.

val queryObject = SkillMetricsDataQuery("general", 
PrefManager.getString(Constant.LANGUAGE,Constant.DEFAULT))

 

  • Fetch the skills based on metrics, by calling fetch in SkillListModel which then makes an API call to fetch groups.

skillListingModel.fetchSkillsMetrics(queryObject, this)

 

  • When the API call is successful, the below mentioned method is called which in turn parses the received response and updates the adapter to display the skills based on different metrics.

override fun onSkillMetricsFetchSuccess(response: Response<ListSkillMetricsResponse>) {
   skillListingView?.visibilityProgressBar(false)
   if (response.isSuccessful && response.body() != null) {
       Timber.d("METRICS FETCHED")
       metricsData = response.body().metrics
       if (metricsData != null) {
           metrics.metricsList.clear()
           metrics.metricsGroupTitles.clear()
           if (metricsData?.rating != null) {
               if (metricsData?.rating?.size as Int > 0) {
                   metrics.metricsGroupTitles.add(utilModel.getString(R.string.metric_rating))
                   metrics.metricsList.add(metricsData?.rating)
                   skillListingView?.updateAdapter(metrics)
               }
           }

           if (metricsData?.usage != null) {
               if (metricsData?.usage?.size as Int > 0) {
                   metrics.metricsGroupTitles.add(utilModel.getString(R.string.metric_usage))
                   metrics.metricsList.add(metricsData?.usage)
                   skillListingView?.updateAdapter(metrics)
               }
           }

           if (metricsData?.newest != null) {
               val size = metricsData?.newest?.size
               if (size is Int) {
                   if (size > 0) {
                       metrics.metricsGroupTitles.add(utilModel.getString(R.string.metric_newest))
                       metrics.metricsList.add(metricsData?.newest)
                       skillListingView?.updateAdapter(metrics)
                   }
               }
           }

           if (metricsData?.latest != null) {
               if (metricsData?.latest?.size as Int > 0) {
                   metrics.metricsGroupTitles.add(utilModel.getString(R.string.metric_latest))
                   metrics.metricsList.add(metricsData?.latest)
                   skillListingView?.updateAdapter(metrics)
               }
           }

           if (metricsData?.feedback != null) {
               if (metricsData?.feedback?.size as Int > 0) {
                   metrics.metricsGroupTitles.add(utilModel.getString(R.string.metric_feedback))
                   metrics.metricsList.add(metricsData?.feedback)
                   skillListingView?.updateAdapter(metrics)
               }
           }

           if (metricsData?.topGames != null) {
               val size = metricsData?.feedback?.size
               if (size is Int) {
                   if (size > 0) {
                       metrics.metricsGroupTitles.add(utilModel.getString(R.string.metrics_top_games))
                       metrics.metricsList.add(metricsData?.topGames)
                       skillListingView?.updateAdapter(metrics)
                   }
               }
           }

           skillListingModel.fetchGroups(this)
       }
   } else {
       Timber.d("METRICS NOT FETCHED")
       skillListingView?.visibilityProgressBar(false)
       skillListingView?.displayError()
   }
}

 

  • When skills are fetched, the data in adapter is updated using skillMetricsAdapter.notifyDataSetChanged()

override fun updateAdapter(metrics: SkillsBasedOnMetrics) {
   swipe_refresh_layout.isRefreshing = false
   if (errorSkillFetch.visibility == View.VISIBLE) {
       errorSkillFetch.visibility = View.GONE
   }
   skillMetrics.visibility = View.VISIBLE
   this.metrics.metricsList.clear()
   this.metrics.metricsGroupTitles.clear()
      this.metrics.metricsList.addAll(metrics.metricsList)
   this.metrics.metricsGroupTitles.addAll(metrics.metricsGroupTitles)
      skillMetricsAdapter.notifyDataSetChanged()
}

 

  • The data is set to the layout in two adapters made earlier. The following is the code to set the title for the metric and adapter to horizontal recyclerview. This is the SkillMetricsAdapter to set data to show item in vertical recyclerview.

override fun onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) {
   if (metrics != null) {
       if (metrics.metricsList[position] != null) {
           holder.groupName?.text = metrics.metricsGroupTitles[position]
       }

       skillAdapterSnapHelper = StartSnapHelper()
       holder.skillList?.setHasFixedSize(true)
       val mLayoutManager = LinearLayoutManager(context, LinearLayoutManager.HORIZONTAL, false)
       holder.skillList?.layoutManager = mLayoutManager
       holder.skillList?.adapter = SkillListAdapter(context, metrics.metricsList[position], skillCallback)
       holder.skillList?.onFlingListener = null
       skillAdapterSnapHelper.attachToRecyclerView(holder.skillList)
   }
}

 

Continue ReadingShowing skills based on different metrics in SUSI Android App using Nested RecyclerViews

Fetching Bots from SUSI.AI Server

Public skills of SUSI.AI are stored in susi_skill_cms repository. A private skill is different from a public skill. It can be viewed, edited and deleted only by the user of who created it. The purpose of private skill is that this acts as a chatbot. All the information of the bot like design, configuration etc is stored within the private skill itself. In order to show the lists of chatbots a user has created to him/her, we need to fetch these lists from the server. The API for fetching private skills is in ListSkillService.java servlet. This servlet is used for both the purposes for fetching public skills and private skills.

How are bots fetched?

All the private skills are stored in data/chatbot/chatbot.json location in the SUSI server. Hence, we can fetch these skills from here.
The API call to fetch private skills is:

https://api.susi.ai/cms/getSkillList.json?private=1&access_token=accessTokenHere

The API call has two parameters:

  1. private = 1: This parameter tells the servlet that we’re asking for private skills because same API is used for fetching public skills as well.
  2. access_token : We have to pass the access-token in order to authenticate the user and access the chatbots of the specific user because the chatbots are stored according to user id of the users in chatbot.json. To get the user id, we need access-token.

For fetching the chatbots from chatbot.json, we firstly authenticate the user. Then we check if the user has any chatbots or not. If the user has chatbots then we loop through chatbot.json to find the chatbots of user by using user id. After getting chatbots details, we simply present it in json structure. You can easily understand this through the following code snippets. For authenticating:

String userId = null;
String privateSkill = call.get("private", null);
if (call.get("access_token", null) != null) {
    ClientCredential credential = new ClientCredential(ClientCredential.Type.access_token, call.get("access_token", null));
    Authentication authentication = DAO.getAuthentication(credential);
    // check if access_token is valid
    if (authentication.getIdentity() != null) {
        ClientIdentity identity = authentication.getIdentity();
        userId = identity.getUuid();
    }
}

You can see that the parameter private is stored in privateSkill. We check if privateSkill is not null followed by checking if the user id provided is valid or not. When both the checks are successful, we create a JSON object in which we store all the bots. To do this, we loop through the chatbot.json file to find the user id provided in API call. If the user id doesn’t exist in chatbot.json file then a message “User has no chatbots.” is displayed. If the user id is found then we again loop through all the chatbots and put their name as the value of key “name”,  language as the value of key “language” and group as the value of key “group”. All these key value pairs for the chatbots are pushed in an array and finally that array is added to the JSON object. This JSON is then received as response of the API call. The following code will demonstrate this:

if(privateSkill != null) {
    if(userId != null) {
        JsonTray chatbot = DAO.chatbot;
        JSONObject result = new JSONObject();
        JSONObject userObject = new JSONObject();
        JSONArray botDetailsArray = new JSONArray();
        JSONArray chatbotArray = new JSONArray();
        for(String user_id : chatbot.keys())
        {
            if(user_id.equals(userId)) {
                userObject = chatbot.getJSONObject(user_id);
                Iterator chatbotDetails = userObject.keys();
                List<String> chatbotDetailsKeysList = new ArrayList<String>();
                while(chatbotDetails.hasNext()) {
                    String key = (String) chatbotDetails.next();
                    chatbotDetailsKeysList.add(key);
                }
                for(String chatbot_name : chatbotDetailsKeysList)
                {
                    chatbotArray = userObject.getJSONArray(chatbot_name);
                    for(int i=0; i<chatbotArray.length(); i++) {
                        String name = chatbotArray.getJSONObject(i).get("name").toString();
                        String language = chatbotArray.getJSONObject(i).get("language").toString();
                        String group = chatbotArray.getJSONObject(i).get("group").toString();
                        JSONObject botDetails = new JSONObject();
                        botDetails.put("name", name);
                        botDetails.put("language", language);
                        botDetails.put("group", group);
                        botDetailsArray.put(botDetails);
                        result.put("chatbots", botDetailsArray);
                    }
                }
            }
        }
        if(result.length()==0) {
            result.put("accepted", false);
            result.put("message", "User has no chatbots.");
            return new ServiceResponse(result);
        }
        result.put("accepted", true);
        result.put("message", "All chatbots of user fetched.");
        return new ServiceResponse(result);
    }
}

So, if chatbot.json contains:

{"c9b58e182ce6466e413d5acafae906ad": {"chatbots": [
 {
   "name": "testing111",
   "language": "en",
   "group": "Knowledge"
 },
 {
   "name": "DNS_Bot",
   "language": "en",
   "group": "Knowledge"
 }
]}}

Then, the JSON received at http://api.susi.ai/cms/getSkillList.json?private=1&access_token=JwTlO8gdCJUns569hzC3ujdhzbiF6I is:

{
  "session": {"identity": {
    "type": "email",
    "name": "test@whatever.com",
    "anonymous": false
  }},
  "chatbots": [
    {
      "name": "testing111",
      "language": "en",
      "group": "Knowledge"
    },
    {
      "name": "DNS_Bot",
      "language": "en",
      "group": "Knowledge"
    }
  ],
  "accepted": true,
  "message": "All chatbots of user fetched."
}

References:

Continue ReadingFetching Bots from SUSI.AI Server

Fetching responses from SUSI.AI Server for Botbuilder Build Views

In SUSI.AI, we use skill editor for creating and editing public/private skills. The editor we use is Ace editor. The skill is written in a code format documented here. This works fine for a developer but for a user with little experience in coding, this can be confusing. Hence, for providing more clarity as to what the skill does, I built conversation view and tree view along with code view for the skill editor.

Conversation view shows the skill in form of actual conversation between the user and the bot while tree views shows the same conversation in form of a tree. Earlier these views were implemented by converting the code view into an object containing user queries and SUSI responses.

While this works for simple skills, it obviously won’t work for a complex skill in which the responses are fetched from an API. Hence we needed live responses from SUSI server.

This is done similar to how preview works. We pass the whole skill in an instant parameter in the chat.json API along with the user query. This gives us the response from SUSI in form of a JSON.

API Call:

We send a GET request to the following URL:

https://api.susi.ai/susi/chat.json?q=userQuery&instant=wholeSkill

This contains two parameters:

  • q: The user query is passed in this parameter.
  • instant: The whole skill code (present in the code view) is passed in this parameter.

The response is a JSON providing response from SUSI.

Getting user queries:

We can not rely on user to provide the user queries in conversation view and tree view because the user has already provided it in the code view. Hence, we fetch the user queries from code view. This is simply done by dissecting the code and putting all the lines which don’t start with ::, !, #, {, } and “ in an array. Then we split the entries of this array wherever a vertical bar (|) is found. This provides us an array containing all the user queries. It’ll be clear from the following function:

fetchUserInputs = () => {
  let code = this.state.code;
  let userInputs = [];
  let userQueries = [];
  var lines = code.split('\n');
  for (let i = 0; i < lines.length; i++) {
    let line = lines[i];
    if (
      line &&
      !line.startsWith('::') &&
      !line.startsWith('!') &&
      !line.startsWith('#') &&
      !line.startsWith('{') &&
      !line.startsWith('}') &&
      !line.startsWith('"')
    ) {
      let user_query = line;
      while (true) {
        i++;
        if (i >= lines.length) {
          break;
        }
        line = lines[i];
        if (
          line &&
          !line.startsWith('::') &&
          !line.startsWith('!') &&
          !line.startsWith('#')
        ) {
          break;
        }
      }
      userQueries.push(user_query);
    }
  }
  for (let i = 0; i < userQueries.length; i++) {
    let queries = userQueries[i];
    let queryArray = queries.trim().split('|');
    for (let j = 0; j < queryArray.length; j++) {
      userInputs.push(queryArray[j]);
    }
  }
  this.setState({ userInputs }, () => this.getResponses(0));
};

Getting response to a single query at a time:

Now, we have an array containing all the user queries but we can not simply run a loop through the array and then get responses for each query because the AJAX call that we’re making to fetch response is asynchronous. Hence, this will result in multiple AJAX calls in a very short period of time. This will cause a failure in fetching responses and conversation view won’t work. We definitely don’t want that.

To solve this problem, we get response for a single query at a time and make the next AJAX call only when the response for the current call is received. You can see in the code snippet provided in last section, after updating state of userInputs, we’re calling getResponses as a callback and passing 0. This 0 is the index of array which will be incremented on every successful AJAX call. The following code snippet will demonstrate this:

$.ajax({
  type: 'GET',
  url: url,
  contentType: 'application/json',
  dataType: 'json',
  success: function(data) {
  let answer;
  if (data.answers[0]) {
    // Putting response in an object along with user query
    if (responseNumber + 1 === userInputs.length) { // Stopping when responses are fetched for all user queries.
      this.setState({ loaded: true });
    }
    this.setState({ responseData }, () => // updating the response data
      this.getResponses(++responseNumber),  // Incrementing the index and calling getResnponses again as a callback when response data state is updated.
    );
  }.bind(this),
  error: function(err) {
    console.log(err);
  }.bind(this),
});

The code snippets I provided are used in conversation view. Same algorithm is used in tree view as well.

References:

Continue ReadingFetching responses from SUSI.AI Server for Botbuilder Build Views

Skills tab of SUSI AI Admin Panel

The Skills tab in SUSI.AI Admin Panel displays all the skills of SUSI in a tabular form. The table is created using the Table component of Ant Design. It’s preferred because we’re going to handle a lot of data here and that can turn out to be heavy if we use Google’s Material-ui.

Displaying the data:

The data is rendered in the form of a table which has seven columns – Name, Group, Language, Type, Author, Status and Action. The first 5 columns displays basic details of a skill. The “Status” column shows whether the skill has been reviewed by admin or not. All the reviewed skills are either “Approved” or “Not Approved”. The final column “Action” contains all the actions that admin can perform on the skill. Currently, an admin can change the review status of a skill. More actions will be added in future as this is still in beta.

All the column names are stored in a variable in the form of an array. The following code will demonstrate the way of making three columns – Name, Group, Language.

this.columns = [
  {
    title: 'Name',
    dataIndex: 'skill_name',
    sorter: false,
    width: '20%',
  },
  {
    title: 'Group',
    dataIndex: 'group',
    width: '15%',
  },
  {
    title: 'Language',
    dataIndex: 'language',
    width: '10%',
  },
];

All the skills are also stored in an array which is a state variable. These columns and skills are then passed to the Table component as props. The following code will demonstrate that:

<Table
  columns={this.columns}
  rowKey={record => record.registered}
  dataSource={this.state.skillsData}
/>

Fetching all the skills from SUSI Server:

All the public skills can be fetched from ListSkillService API of SUSI Server. We can filter all the skills using various filters. We want to display the skills alphabetically on Skills tab in Admin Panel. Hence, we filter the skills accordingly. To do this, we pass three parameters in the GET request. They are as follows:

  • applyFilter: true
  • filter_name: ascending
  • filter_type: lexicographical

This returns all the skills in an alphabetical order. The API request url looks like this:

https://api.susi.ai/cms/getSkillList.json?applyFilter=true&filter_name=ascending&filter_type=lexicographical

After fetching the skills, we put all the parameters of these skills required for our table in an object which is then pushed into an array. One object is created for each skill.
If the API call fails for some reason then a Google’s Material-ui Snackbar appears with a message that an error occurred.

Changing review status of a skill:

An admin can change the review status of any skill. This is done by making a GET request to ChangeSkillStatusService API. The request contains five parameters. They are as follows:

  1. Model: Model of the skill is passed here (string)
  2. Group: Group of the skill is passed here (string)
  3. Language: Language of the skill is passed here (string)
  4. Reviewed: true is passed is skill has been approved and false if skill is not approved. (boolean)
  5. Access_token: The access token of user is passed here. This is for verifying the user’s BASE ROLE. This is taken from cookies using cookies.get(‘loggedIn’). (string)

The call url looks like this:

https://api.susi.ai/cms/changeSkillStatus.json?model=general&group=Knowledge&language=en&skill=aboutsusi&reviewed=true&access_token=yourAccessTokenHere

If the API call is successful, then the review status of a skill is successfully changed. Otherwise, an error message is thrown.

References:

Continue ReadingSkills tab of SUSI AI Admin Panel

Displaying Skills Feedback on SUSI iOS

SUSI allows the user to rate the SUSI Skills with the five-star rating system. SUSI offer a good feedback system where the user can post feedback to any skill by using iOS, Android, and Web clients. In Skill Detail, there is a skill feedback text field where the user can write feedback about SUSI Skill. We display the users posted feedbacks on Skill Detail screen. In this post, we will see how the displaying skills feedback feature implemented on SUSI iOS.

Implementation –

We are displaying three feedback on Skill Detail screen, to see all feedback, there is a “See All Review” option, by clicking user is directed to a new screen where he/she can see all feedback related to particular skill.

We use the endpoint below for getting skill feedback from server side –

https://api.susi.ai/cms/getSkillFeedback.json

With the following params:

  • Model
  • Group
  • Language
  • Skill Name

The API endpoint above return the all the feedback array related to particular susi skill. We store feedbacks in an array of Feedback object, which holds three value:

    • Feedback String – Feedback string posted by the user
    • Email – Email address of feedback poster user
    • Time Stamp – Time of posting feedback
class Feedback: NSObject {
var feedbackString: String = ""
var email: String = ""
var timeStamp: String = ""
...
}

To display feedbacks, we are using UITableView with two prototype cells, one for feedbacks and one for “See All Review” option.

There can be different cases eg. when the total number of feedback for skill is less than three or three. When the feedback count is three or less than three, there is no need to show “See All Review” option. Also, tableView height is different for different feedback count. For varying tableView height, we have created an outlet for tableView height constraints and vary accordingly.

@IBOutlet weak var feedbackTableHeighConstraint: NSLayoutConstraint!

Now, let’s see how the number of cells, height for cells and different cells presented according to feedback count with UITableViewDelegate and UITableViewDataSource methods.

Handling number of tableView rows –

func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
guard let feedbacks = feedbacks, feedbacks.count > 0 else {
feedbackTableHeighConstraint.constant = 0.0
return 0
}
if feedbacks.count < 4 {
feedbackTableHeighConstraint.constant = CGFloat(72 * feedbacks.count)
return feedbacks.count
} else {
feedbackTableHeighConstraint.constant = 260.0
return 4
}
}

Where feedbacks is the array of Feedback object which holds the feedbacks we are getting from the server side for a skill.

var feedbacks: [Feedback]?

In the above method, we see that how we are handling the number of cells case. Now let’s see how to handle which cells to be present on basis of the number of cells case –

func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
guard let feedbacks = feedbacks, feedbacks.count > 0 else {
feedbackTableHeighConstraint.constant = 0.0
return UITableViewCell()
}
if feedbacks.count < 4 {
if let cell = tableView.dequeueReusableCell(withIdentifier: "feedbackDisplayCell", for: indexPath) as? FeedbackDisplayCell {
cell.feedback = feedbacks[indexPath.row - Int(indexPath.row/2)]
return cell
}
} else if feedbacks.count > 3 {
if indexPath.row == 3 {
let cell = tableView.dequeueReusableCell(withIdentifier: "allFeedbackCell", for: indexPath)
return cell
} else {
if let cell = tableView.dequeueReusableCell(withIdentifier: "feedbackDisplayCell", for: indexPath) as? FeedbackDisplayCell {
cell.feedback = feedbacks[indexPath.row]
return cell
}
}
}
return UITableViewCell()
}

If the number of feedbacks is greater than three than we provide “See All Review” option to the user to see all the feedback related to skill. We are displaying all feedbacks using UITableViewController. When the user clicks the “See All Review” option, we pass the feedbacks (Array of all the feedback) to new UITableViewController. By passing feedbacks, we are reducing one network call.

let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let allFeedbackVC = storyboard.instantiateViewController(withIdentifier: "AllFeedbackController") as? AllFeedbackViewController {
allFeedbackVC.feedbacks = self.feedbacks
let nvc = AppNavigationController(rootViewController: allFeedbackVC)
self.present(nvc, animated: true, completion: nil)
}

On all skill feedback screen, we are displaying the full review. For the different size of text, we are setting the different size of cell size by using the method below:

if let feedbacks = feedbacks {
let estimatedLabelHeight = UILabel().heightForLabel(text: feedbacks[indexPath.row].feedbackString, font: UIFont.systemFont(ofSize: 14.0), width: 250.0)
return 64 + estimatedLabelHeight
} else {
return 80
}

 

Final Output –

Resources –

  1. SUSI API Link: https://api.susi.ai/
  2. SUSI iOS Link: https://github.com/fossasia/susi_iOS
  3. Apple’s Documentation on UITableViewDelegate: https://developer.apple.com/documentation/uikit/uitableviewdelegate?changes=_6
  4. Apple’s Documentation on UITableViewDataSource: https://developer.apple.com/documentation/uikit/uitableviewdatasource
Continue ReadingDisplaying Skills Feedback on SUSI iOS

Using a Git repo as a Storage & Managing skills through susi_skill_cms

In this post, I’ll be talking about SUSI’s skill management and the workflow of creating new skills

The SUSI skills are maintained in a separate github repository susi_skill_data which provides the features of version controlling and the ability to rollback to a previous version implemented in SUSI Server.

The workflow is as explained in the featured image of this blog, SUSI CMS provides the user with a GUI through which user can talk to the SUSI Server and using it’s api calls, it can manipulate the susi skills present/stored on the susi_skill_data repository.

When the user opts to create a new skill, a new createSkill component is loaded with an editor to define rules of the skill. Once the form is submitted, an AJAX POST request is made to the server which actually commits the skill data to the repository and thus it is visible in the CMS from that point on.

Grab the skill details within the editor and put them in a form which is to be sent via the POST request.

let form = new FormData();
form.append('model', 'general');
form.append('group', this.state.groupValue);
form.append('language', this.state.languageValue);
form.append('skill', this.state.expertValue.trim().replace(/\s/g,'_'));
form.append('image', this.state.file);
form.append('content', this.state.code);
form.append('image_name', this.state.imageUrl.replace('images/',''));
form.append('access_token', cookies.get('loggedIn'));


Configure POST request settings object

let settings = {
   'async': true,
   'crossDomain': true,
   'url': urls.API_URL + '/cms/createSkill.json',
   'method': 'POST',
   'processData': false,
   'contentType': false,
   'mimeType': 'multipart/form-data',
   'data': form
};


Make an AJAX request using the settings above to upload the skill to the server and send a notification when the request is successful.

$.ajax(settings)
   .done(function (response) {
   self.setState({
          loading:false
   });
notification.open({
    message: 'Accepted',
    description: 'Your Skill has been uploaded to the server',
    icon: <Icon type='check-circle' style={{ color: '#00C853' }}       />,
});


Parse the received response as JSON and if the accept key in the response is true, we push the new skill data to the history API and set relevant states.

let data = JSON.parse(response);
if(data.accepted===true){
  self.props.history.push({
	pathname: '/' + self.state.groupValue  +
  	'/' + self.state.expertValue.trim().replace(/\s/g,'_') +
  	'/' + self.state.languageValue,
	state: {
  	from_upload: true,
  	expertValue:  self.state.expertValue,
  	groupValue: self.state.groupValue ,
  	languageValue: self.state.languageValue,
}});


If the accepted key of the server response is not true, display a notification.

else{
	self.setState({
  		loading:false
	});
	notification.open({
	  	message: 'Error Processing your Request',
	  	description: String(data.message),
	  	icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
	});
}})


Handle cases when AJAX request fails and send a corresponding notification

.fail(function (jqXHR, textStatus) {
 ...
  notification.open({
    message: 'Error Processing your Request',
    description: String(textStatus),
    icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
  });
});


I hope after reading this post, the objectives of susi_skill_data are more clear and you understood how CMS handles the creation of skills.

Resources

1.AJAX Jquery – AJAX request using Jquery
2. React State – Read about React states and lifecycle hooks.

Continue ReadingUsing a Git repo as a Storage & Managing skills through susi_skill_cms