Individual skill usage subsections in SUSI Skill CMS

In SUSI.AI Skills CMS several interactive skill related statistics are displayed on the skill page for each skill which includes user ratings, ratings over time, user feedback and skill usage data displayed interactively. The skill usage section is further subdivided to get more insight into how the skill has been used and from where. Therefore we have three subsections which display Time wise skill usage, device wise usage, and country wise usage. All this data can help evaluate which devices are mostly using the skill or data like in which country the skill is more popular than others. So in this post, we mainly discuss the UI of how these sections are implemented.

Implementation

Adding a Card component to the skill page component at the bottom of the skill page component.

<SkillUsageCard
 skill_usage={this.state.skill_usage}
 device_usage_data={this.state.device_usage_data}
 countryWiseSkillUsage={this.state.countryWiseSkillUsage}
/>

 

In the render function of the newly made component, we import the Paper component from material-ui and render it at the top to contain the subsections to give it a card-like UI.

<div>
   <Paper className="margin-b-md margin-t-md">
           ...
   </Paper>
</div>

 

Create div for the time wise skill usage. Calculate total skill usage for displaying the total skill usage count and also it helps to decide whether we need to render the section or not. So if the total skill usage by time count is greater than zero then render the line chart for visual analysis and display the total skill usage count too.

let totalSkillUsage = 0;
if (this.props.skill_usage) {
 // eslint-disable-next-line
 totalSkillUsage = this.props.skill_usage.reduce((totalCount, day) => {
        if (day) {
         return totalCount + day.count;
        }
        return totalCount;
 }, 0);
}

<div className="time-chart">
 <div>
        <ResponsiveContainer width={this.state.width} height={300}>
         <LineChart
           ...
         >
           <XAxis dataKey="date" padding={{ right: 20 }} />
           <YAxis allowDecimals={false} />
           <Tooltip wrapperStyle={{ height: '60px' }} />
           <Legend />
           <Line
             ...
           />
         </LineChart>
        </ResponsiveContainer>
 </div>
</div>
<div className="total-hits">
 <div className="large-text">{totalSkillUsage}</div>
 Hits this week
</div>

 

Create div for the Device wise usage. Conditionally render it in case the device wise data is available in the props.

<div className="device-usage">
 <div className="sub-title">Device wise Usage</div>
 {this.props.device_usage_data &&
 this.props.device_usage_data.length ? (
        <div className="pie-chart">
         <ResponsiveContainer width={600} height={350}>
           <PieChart>
             <Pie
               ...
             >
               {this.props.device_usage_data.map((entry, index) => (
                 <Cell key={index} fill={entry.color} />
               ))}
             </Pie>
             <Legend wrapperStyle={{ position: 'relative' }} />
           </PieChart>
         </ResponsiveContainer>
        </div>
</div>

 

Create a div for the country wise usage. We get the country wise usage data from the props and then we plug in the data in the geo chart component and also display the data as a table on the side. In case no data comes in or is unavailable we do not render the component at all.

<div>
 {countryWiseSkillUsage && countryWiseSkillUsage.length ? (
        <div className="country-usage-container">
         <div className="country-usage-graph">
           <GeoChart data={countryWiseSkillUsage} />
         </div>
         <div className="country-usage-list">
           <Table>
             ...
         </div>
        </div>
 ) : (
        <div className="unavailable-message">
         Country wise usage distribution is not available.
        </div>
 )}
</div>

 

This is how the three subsection in the skill usage component are implemented

Resources

Continue Reading

Added “table” type action support in SUSI android app

SUSI.AI has many actions supported by it, for eg: answer, anchor, map, piechart, websearch and rss.These actions are a few of those that can be supported in the SUSI.AI android app, but there are many actions implemented on the server side and the web client even has the implementation of how to handle the “table” type response.

The table response is generally a JSON array response with different json objects, where each json object have similar keys, and the actions key in the JSON response has the columns of the table response which are nothing but the keys in the data object of the response.

To implement the table type response in the susi android app a separate file needed to made to parse the table type response, since the keys and values both are required to the display the response. The file ParseTableSusiResponseHelper.kt was made which parsed the JSON object using the Gson converter factory to get the key value of the actions :

“actions”: [

       {

         “columns”: {

           “ingredients”: “Ingredients”,

           “href”: “Instructions Link”,

           “title”: “Recipe”

         },

         “count”: -1,

         “type”: “table”

       }

     ]

 

The inside the columns the keys and the values, both were extracted, values were to displayed in the title of the column and keys used were to extract the values from the “data” object of the response.

The files TableColumn.java, TableData.java are POJO classes that were used for storing the table columns and the data respectively. The TableDatas.java class was used to store the column list and the data list for the table response.

To fetch the table type response from the server a TableSusiResponse.kt file was added that contained serializable entities which were used to map the response values fetched from the server. A variable that contained the data stored in the “answers” key of the response was made of type of an ArrayList of TableAnswers.

@SerializedName(“answers”)
@Expose
val answers: List<TableAnswer> = ArrayList()

The TableAnswer.kt is another file added that contains serializable variables to store values inside the keys of the “answers” object. The actions object shown above is inside the answers object and it was stored in the form of an ArrayList of TableAction.

@SerializedName(“actions”)
@Expose
val actions: List<TableAction> = ArrayList()

Similar to TableAnswer.kt file TableAction.kt file also contains serializable variables that map the values stored in the “actions” object.

In the retrofit service interface SusiService.java a new call was added to fetch the data from the server as follows :

@GET(“/susi/chat.json”)
Call<TableSusiResponse> getTableSusiResponse(@Query(“timezoneOffset”) int timezoneOffset,
                                           @Query(“longitude”) double longitude,
                                           @Query(“latitude”) double latitude,
                                           @Query(“geosource”) String geosource,
                                           @Query(“language”) String language,
                                           @Query(“q”) String query);

Now, after the data was fetched, the table response can be parsed using the Gson converter factory in the ParseTableSusiResponseHelper.kt file. Below is the implementation :

fun parseSusiResponse(response: Response<TableSusiResponse>) {
  try {
      var response1 = Gson().toJson(response)
      var tableresponse = Gson().fromJson(response1, TableBody::class.java)
      for (tableanswer in tableresponse.body.answers) {
          for (answer in tableanswer.actions) {
              var map = answer.columns
              val set = map?.entries
              val iterator = set?.iterator()
              while (iterator?.hasNext().toString().toBoolean()) {
                  val entry = iterator?.next()
                  listColumn.add(entry?.key.toString())
                  listColVal.add(entry?.value.toString())
              }
          }
          val map2 = tableanswer.data
          val iterator2 = map2?.iterator()
          while (iterator2?.hasNext().toString().toBoolean()) {
              val entry2 = iterator2?.next()
              count++;
              for (count in 0..listColumn.size – 1) {
                  val obj = listColumn.get(count)
                  listTableData.add(entry2?.get(obj).toString())
              }
          }
          tableData = TableDatas(listColVal, listTableData)
      }
  } catch (e: Exception) {
      tableData = null
  }
}

 

Now the data is also parsed, we pass the two lists the ColumnList and DataList to the variable of TableDatas.

Three viewholder classes were added to display the table response properly in the app and corresponding to these viewholders a couple of adapters were also made that are responsible for setting the values in the recyclerview present in the views. The first viewholder is the TableViewHolder, it contains the horizontal recyclerview that is used to display the items fetched from the “data” object of the response. The recyclerview in the TableViewHolder has each entity of the type TabViewHolder, this is a simple cardview but also contains another recyclerview inside it which is used to store the keys and values of each of the object inside the “data” object.

TableViewHolder.java file has a setView() method that uses the the object of ChatMessage to get the list of columns and data to be set in the view.

 

Changes were made in the ChatPresenter.kt file to catch the tableresponse when a table type action is detected. Below is the implementation :

if (response.body().answers[0].actions[i].type.equals(“table”)) {
  tableResponse(query)
  return
}

The tableResponse function is as follows :

fun tableResponse(query: String) {
  val tz = TimeZone.getDefault()
  val now = Date()
  val timezoneOffset = -1 * (tz.getOffset(now.time) / 60000)
  val language = if (PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT).equals(Constant.DEFAULT)) Locale.getDefault().language else PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT)
  chatModel.getTableSusiMessage(timezoneOffset, longitude, latitude, source, language, query, this)
}

 

It calls the chatModel to get the list of columns and data to be set. The ChatFeedRecyclerAdapter.java files checks for the table response code, and if it matches then the view used for displaying SUSI’s message is the TableViewHolder. Here is how this viewholder is inflated :

case TABLE:
  view = inflater.inflate(R.layout.susi_table, viewGroup, false);
  return new TableViewHolder(view, clickListener);

Below is the final result when the table response is fetched for the query “Bayern munich team players” is :

References :

  1. SUSI server response for table query : https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=barcelona+team+players
  2. GSON for converting java objects to JSON and JSON to java : http://www.vogella.com/tutorials/JavaLibrary-Gson/article.html
Continue Reading

Implementing Table View in Devices Tab in Settings

We can connect to the SUSI.AI Smart Speaker using our mobile apps (Android or iOS). But there needs to be something implemented which can tell you what all devices are linked to your account. This is in consistency with the way how Google Home devices and Amazon Alexa devices have this feature implemented in their respective apps, which allow you to see the list of devices connected to your account. This blog post explains how this feature has been implemented on the SUSI.AI Web Client.

Fetching data of the connected devices from the server

The information of the devices connected to an account is stored in the Accounting object of that user. This is a part of a sample Accounting object of a user who has 2 devices linked to his/her account. This is the data that we wish to fetch. This data is accessible at the /aaa/listUserSettings.json endpoint.

{
  "devices": {
    "37-AE-5F-7B-CA-3F": {
      "name": "Device 1",
      "room": "Room 1",
      "geolocation": {
        "latitude": "50.34567",
        "longitude": "60.34567"
      }
    },
    "9D-39-02-01-EB-95": {
      "name": "Device 2",
      "room": "Room 2",
      "geolocation": {
        "latitude": "52.34567",
        "longitude": "62.34567"
      }
    }
  }
} 

 

In the Settings.react.js file, we make an AJAX call immediately after the component is mounted on the DOM. This AJAX call is made to the /aaa/listUserSettings.json endpoint. The received response of the AJAX call is then used and traversed to store the information of each connected device in a format that would be more suitable to use as a prop for the table.

apiCall = () => {
    $.ajax({
      url: BASE_URL + '/aaa/listUserSettings.json?' + 'access_token=' +
  cookies.get('loggedIn');,
      type: 'GET',
      dataType: 'jsonp',
      crossDomain: true,
      timeout: 3000,
      async: false,
      success: function(response) {
        let obj = [];
         // Extract information from the response and store them in obj object
        obj.push(myObj);
        this.setState({
          dataFetched: true,
          obj: obj,
        });
      }
      }.bind(this),
  };

 

This is how the extraction of keys takes place inside the apiCall() function. We first extract the keys of the ‘devices’ JSONObject inside the response. The keys of this JSONObject are the Mac Addresses of the individual devices. Then we traverse inside the JSONObject corresponding to each Mac Address and store the name of the device, room and also the geolocation information of the device in separate variables, and then finally push all this information inside an object, namely ‘myObj’. This JSONObject is then pushed to a JSONArray, namely ‘obj’. Then a setState() function is called which sets the value of ‘obj’ to the updated ‘obj’ variable.

let keys = Object.keys(response.devices);
keys.forEach(i => {
    let myObj = {
        macid: i,
        devicename: response.devices[i].name,
        room: response.devices[i].room,
        latitude: response.devices[i].geolocation.latitude,
        longitude: response.devices[i].geolocation.longitude,
    };
}

 

This way we fetch the information of devices and store them in a variable named ‘obj’. This variable will now serve as the data for the table which we want to create.

Creating table from this data

The data is then passed on to the Table component as a prop in the Settings.react.js file.

<TableComplex
    // Other props
    tableData={this.state.obj}
/>

 

The other props passed to the Table component are the functions for handling the editing and deleting of rows, and also for handling the changes in the textfield when the edit mode is on. These props are as follows:

<TableComplex
    startEditing={this.startEditing}
    stopEditing={this.stopEditing}
    handleChange={this.handleChange}
    handleRemove={this.handleRemove}
/> 

 

The 4 important functions which are passed as props to the Table component are:

  1. startEditing() function:

When the edit icon is clicked for any row, then the edit mode should be enabled for that row. This also changes the edit icon to a check icon. The columns corresponding to the room and the name of the device should turn into text fields to enable the user to edit them. The implementation of this function is as follows:

startEditing = i => {
    this.setState({ editIdx: i });
}; 

 

‘editIdx’ is a variable which contains the row index for which the edit mode is currently on. This information is passed to the Table component which then handles the editing of the row.

  1. stopEditing() function:

When the check icon is clicked for any row, then the edit mode should be disabled for that row and the updated data should be stored on the server. The columns corresponding to the room and the name of the device should turn back into label texts. The implementation of this function is as follows:

stopEditing = i => {
  let data = this.state.obj;
  this.setState({ editIdx: -1 });
  // AJAX call to server to store the updated data
  $.ajax({
    url:
      BASE_URL + '/aaa/addNewDevice.json?macid=' + macid + '&name=' + devicename + '&room=' + room + '&latitude=' + latitude + '&longitude=' + longitude + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};

 

The value of ‘editIdx’ is also set to -1 as the editing mode is now off for all rows.

  1. handleChange() function:

When the edit mode is on for any row, if we change the value of any text field, then that updated value is stored so that we can edit it without the value getting reset to the initial value of the field.

handleChange = (e, name, i) => {
  const value = e.target.value;
  let data = this.state.obj;
  this.setState({
    obj: data.map((row, j) => (j === i ? { ...row, [name]: value } : row)),
  });
};

 

  1. handleRemove() function:

When the delete icon is clicked for any row, then that row should get deleted. This change should be reflected to us on the site, as well as on the server. Hence, The implementation of this function is as follows:

handleRemove = i => {
  let data = this.state.obj;
  let macid = data[i].macid;

  this.setState({
    obj: data.filter((row, j) => j !== i),
  });
  // AJAX call to server to delete the info of device with specified Mac Address
  $.ajax({
    url:
      BASE_URL + '/aaa/removeUserDevices.json?macid=' + macid + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};  

 

This is how the Table View has been implemented in the Devices tab in Settings on SUSI.AI Web Client.

Resources

 

Continue Reading

List all the Users Registered on SUSI.AI

In this blog, I’ll be telling on how SUSI admins can access list of all the registered users from SUSI-server. Following this, they may modify/edit user role of any registered user.

What is User Role?

A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely :

  • BOT
  • ANONYMOUS
  • USER  
  • REVIEWER
  • ACCOUNTCREATOR
  • ADMIN
  • BUREAUCRAT

* Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely.

All the users who are not logged in but interacting with SUSI are anonymous users. These are only subject to chat with SUSI, login, signup or may use forgot password service. Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them. Privileged users are those who have special rights with them. These are more like moderators with much special rights than any other user. At the top level of the hierarchy are the admins. These users have more rights than anyone. They can change role of any other user, override decision of any privileged user as well.

Let us now look at the control flow of this.

First things first, make a component of User List in the project. Let us name it ListUsers and since it has to be accessible by those users who possess ADMIN rights, you will find it enclosed in Admin package in components folder. Open up

index.js file, import Listusers component  and add route to it in the following way :

...//other import statements
import ListUser from "./components/Admin/ListUser/ListUser";
...//class definition and other methods
<Route path="/listUser" component={ListUser}/>
//other routes defined

Find a suitable image for “List Users” option and add the option for List Users in static appbar component along with the image. We have used Material UI’s List image in our project.

...// other imports

import List from 'material-ui/svg-icons/action/list';

Class and method definition

<MenuItem primaryText="List Users"
          onTouchTap={this.handleClose}
          containerElement={<Link to="/listUser" />}
                rightIcon={<List/>}
      />

...//other options in top right corner menu

Above code snippet will add an option to redirect admins to ‘/listUsers’ route. Let us now have a closer look at functionality of both client and server. By now you must have known what ComponentDidMount does. {If not, I’ll tell you. This is a method which is given first execution after the page is rendered. For more information, visit this link}. As mentioned earlier as well that this list will be available only for admins and may be even extended for privileged users but not for anonymous or any other user, an AJAX call is made to server in ComponentDidMount of ‘listuser’ route which returns the base user role of current user. If user is an Admin, another method, fetchUsers() is called.

let url;
        url = "http://api.susi.ai/aaa/account-permissions.json";
        $.ajax({
            url: url,
            dataType: 'jsonp',
            jsonpCallback: 'py',
            jsonp: 'callback',
            crossDomain: true,
            success: function (response) {
                console.log(response.userRole)
                if (response.userRole !== "admin") {
                    console.log("Not an admin")
                } else {
                    this.fetchUsers();
                    console.log("Admin")
                }
            }.bind(this),
});

In fetchUsers method, an AJAX call is made to server which returns username in JSONArray. The response looks something likes this :

{
	"users" : {
		"email:""[email protected]",
...
},
"Username":["[email protected]", "[email protected]"...]
}

Now, only rendering this data in a systematic form is left. To give it a proper look, we have used material-ui’s table. Import Table, TableBody, TableHeader,

   TableHeaderColumn, TableRow, TableRowColumn from material-ui/table.

In fetchUsers method, response is catched in data Oblect. Now the keys are extracted from the JSON response and mapped with an array. Iterating through array received as username array, we get list of all the registered users. Now, popuulate the data in the table you generated.

return (
                        <TableRow key={i}>
                            <TableRowColumn>{++i}>
                            <TableRowColumn>{name}</TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                        </TableRow>
                    )

Above piece of code may help you while populating the table. These details are returned from susi server which gets a list of all the registered in the following manner. First, it checks if base url of this user is something apart from admin. If not, it returns error which may look like this :

Failed to load resource: the server responded with a status of 401 (Base user role not sufficient. Your base user role is 'ANONYMOUS', your user role is 'anonymous')

Otherwise, it will generate a client identity, use to to get an authorization object which will loop through authorization.json file and return all the users encoded as JSONArray.

Additional Resources

  1. Official Material UI Documentation on Tables from marterial-ui
  2. Answer by Marco Bonelli on Stackoverflow on How to map JSON response in JavaScript?
  3. Answer by janpieter_z on Stackoverflow – on Render JSON data in ReactJS table
Continue Reading

Extending Markdown Support in Yaydoc

Yaydoc, our automatic documentation generator, builds static websites from a set of markup documents in markdown or reStructuredText format. Yaydoc uses the sphinx documentation generator internally hence reStructuredText support comes out of the box with it. To support markdown we use multiple techniques depending on the context. Most of the markdown support is provided by recommonmark, a docutils bridge for sphinx which basically converts markdown documents into proper docutil’s abstract syntax tree which is then converted to HTML by sphinx. While It works pretty well for most of the use cases, It does fall short in some instances. They are discussed in the following paragraphs.

The first problem was inclusion of other markdown files in the starting page. This was due to the fact that markdown does not supports any include mechanism. And if we used the reStructuredText include directive, the included text was parsed as reStructuredText. This problem was solved earlier using pandoc – an excellent tool to convert between various markup formats. What we did was that we created another directive mdinclude which converts the markdown to reStructuredText before inclusion. Although this was solved a while ago, The reason I’m discussing this here is that this was the inspiration behind the solution to our recent problem.

The problem we encountered was that recommonmark follows the Commonmark spec which is an ongoing effort towards standardization of markdown which has been somewhat lacking till now. The process is currently going on so the recommonmark library doesn’t yet support the concept of extensions to support various features of different markdown flavours not in the core commonmark spec. We could have settled for only supporting the markdown features in the core spec but tables not being present in the core spec was problematic. We had to support tables as it is widely used in most of the docs present in github repositories as GFM(Github Flavoured Markdown) renders ascii tables nicely.

The solution was to use a combination of recommonmark and pandoc. recommonmark provides a eval_rst code block which can be used to embed non-section reStructuredText within markdown. I created a new MarkdownParser class which inherited the CommonMarkParser class from recommonmark. Within it, using regular expressions, I convert any text within `<!– markdown+ –>` and `<!– endmarkdown+ –>`  into reStructuredText and enclose it within eval_rst code block. The result was that tables when enclosed within those trigger html comments would be converted to reST tables and then enclosed within eval_rst block which resulted in recommonmark renderering them properly. Below is a snippet which shows how this was implemented.

import re
from recommonmark.parser import CommonMarkParser
from md2rst import md2rst


MARKDOWN_PLUS_REGEX = re.compile('<!--\s+markdown\+\s+-->(.*?)<!--\s+endmarkdown\+\s+-->', re.DOTALL)
EVAL_RST_TEMPLATE = "```eval_rst\n{content}\n```"


def preprocess_markdown(inputstring):
    def callback(match_object):
        text = match_object.group(1)
        return EVAL_RST_TEMPLATE.format(content=md2rst(text))

    return re.sub(MARKDOWN_PLUS_REGEX, callback, inputstring)


class MarkdownParser(CommonMarkParser):
    def parse(self, inputstring, document):
        content = preprocess_markdown(inputstring)
        CommonMarkParser.parse(self, content, document)

Resources

Continue Reading
Close Menu