Adding 3D Home Screen Quick Actions to SUSI iOS App

Home screen quick actions are a convenient way to perform useful, app-specific actions right from the Home screen, using 3D Touch. Apply a little pressure to an app icon with your finger—more than you use for tap and hold—to see a list of available quick actions. Tap one to activate it. Quick actions can be static or dynamic.

We have added some 3D home screen quick action to our SUSI iOS app. In this post, we will see how they are implemented and how they work.

The following 3D home screen quick actions are added to SUSI iOS:

  • Open SUSI Skills – user can directly go to SUSI skills without opening a chat screen.
  • Customize Settings – user can customize their setting directly by using this quick action.
  • Setup A Device – when the user quickly wants to configure his/her device for SUSI Smart Speaker, this is quick action is very helpful in that.
  • Change SUSI’s Voice – user can change SUSI message reading language accents directly from this quick action.

Each Home screen quick action includes a title, an icon on the left or right (depending on your app’s position on the home screen), and an optional subtitle. The title and subtitle are always left-aligned in left-to-right languages.

Step 1 – Adding the Shortcut Items

We add static home screen quick actions using the UIApplicationShortcutItems array in the app Info.plist file. Each entry in the array is a dictionary containing items matching properties of the UIApplicationShortcutItem class. As seen in screenshot below, we have 4 shortcut items and each item have three properties UIApplicationShortcutItemIconType/UIApplicationShortcutItemIconFile, UIApplicationShortcutItemTitle, and UIApplicationShortcutItemType.

  • UIApplicationShortcutItemIconType and UIApplicationShortcutItemIconFile is the string for setting icon for quick action. For the system icons, we use UIApplicationShortcutItemIconType property and for the custom icons, we use UIApplicationShortcutItemIconFile.
  • UIApplicationShortcutItemTitle is a required string that is displayed to the user.
  • UIApplicationShortcutItemType is a required app specific string used to identify the quick action.

Step 2 – Handling the Shortcut

AppDelegate is the place where we handle all the home screen quick actions. We define these variables:

var shortcutHandled: Bool!
var shortcutIdentifier: String?

When a user chooses one of the quick actions the launch of the system or resumes the app and calls the performActionForShortcutItem method in app delegate:

func application(_ application: UIApplication,
                     performActionFor shortcutItem: UIApplicationShortcutItem,
                     completionHandler: @escaping (Bool) -> Void) {
        shortcutIdentifier = shortcutItem.type
        shortcutHandled = true
        completionHandler(shortcutHandled)
    }

Whenever the application becomes active, applicationDidBecomeActive function is called:

func applicationDidBecomeActive(_ application: UIApplication) {
        // Handel Home Screen Quick Actions
        handelHomeActions()
    }

Inside the applicationDidBecomeActive function we call the handleHomeAction() method which handles the home screen quick action.

func handelHomeActions() {
       if shortcutHandled == true {
           shortcutHandled = false
           if shortcutIdentifier == ControllerConstants.HomeActions.openSkillAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.customizeSettingsAction {
               // Handle action accordingly
           } else if shortcutIdentifier == ControllerConstants.HomeActions.setupDeviceAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.changeVoiceAction {
               // Handle action accordingly
               }
           }
       }
   }

Final Output:

Resources –

  1. Home Screen Quick Actions – Human Interface Guidelines by Apple
  2. Adding 3D Touch Quick Actions by Use Your Leaf
  3. Apple’s documentation on performActionFor:completionHandler
Continue ReadingAdding 3D Home Screen Quick Actions to SUSI iOS App

Implementing API to allow Admins to modify config of devices of any user

As any user can add or remove devices from their account, there needed to be a way by which Admins can manage the user devices. The Admins and higher user roles should have the access to modify the config of devices of any user. This blog post explains how an API has been implemented to facilitate Admins and higher user roles to change config of devices of any user.

Implementing a servlet to allow changing review status of a Skill

The basic task of the servlet is to allow Admin and higher user roles to modify the config of devices of any user. The Admin should be allowed to edit the name of the device and also the room of the device, similar to how a user can edit his own devices.

Here is the implementation of the API:

  1. The API should be usable to only the users who have a user role Admin or higher. Only those with minimum Admin rights should be allowed to control what Skills are displayed on the CMS site. This is implemented as follows:

   @Override
    public UserRole getMinimalUserRole() {
        return UserRole.ADMIN;
    }

 

  1. The endpoint for the API is ‘/cms/modifyUserDevices.json’. This is implemented as follows:

   @Override
    public String getAPIPath() {
        return "/cms/modifyUserDevices.json";
    }

 

  1. The main method of the servlet is the serviceImpl() method. This is where the actual code goes which will be executed each time the API is called. This is implemented as follows:

    JSONObject result = new JSONObject(true);
    Collection<ClientIdentity> authorized = DAO.getAuthorizedClients();
    List<String> keysList = new ArrayList<String>();
    authorized.forEach(client -> keysList.add(client.toString()));
    String[] keysArray = keysList.toArray(new 
    String[keysList.size()]);

    List<JSONObject> userList = new ArrayList<JSONObject>();
    for (Client client : authorized) {
        JSONObject json = client.toJSON();

        if(json.get("name").equals(email)) {
            ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, client.getName());
            Authorization authorization = DAO.getAuthorization(identity);

            ClientCredential clientCredential = new ClientCredential(ClientCredential.Type.passwd_login, identity.getName());
            Authentication authentication = DAO.getAuthentication(clientCredential);

            Accounting accounting = DAO.getAccounting(authorization.getIdentity());

            if(accounting.getJSON().has("devices")) {

                JSONObject userDevice = accounting.getJSON().getJSONObject("devices");
                if(userDevice.has(macid)) {
                    JSONObject deviceInfo = userDevice.getJSONObject(macid);
                    deviceInfo.put("name", name);
                    deviceInfo.put("room", room);
                }
                else {
                    throw new APIException(400, "Specified device does not exist.");
                }

            } else {
                json.put("devices", "");
            }
            accounting.commit();
        }
    }

 

Firstly, the list of authorized clients is fetched using DAO.getAuthorizedClients() and is put in an ArrayList. Then we traverse through each element of this ArrayList and check if the device exists by checking if there’s a key-value pair corresponding to the macid passed in the query parameter. If the device doesn’t exist, then an exception is thrown. However, if the macid exists in the traversed element of the ArrayList, then we put the name and the room of the device as passed as query parameters in that particular element of the ArrayList, so as to overwrite the existing name and room of the device of the user.

This is how an API has been implemented which allows Admins and higher user roles to modify the config of devices of any user.

Resources

Continue ReadingImplementing API to allow Admins to modify config of devices of any user

Displaying name of users in Users tab in Admin Panel

In the Users tab in the Admin Panel, we have a lot of user information displayed in a tabular form. This information is fetched from the accounting objects of each user. As the users are now able to also store their name in their respective accounting object, hence we needed to implement a feature to display the name of the users in the Users table in a separate column. This blog post explains how the user names are fetched from the respective accounting objects and are then displayed in the Users table in the Admin Panel.

How is name of user stored on the server?

The name of any user is stored in the user’s accounting object. All the settings of a user are stored in a JSONObject with the key name as ‘settings’. The name of a user is also stored in ‘settings’ JSONObject. This is shown as follows:

Modifying GetUsers.java to return name of users

The endpoint /aaa/getUsers.json is used to return the accounting info of all users. This includes their signup time, last login time, last login IP, etc. We needed to modify it to return the name of users also along with the already returned data. This is implemented as follows:

   if(accounting.getJSON().has("settings")) {
        JSONObject settings = accounting.getJSON().getJSONObject("settings");
        if(settings.has("userName")) {
            json.put("userName", settings.get("userName"));
        }
        else {
            json.put("userName", "");
        }
    } else {
        json.put("userName", "");
    }
    accounting.commit();

 

Fetching names of all users from the server

We need to make an AJAX call to ‘/aaa/getUsers.json’ as soon as we switch to the Users tab in the Admin Panel. We need to extract all the required data from the JSON response object and put them in state variables so that they can further be used as data indexes for different columns of the table. The implementation of the AJAX call is as follows:

   let url =
      `${urls.API_URL}/aaa/getUsers.json?access_token=` +
      cookies.get('loggedIn') +
      '&page=' +
      page;
    $.ajax({
      url: url,
      dataType: 'jsonp',
      jsonp: 'callback',
      crossDomain: true,
      success: function(response) {
        let userList = response.users;
        let users = [];
        userList.map((data, i) => {
          let user = {
            userName: data.userName,
          };
          users.push(user);
          return 1;
        });
        this.setState({
          data: users,
        });
      }.bind(this)
    });

 

Displaying name of users in Users tab in Admin Panel

We needed to add another column titled ‘User Name’ in the Users table in the Admin Panel. The ‘dataIndex’ attribute of the Ant Design table component specifies the data value which is to be used for that particular column. For our purpose, our data value which needs to be displayed in the ‘User Name’ column is ‘userName’. We also specify a width of the column as another attribute. The implementation is as follows:

   this.columns = [
      // other columns
      {
        title: 'User Name',
        dataIndex: 'userName',
        width: '12%',
      }
      // other columns
    ];

 

This is how the names of users are fetched from their accounting object and are then being displayed in the Users tab in Admin Panel.

Resources

Continue ReadingDisplaying name of users in Users tab in Admin Panel

Adding Data Point Markers to OSM

PSLab Android app supports multiple sensors external and internal. Users can view sensor readings and record them into a csv file for later use. They can also have their location embedded into the data set they are recording. Saving the location related to each sensor reading is important. Say in a school experiment teacher can ask the student to measure dust particle concentration in city and inside his house. If the data set didn’t have any reference to the location where it was recorded, it is just incomplete. To facilitate this feature, we have enabled location in data recordings.

Enabling location is just not enough. User should be able to view the locations. Probably on a map itself would be a good idea. With the use of Open Street Maps, we can add markers to specific locations. These markers can be used as a point to show on map where a specific data set had been recorded by the user. This can be implemented further to add additional features such as standardized labels to view which data set is at which location etc.

Figure 1: Markers on Map

Unlike Google Maps API, in Open Street Maps there is no direct implementation to add a marker. But it is not a hard implementation either. We can simply create a class extending map overlays and use that as a base to add markers.

We can start by creating a new class that extends ItemizedOverlay<OverlayItem> class as follows. In this class, it would be wise to have an array list full of markers we are using in the map to modularize the whole markers related tasks into this one class rather than implementing in every place where map is used.

public class MapOverlay extends ItemizedOverlay<OverlayItem> {

    private ArrayList<OverlayItem> overlayItemList = new   ArrayList<OverlayItem>();

}

 

Once the class is initiated, have the following methods implemented. The following method will add markers to the array list we have created at the beginning.

public void addItem(GeoPoint p, String title, String snippet){
  OverlayItem newItem = new OverlayItem(title, snippet, p);
  overlayItemList.add(newItem);
  populate(); 
}

 

This method will be used to handle focusing events related to map markers.

@Override
public boolean onSnapToItem(int arg0, int arg1, Point arg2, IMapView arg3){
  return false;
}

 

Following method is used by the map itself to generate markers from the marker list.

@Override
protected OverlayItem createItem(int arg0) {
  return overlayItemList.get(arg0);
}

 

This method is an overriden method we will have to include in the class body.

@Override
public int size() {
  return overlayItemList.size();
}

 

Once the overlay class is completed, we can move on to actual implementation of a map on Openstreetmap view.

From the main activity where the map is viewed, initiate the marker overlay class and pass the drawable that needs to be shown as the marker to the class constructor as follows:

MapOverlay mapoverlay = null;
Drawable marker=getResources().getDrawable(android.R.drawable.map_hand);
int markerWidth = marker.getIntrinsicWidth();
int markerHeight = marker.getIntrinsicHeight();
marker.setBounds(0, markerHeight, markerWidth, 0);

ResourceProxy resourceProxy = new DefaultResourceProxyImpl(getApplicationContext());
mapoverlay = new MapOverlay(marker, resourceProxy);
mapView.getOverlays().add(mapoverlay);
         
GeoPoint point = new GeoPoint(55.75, 37.616667);
mapoverlay.addItem(point, "Russia", "Russia");

 

We can add as many GeoPoints as we want to markers list and they all will be displayed on the map like this;

Figure 2: Final Output of Markers

Reference:

http://android-coding.blogspot.com/2012/06/example-of-implementing-openstreetmap.html

Continue ReadingAdding Data Point Markers to OSM

Disable editing for non-editable skills for non-admin users

As the Skills in SUSI Skill CMS are publicly editable, any user has the access to edit them. Hence, there needed to be a better control over who can edit the Skills in CMS. We needed to implement a feature to allow Admins and higher user roles to change the status of a Skill to non-editable. The subsequent implementation on CMS would require disabling editing for non-editable Skills for non-admin users. This blog post explains how this feature has been implemented in SUSI.AI.

Adding a boolean parameter ‘editable’ to the Skill metadata

We needed to add a boolean parameter in the Skill metadata for each Skill. The boolean parameter is ‘editable’. If its value is true, then it implies that editing should be allowed for that Skill. If it is set to false, then the Skill should not be editable for non-admin users. By default, its value has been set to true for all Skills. This is implemented as follows in the SusiSkill.java file:

    // in the getSkillMetadata() method
  skillMetadata.put("editable", getSkillEditStatus(model, group, language, skillname));

    // declaration of the getSkillEditStatus() method
    public static boolean getSkillEditStatus(String model, String group, String language, String skillname) {
        // skill status
        JsonTray skillStatus = DAO.skillStatus;
        if (skillStatus.has(model)) {
            JSONObject modelName = skillStatus.getJSONObject(model);
            if (modelName.has(group)) {
                JSONObject groupName = modelName.getJSONObject(group);
                if (groupName.has(language)) {
                    JSONObject languageName = groupName.getJSONObject(language);
                    if (languageName.has(skillname)) {
                        JSONObject skillName = languageName.getJSONObject(skillname);

                        if (skillName.has("editable")) {
                            return false;
                        }
                    }
                }
            }
        }
        return true;
    }

 

Allowing Admin and higher user roles to change edit status of any Skill

This is facilitated by the endpoint ‘/cms/changeSkillStatus.json’. Its minimum base user role is set to Admin so that only Admins and higher user roles are able to change status of any Skill. A sample API call to this endpoint to change the edit status of any Skill to ‘false’ is as follows:

http://127.0.0.1:4000/cms/changeSkillStatus.json?model=general&group=Knowledge&language=en&skill=aboutsusi&editable=false&access_token=zdasIagg71NF9S2Wu060ZxrRdHeFAx

 

If we want to change the edit status of any Skill to ‘false’, then we need to add the Skill to the ‘skillStatus.json’ file. For this, we need to traverse inside the JSONObject in the ‘skillStatus.json’ file. We need to traverse inside the model, group and language as specified in the query parameters. This is done as follows:

   if(editable.equals("false")) {
       skill_status.put("editable", false);
   }

   JsonTray skillStatus = DAO.skillStatus;

   if (skillStatus.has(model_name)) {
        modelName = skillStatus.getJSONObject(model_name);
        if (modelName.has(group_name)) {
            groupName = modelName.getJSONObject(group_name);
            if (groupName.has(language_name)) {
                languageName = groupName.getJSONObject(language_name);

                if (languageName.has(skill_name)) {
                    skillName = languageName.getJSONObject(skill_name);

                    if(editable != null && editable.equals("false")) {
                        skillName.put("editable", false);
                    }
                    else if(editable != null && editable.equals("true")) {
                        skillName.remove("editable");
                    }

                    skillStatus.commit();
                    result.put("accepted", true);
                    result.put("message", "Skill status changed successfully.");
                    return new ServiceResponse(result);
                }
            }
        }
    }

 

If we want to change the edit status of any Skill to ‘true’, then we need to remove the Skill from the ‘skillStatus.json’ file. We also need to remove all the empty JSONObjects inside the ‘skillStatus.json’ file, if they are created in the process of removing Skills from it. This is done as follows:

   if (skillStatus.has(model_name)) {
        modelName = skillStatus.getJSONObject(model_name);
        if (modelName.has(group_name)) {
            groupName = modelName.getJSONObject(group_name);
            if (groupName.has(language_name)) {
                languageName = groupName.getJSONObject(language_name);
                if (languageName.has(skill_name)) {
                    skillName = languageName.getJSONObject(skill_name);
                    if(editable != null && editable.equals("true")) {
                        skillName.remove("editable");
                    }
                    if(skillName.length() == 0) {
                        languageName.remove(skill_name);
                        if(languageName.length() == 0) {
                            groupName.remove(language_name);
                            if(groupName.length() == 0) {
                                modelName.remove(group_name);
                                if(modelName.length() == 0) {
                                    skillStatus.remove(model_name);
                                }
                            }
                        }
                    }
                    skillStatus.commit();
                }
            }
        }
    }

 

Disabling editing for non-editable Skills for non-admin users on Skill CMS

For the Skills whose edit status has been set to ‘false’ by the Admins, we need to allow the non-admin users to only be able to view the code of the Skill, and not permit them to change the code and save the changes to the Skill. We need to display a message to the users about the possible reasons. All the code for displaying the message is put in an if() condition as follows:

   if (
      cookies.get('loggedIn') &&
      !this.state.editable &&
      !this.state.showAdmin
    )

 

This is how the Skill edit page for a non-editable Skill would look like for a non-admin user:

For an Admin user, this would look exactly same like an editable Skill page. Admin user would be able to edit and make changes to the Skill code and save the changes.

This is how editing of non-editable Skills have been disabled for non-admin users.

Resources

Continue ReadingDisable editing for non-editable skills for non-admin users

Giving users option to switch between All and Reviewed Only Skills on SUSI Skill CMS

There are a lot of Skills on SUSI Skill CMS. Any registered user has the access to creating his/her own Skills. Hence, we need to give the users an option on SUSI Skill CMS whether they want to see all the Skills, or only those Skills that have been tested thoroughly and have been approved by the Admin and higher user roles. This blog post explains how this feature has been implemented on the SUSI Skill CMS.

How is review status of any Skill changed on the server?

The API endpoint which allows Admin and higher user roles to change the review status of any Skill on the server is ‘/cms/changeSkillStatus.json’. It takes the following parameters:

  • model: Model of the Skill
  • group: Group of the Skill
  • language: Language of the Skill
  • skill: Skill name
  • reviewed: A boolean parameter which if true, signifies that the Skill has been approved.

Sample API call:

https://api.susi.ai/cms/changeSkillStatus.json?model=general&group=Knowledge&language=en&skill=aboutsusi&reviewed=true&access_token=yourAccessToken

 

Fetching reviewed only Skills from the server

The ‘/cms/getSkillList.json’ endpoint has been modified to facilitate returning only the Skills whose review status is true. This is done by the following API call:

https://api.susi.ai/cms/getSkillList.json?reviewed=true

 

Creating checkbox to switch between All and Reviewed Only Skills

Checkbox is one of the many Material-UI components. Hence, we need to first import it before we can use it directly in our BrowseSkill component.

import Checkbox from 'material-ui/Checkbox';

 

In the constructor of the BrowseSkill class, we set states for two variables as follows:

constructor(props) {
  super(props);
    this.state = {
      // other state variables
      showSkills: '',
      showReviewedSkills: false,
    };
}

 

In the above code for the constructor, we have set two state variables. ‘showSkills’ is a string which can either be an empty string, or ‘&reviewed=true’. We want to append this string to the ‘/cms/getSkillList.json’ API call because it would determine whether we want to fetch All Skills or reviewed only Skills. The second variable ‘showReviewedSkills’ is a boolean used to keep record of the current state of the page. If it is true, then it means that currently, only the reviewed Skills are being displayed on the CMS site.

Implementation of the checkbox

This is how the Checkbox has been implemented for the purpose of switching between All and Reviewed Only Skills:

 <Checkbox
    label="Show Only Reviewed Skills"
    labelPosition="right"
    className="select"
    checked={this.state.showReviewedSkills}
    labelStyle={{ fontSize: '14px' }}
    iconStyle={{ left: '4px' }}
    style={{
      width: '256px',
      paddingLeft: '8px',
      top: '3px',
    }}
    onCheck={this.handleShowSkills}
  />

 

As can be seen from the above code, the initial state of the checkbox is unchecked as initially, the value of the state variable ‘showReviewedSkills’ is set to false in the constructor. This means that initially all Skills will be shown to the user. On clicking on the checkbox, handleShowSkills() function is called. Its implementation is as follows:

  handleShowSkills = () => {
    let value = !this.state.showReviewedSkills;
    let showSkills = value ? '&reviewed=true' : '';
    this.setState(
      {
        showReviewedSkills: value,
        showSkills: showSkills,
      },
      function() {
        this.loadCards();
      },
    );
  };

 

In the handleShowSkills() function, firstly we store the current value of the state variable ‘showReviewedSkills’. The value of ‘showSkills’ string is determined according to the value of ‘showReviewedSkills’. Then the states of both these variables are updated in the setState() function. Lastly, loadCards() function is called.

In the loadCards() function, we append the value of the state variable ‘showSkills’ to the AJAX call to the ‘/cms/getSkillList.json’ endpoint. The URL used for the API call is as follows:

https://api.susi.ai/cms/getSkillList.json?group=' +
  this.props.routeValue +
  '&language=' +
  this.state.languageValue +
  this.state.filter +
  this.state.showSkills;

 

This is how the implementation of the feature to give users an option to switch between All and Reviewed Only Skills has been done.

Resources

Continue ReadingGiving users option to switch between All and Reviewed Only Skills on SUSI Skill CMS

Fetching Info of All Users and their connected devices for the SUSI.AI Admin Panel

Fetching the data of all users is required for displaying the list of users on the SUSI.AI Admin panel. It was also required to fetch the information of connected devices of the user along with the other user data. The right to fetch the data of all users should only be permitted to user roles “OPERATOR” and above. This blog post explains how the data of connected devices of all users is fetched, which can then be used in the Admin panel.

How is user data stored on the server?

All the personal accounting information of any user is stored in the user’s accounting object. This is stored in the “accounting.json” file. The structure of this file is as follows:

{
  "email:akjn11@gmail.com": {
    "devices": {
      "8C-39-45-23-D8-95": {
        "name": "Device 2",
        "room": "Room 2",
        "geolocation": {
          "latitude": "54.34567",
          "longitude": "64.34567"
        }
      }
    },
    "lastLoginIP": "127.0.0.1"
  },
  "email:akjn22@gmail.com": {
    "devices": {
      "1C-29-46-24-D3-55": {
        "name": "Device 2",
        "room": "Room 2",
        "geolocation": {
          "latitude": "54.34567",
          "longitude": "64.34567"
        }
      }
    },
    "lastLoginIP": "127.0.0.1"
  }
}

 

As can be seen from the above sample content of the “accounting.json” file, we need to fetch this data so that it can then be used to display the list of users along with their connected devices on the Admin panel.

Implementing API to fetch user data and their connected devices

The endpoint of the servlet is “/aaa/getUsers.json” and the minimum user role for this servlet is “OPERATOR”. This is implemented as follows:

   @Override
    public String getAPIPath() {
        return "/aaa/getUsers.json";
    }

    @Override
    public UserRole getMinimalUserRole() {
        return UserRole.OPERATOR;
    }

 

Let us go over the main method serviceImpl() of the servlet:

  • We need to traverse through the user data of all authorized users. This is done by getting the data using DAO.getAuthorizedClients() and storing them in a Collection. Then we extract all the keys from this collection, which is then used to traverse into the Collection and fetch the user data. The implementation is as follows:

    Collection<ClientIdentity> authorized = DAO.getAuthorizedClients();
    List<String> keysList = new ArrayList<String>();
    authorized.forEach(client -> keysList.add(client.toString()));

    for (Client client : authorized) {
        // code           
    }

 

  • Then we traverse through each client and generate a client identity to get the user role of the client. This is done using the DAO.getAuthorization() method. The user role of the client is also put in the final object which we want to return. This is implemented as follows:

    JSONObject json = client.toJSON();
    ClientIdentity identity = new 
    ClientIdentity(ClientIdentity.Type.email, client.getName());
    Authorization authorization = DAO.getAuthorization(identity);
    UserRole userRole = authorization.getUserRole();
    json.put("userRole", userRole.toString().toLowerCase());

 

  • Then the client credentials are generated and it is checked whether the user is verified or not. If the user is verified, then in the final object, “confirmed” is set to true, else it is set to false.

    ClientCredential clientCredential = new ClientCredential (ClientCredential.Type.passwd_login, identity.getName());
    Authentication authentication = DAO.getAuthentication(clientCredential);

    json.put("confirmed", authentication.getBoolean("activated", false));

 

  • Then we fetch the accounting object of the user using DAO.getAccounting(), and extract all the user data and put them in separate key value pairs in the final object which we want to return. As the information of all connected devices of a user is also stored in the user’s accounting object, that info is also extracted the same way and put into the final object.

    Accounting accounting = DAO.getAccounting(authorization.getIdentity());
    if (accounting.getJSON().has("lastLoginIP")) {
        json.put("lastLoginIP", accounting.getJSON().getString("lastLoginIP"));
    } else {
        json.put("lastLoginIP", "");
    }

    if(accounting.getJSON().has("signupTime")) {
        json.put("signupTime", accounting.getJSON().getString("signupTime"));
    } else {
        json.put("signupTime", "");
    }

    if(accounting.getJSON().has("lastLoginTime")) {
        json.put("lastLoginTime", accounting.getJSON().getString("lastLoginTime"));
    } else {
        json.put("lastLoginTime", "");
    }

    if(accounting.getJSON().has("devices")) {
        json.put("devices", accounting.getJSON().getJSONObject("devices"));
    } else {
        json.put("devices", "");
    }
    accounting.commit();

 

This is how the data of all users is fetched by any Admin or higher user role, and is then used to display the user list on the Admin panel.

Resources

Continue ReadingFetching Info of All Users and their connected devices for the SUSI.AI Admin Panel

Testing Endpoints on Local Server

All servlets in SUSI.AI have a BaseUserRole defined. It represents the access level you need to access the endpoint corresponding to that servlet. The lowermost BaseUserRole a SUSI.AI servlet can have is ANONYMOUS, which means that anyone can access the endpoint corresponding to these endpoints. But if the BaseUserRole is higher than that, then you need an access token to access the endpoint. This blog post explains how you can get access token to access the endpoints on a local server.

What are endpoints in an API?

An endpoint in API is one end of a communication channel. When an API interacts with another system, the touchpoints of this communication are considered endpoints. For APIs, an endpoint can include a URL of a server or service. Each endpoint is the location from which APIs can access the resources they need to carry out their function.

APIs work using ‘requests’ and ‘responses.’ When an API requests information from a web application or web server, it will receive a response. The place that APIs send requests and where the resource lives, is called an endpoint.

For example, the endpoint for https://api.susi.ai/cms/getSkillRating.json?queryParameters would be /cms/getSkillRating.json.

Servlets and Endpoints in SUSI.AI

All servlets in our SUSI project define an endpoint and also define a BaseUserRole, that is, the amount of privileges required to access the information on those endpoints. If the BaseUserRole defined is ANONYMOUS, then anyone can access the endpoint directly. But if the BaseUserRole is anything higher than that, then we would need an access token to access that.

How to get Access Token?

If you’re trying to access the endpoints with BaseUserRole higher than ANONYMOUS on the actual hosted server, then you can simply login to https://chat.susi.ai and get the access token from the Network tab of the Developers Tool. We can then use that token and pass that as a query parameter along with the other parameters of that particular endpoint. For example,

http://localhost:4000/aaa/listUserSettings.json?access_token=6O7cqoMbzlClxPwg1is31Tz5pjVwo3

 

But, the problem arises when you are trying to access such endpoints on local server. The local User data is completely different from the server User data. Hence, we need to generate an access token in localhost itself.

To generate access token for local server, we need to follow these steps :

  1. First, we need to hit the /aaa/signup.json endpoint with a new account credentials which we want to register for the localhost session. This is done as shown in below example:

http://localhost:4000/aaa/signup.json?signup=anyEmail&password=anyPassword

 

  1. Then, we need to hit the /aaa/login.json endpoint with the same credentials you registered in the previous step. This is done as shown in below example:

http://localhost:4000/aaa/login.json?login=yourEmail&type=access-token&password=yourPassword

 

If you’ve entered the registered credentials correctly, then the output of the /aaa/login.json endpoint would be a JSON as shown below:

{
  "accepted": true,
  "valid_seconds": 604800,
  "access_token": "7JPi7zNwemg1YYnr4d9JIdZMaIWizV",
  "message": "You are logged in as anyemail",
  "session": {"identity": {
    "type": "host",
    "name": "127.0.0.1_4e75edbb",
    "anonymous": true
  }}
}

 

As it can be seen from the above JSON response, we get the access token which we needed. Hence, copy this access token and store it somewhere because you can now use this access token to access the endpoints with BaseUserRole as User for this localhost session.

Note that you’ll have to follow all the above steps again if you start a fresh localhost session.

Resources

Continue ReadingTesting Endpoints on Local Server

Implementing Map View in Devices Tab in Settings

The Table View implemented in the Devices tab in settings on SUSI.AI Web Client has a column “geolocation” which displays the latitudinal and longitudinal coordinates of the device. These coordinates needed to be displayed on a map. Hence, we needed a Map View apart from the Table View, dedicated to displaying the devices pinpointed on a map. This blog post explains how this feature has been implemented on the SUSI.AI Web Client.

Modifying the fetched data of devices to suitable format

We already have the fetched data of devices which is being used for the Table View. We need to extract the geolocation data and store it in a different suitable format to be able to use it for the Map View. The required format is as follows:

[
   {
      "location":{
         "lat": latitude1,
         "lng": longitude2
      }
   },
   {
      "location":{
         "lat": latitude1,
         "lng": longitude2
      }
   }
]

 

To modify the fetched data of devices to this format, we modify the apiCall() function to facilitate extraction of the geolocation info of each device and store them in an object, namely ‘mapObj’. Also, we needed variables to store the latitude and longitude to use as the center for the map. ‘centerLat’ and ‘centerLng’ variables store the average of all the latitudes and longitudes respectively. The following code was added to the apiCall() function to facilitate all the above requirements:

let mapObj = [];
let locationData = {
  lat: parseFloat(response.devices[i].geolocation.latitude),
  lng: parseFloat(response.devices[i].geolocation.longitude),
};
centerLat += parseFloat(response.devices[i].geolocation.latitude);
centerLng += parseFloat(response.devices[i].geolocation.longitude);
let location = {
  location: locationData,
};
mapObj.push(location);
centerLat = centerLat / mapObj.length;
centerLng = centerLng / mapObj.length;
if (mapObj.length) {
  this.setState({
    mapObj: mapObj,
    centerLat: centerLat,
    centerLng: centerLng,
  });
}

 

The following code was added in the return function of Settings.react.js file to use the Map component. All the modified data is passed as props to this component.

<MapContainer
  google={this.props.google}
  mapData={this.state.mapObj}
  centerLat={this.state.centerLat}
  centerLng={this.state.centerLng}
  devicenames={this.state.devicenames}
  rooms={this.state.rooms}
  macids={this.state.macids}
/>

 

The implementation of the MapContainer component is as follows:

 componentDidUpdate() {
    this.loadMap();
  }

  loadMap() {
    if (this.props && this.props.google) {
      const {google} = this.props;
      const maps = google.maps;
      const mapRef = this.refs.map;
      const node = ReactDOM.findDOMNode(mapRef);

      const mapConfig = Object.assign({}, 
        
          center: { lat: this.props.centerLat, lng: this.props.centerLng },
          zoom: 2,
        }
      )
      this.map = new maps.Map(node, mapConfig);
    }
  }

 

Let us go over the code of MapContainer component step by step.

  1. Firstly, the componentDidUpdate() function calls the loadMap function to load the google map.

  componentDidUpdate() {
    this.loadMap();
  }

 

  1. In the loadMap() function, we first check whether props have been passed to the MapContainer component. This is done by enclosing all contents of loadMap function inside an if statement as follows:

  if (this.props && this.props.google) {
    // All code of loadMap() function
  } 

 

  1. Then we set the prop value to google, and maps to google maps props. This is done as follows:

  const {google} = this.props;
  const maps = google.maps;

 

  1. Then we look for HTML div ref ‘map’ in the React DOM and name it ‘node’. This is done as follows:

  const mapRef = this.refs.map;
  const node = ReactDOM.findDOMNode(mapRef);

 

  1. Then we set the center and the default zoom level of the map using the props we provided to the MapContainer component.

  {
    center: { lat: this.props.centerLat, lng: this.props.centerLng },
    zoom: 2,
  }

 

  1. Then we create a new Google map on the specified node (ref=’map’) with the specified configuration set above. This is done as follows:

this.map = new maps.Map(node, mapConfig);

 

  1. In the render function of the MapContainer component, we return a div with a ref ‘map’ as follows:

 render() {
    return (
      <div ref="map" style={style}>
        loading map...
      </div>
    );
  }

 

This is how the Map View has been implemented in the Devices tab in Settings on SUSI.AI Web Client.

Resources

Continue ReadingImplementing Map View in Devices Tab in Settings

Implementing Table View in Devices Tab in Settings

We can connect to the SUSI.AI Smart Speaker using our mobile apps (Android or iOS). But there needs to be something implemented which can tell you what all devices are linked to your account. This is in consistency with the way how Google Home devices and Amazon Alexa devices have this feature implemented in their respective apps, which allow you to see the list of devices connected to your account. This blog post explains how this feature has been implemented on the SUSI.AI Web Client.

Fetching data of the connected devices from the server

The information of the devices connected to an account is stored in the Accounting object of that user. This is a part of a sample Accounting object of a user who has 2 devices linked to his/her account. This is the data that we wish to fetch. This data is accessible at the /aaa/listUserSettings.json endpoint.

{
  "devices": {
    "37-AE-5F-7B-CA-3F": {
      "name": "Device 1",
      "room": "Room 1",
      "geolocation": {
        "latitude": "50.34567",
        "longitude": "60.34567"
      }
    },
    "9D-39-02-01-EB-95": {
      "name": "Device 2",
      "room": "Room 2",
      "geolocation": {
        "latitude": "52.34567",
        "longitude": "62.34567"
      }
    }
  }
} 

 

In the Settings.react.js file, we make an AJAX call immediately after the component is mounted on the DOM. This AJAX call is made to the /aaa/listUserSettings.json endpoint. The received response of the AJAX call is then used and traversed to store the information of each connected device in a format that would be more suitable to use as a prop for the table.

apiCall = () => {
    $.ajax({
      url: BASE_URL + '/aaa/listUserSettings.json?' + 'access_token=' +
  cookies.get('loggedIn');,
      type: 'GET',
      dataType: 'jsonp',
      crossDomain: true,
      timeout: 3000,
      async: false,
      success: function(response) {
        let obj = [];
         // Extract information from the response and store them in obj object
        obj.push(myObj);
        this.setState({
          dataFetched: true,
          obj: obj,
        });
      }
      }.bind(this),
  };

 

This is how the extraction of keys takes place inside the apiCall() function. We first extract the keys of the ‘devices’ JSONObject inside the response. The keys of this JSONObject are the Mac Addresses of the individual devices. Then we traverse inside the JSONObject corresponding to each Mac Address and store the name of the device, room and also the geolocation information of the device in separate variables, and then finally push all this information inside an object, namely ‘myObj’. This JSONObject is then pushed to a JSONArray, namely ‘obj’. Then a setState() function is called which sets the value of ‘obj’ to the updated ‘obj’ variable.

let keys = Object.keys(response.devices);
keys.forEach(i => {
    let myObj = {
        macid: i,
        devicename: response.devices[i].name,
        room: response.devices[i].room,
        latitude: response.devices[i].geolocation.latitude,
        longitude: response.devices[i].geolocation.longitude,
    };
}

 

This way we fetch the information of devices and store them in a variable named ‘obj’. This variable will now serve as the data for the table which we want to create.

Creating table from this data

The data is then passed on to the Table component as a prop in the Settings.react.js file.

<TableComplex
    // Other props
    tableData={this.state.obj}
/>

 

The other props passed to the Table component are the functions for handling the editing and deleting of rows, and also for handling the changes in the textfield when the edit mode is on. These props are as follows:

<TableComplex
    startEditing={this.startEditing}
    stopEditing={this.stopEditing}
    handleChange={this.handleChange}
    handleRemove={this.handleRemove}
/> 

 

The 4 important functions which are passed as props to the Table component are:

  1. startEditing() function:

When the edit icon is clicked for any row, then the edit mode should be enabled for that row. This also changes the edit icon to a check icon. The columns corresponding to the room and the name of the device should turn into text fields to enable the user to edit them. The implementation of this function is as follows:

startEditing = i => {
    this.setState({ editIdx: i });
}; 

 

‘editIdx’ is a variable which contains the row index for which the edit mode is currently on. This information is passed to the Table component which then handles the editing of the row.

  1. stopEditing() function:

When the check icon is clicked for any row, then the edit mode should be disabled for that row and the updated data should be stored on the server. The columns corresponding to the room and the name of the device should turn back into label texts. The implementation of this function is as follows:

stopEditing = i => {
  let data = this.state.obj;
  this.setState({ editIdx: -1 });
  // AJAX call to server to store the updated data
  $.ajax({
    url:
      BASE_URL + '/aaa/addNewDevice.json?macid=' + macid + '&name=' + devicename + '&room=' + room + '&latitude=' + latitude + '&longitude=' + longitude + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};

 

The value of ‘editIdx’ is also set to -1 as the editing mode is now off for all rows.

  1. handleChange() function:

When the edit mode is on for any row, if we change the value of any text field, then that updated value is stored so that we can edit it without the value getting reset to the initial value of the field.

handleChange = (e, name, i) => {
  const value = e.target.value;
  let data = this.state.obj;
  this.setState({
    obj: data.map((row, j) => (j === i ? { ...row, [name]: value } : row)),
  });
};

 

  1. handleRemove() function:

When the delete icon is clicked for any row, then that row should get deleted. This change should be reflected to us on the site, as well as on the server. Hence, The implementation of this function is as follows:

handleRemove = i => {
  let data = this.state.obj;
  let macid = data[i].macid;

  this.setState({
    obj: data.filter((row, j) => j !== i),
  });
  // AJAX call to server to delete the info of device with specified Mac Address
  $.ajax({
    url:
      BASE_URL + '/aaa/removeUserDevices.json?macid=' + macid + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};  

 

This is how the Table View has been implemented in the Devices tab in Settings on SUSI.AI Web Client.

Resources

 

Continue ReadingImplementing Table View in Devices Tab in Settings