Building Preference Screen in SUSI Android

SUSI provides various preferences to the user in the settings to customize the app. This allows the user to configure the application according to his own choice. There are different preferences available such as to select the theme or the language for text to speech. Preference Setting Activity is an important part of an Android application. Here we will see how we can implement it in an Android app taking SUSI Android (https://github.com/fossasia/susi_android) as the example.

Firstly, we will proceed by adding the Gradle Dependency for the Setting Preferences

compile 'com.takisoft.fix:preference-v7:25.4.0.3'

Then to create the custom style for our setting preference screen we can set

@style/PreferenceFixTheme

as the base theme and can apply various other modifications and color over this. By default it has the usual Day and Night theme with NoActionBar extension.

Now to make different preferences we can use different classes as shown below:

SwitchPreferenceCompat: This gives us the Switch Preference which we can use to toggle between two different modes in the setting.

EditTextPreference: This preference allows the user to give its own choice of number or string in the settings which can be used for different actions.

For more details on this you can refer the this link.

Implementation in SUSI Android

In SUSI Android we have created an activity named activity_settings which holds the Preference Fragment for the setting.

<?xml version="1.0" encoding="utf-8"?>


<fragment

  xmlns:android="http://schemas.android.com/apk/res/android"

  xmlns:tools="http://schemas.android.com/tools"

  android:id="@+id/chat_pref"

  android:name="org.fossasia.susi.ai.activities.SettingsActivity$ChatSettingsFragment"

  android:layout_width="match_parent"

  android:layout_height="match_parent"

  tools:context="org.fossasia.susi.ai.activities.SettingsActivity"/>

The Preference Settings Fragment contains different Preference categories that are implemented to allow the user to have different customization option while using the app. The pref_settings.xml is as follows

<?xml version="1.0" encoding="utf-8"?>

<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android"

  xmlns:app="http://schemas.android.com/apk/res-auto"

  android:title="@string/settings_title">


  <PreferenceCategory

      android:title="@string/server_settings_title">

      <PreferenceScreen

          android:title="@string/server_pref"

          android:key="Server_Select"

          android:summary="@string/server_select_summary">

      </PreferenceScreen>

  </PreferenceCategory>


  <PreferenceCategory

      android:title="@string/settings_title">

      <com.takisoft.fix.support.v7.preference.SwitchPreferenceCompat

          android:id="@+id/enter_key_pref"

          android:defaultValue="true"

          android:key="@string/settings_enterPreference_key"

          android:summary="@string/settings_enterPreference_summary"

          android:title="@string/settings_enterPreference_label" />

  </PreferenceCategory>

All the logic related to Preferences and their action is written in SettingsActivity Java class. It listens for any change in the preference options and take actions accordingly in the following way.

public class SettingsActivity extends AppCompatActivity {


  private static final String TAG = "SettingsActivity";

  private static SharedPreferences prefs;


  @Override

  protected void onCreate(Bundle savedInstanceState) {

      super.onCreate(savedInstanceState);

      prefs = getSharedPreferences(Constant.THEME, MODE_PRIVATE);

      Log.d(TAG, "onCreate: " + (prefs.getString(Constant.THEME, DARK)));

      if(prefs.getString(Constant.THEME, "Light").equals("Dark")) {

          setTheme(R.style.PreferencesThemeDark);

      }

      else {

          setTheme(R.style.PreferencesThemeLight);

      }

      setContentView(R.layout.activity_settings);


  }

The class contains a ChatSettingFragment which extends the PreferenceFragmentCompat to give access to override functions such as onPreferenceClick. The code below shows the implementation of it.

public boolean onPreferenceClick(Preference preference) {

              Intent intent = new Intent();

              intent.setComponent( new ComponentName("com.android.settings","com.android.settings.Settings$TextToSpeechSettingsActivity" ));

              startActivity(intent);

              return true;

          }

      });


      rate=(Preference)getPreferenceManager().findPreference(Constant.RATE);

      rate.setOnPreferenceClickListener(new Preference.OnPreferenceClickListener() {

          @Override

          public boolean onPreferenceClick(Preference preference) {

              startActivity(new Intent(Intent.ACTION_VIEW, Uri.parse("http://play.google.com/store/apps/details?id=" + getContext().getPackageName())));

              return true;

          }

      });
}

For diving more into the code we can refer to the github repo of Susi Android (https://github.com/fossasia/susi_android).

Resources

Continue ReadingBuilding Preference Screen in SUSI Android

Implementing the Message Response Status Indicators In SUSI WebChat

SUSI Web Chat now has indicators reflecting the message response status. When a user sends a message, he must be notified that the message has been received and has been delivered to server. SUSI Web Chat implements this by tagging messages with ticks or waiting clock icons and loading gifs to indicate delivery and response status of messages ensuring good UX.

This is implemented as:

  • When the user sends a message, the message is tagged with a `clock` icon indicating that the message has been received and delivered to server and is awaiting response from the server
  • When the user is waiting for a response from the server, we display a loading gif
  • Once the response from the server is received, the loading gif is replaced by the server response bubble and the clock icon tagged to the user message is replaced by a tick icon.

Lets visit SUSI WebChat and try it out.

Query : Hey

When the message is sent by the user, we see that the displayed message is tagged with a clock icon and the left side response bubble has a loading gif indicating that the message has been delivered to server and are awaiting response.

When the response from server is delivered, the loading gif disappears and the user message tagged with a tick icon.

 

How was this implemented?

The first step is to have a boolean flag indicating the message delivery and response status.

let _showLoading = false;

getLoadStatus(){
  return _showLoading;
},

The `showLoading` boolean flag is set to true when the user just sends a message and is waiting for server response.  When the user sends a message, the CREATE_MESSAGE action is triggered. Message Store listens to this action and along with creating the user message, also sets the showLoading flag as true.

case ActionTypes.CREATE_MESSAGE: {

  let message = action.message;
  _messages[message.id] = message;
  _showLoading = true;
  MessageStore.emitChange();
  
  break;
}

The showLoading flag is used in MessageSection to display the loading gif. We are using a saved gif to display the loading symbol. The loading gif is displayed at the end after displaying all the messages in the message store. Since this loading component must be displayed for every user message, we don’t save this component in MessageStore as a loading message as that would lead to repeated looping thorugh the messages in message store to add and delete loading component.

import loadingGIF from '../../images/loading.gif';

function getLoadingGIF() {

  let messageContainerClasses = 'message-container SUSI';

  const LoadingComponent = (
    <li className='message-list-item'>
      <section className={messageContainerClasses}>
        <img src={loadingGIF}
          style={{ height: '10px', width: 'auto' }}
          alt='please wait..' />
      </section>
    </li>
  );
  return LoadingComponent;
}

We then use this flag in MessageListItem class to tag the user messages with the clock icons. We used Material UI SVG Icons to display the clock and tick messages. We display these beside the time in the messages.

import ClockIcon from 'material-ui/svg-icons/action/schedule';

statusIndicator = (
  <li className='message-time' style={footerStyle}>
    <ClockIcon style={indicatorStyle}
      color={UserPreferencesStore.getTheme()==='light' ? '#90a4ae' : '#7eaaaf'}/>
  </li>
);

When the response from server is received, the CREATE_SUSI_MESSAGE action is triggered to render the server response. This action is again collected in MessageStore where the `showLoading` boolean flag is reset to false. This event also triggers the state of MessageSection where we are listening to showLoading value from MessageStore, hence triggering changes in MessageSection and accordingly in MessageListItem where showLoading is passed as props, removing the loading gif component and displaying the server response and replacing the clock icon with tick icon on the user message.

case ActionTypes.CREATE_SUSI_MESSAGE: {
  
  let message = action.message;
  MessageStore.resetVoiceForThread(message.threadID);
  _messages[message.id] = message;
  _showLoading = false;
  MessageStore.emitChange();
  
  break;
}

This is how the status indicators were implemented for messages. The complete code can be found at SUSI WebChat Repo.

Resources

Continue ReadingImplementing the Message Response Status Indicators In SUSI WebChat

How SUSI WebChat Implements RSS Action Type

SUSI.AI now has a new action type called RSS. As the name suggests, SUSI is now capable of searching the internet to answer user queries. This web search can be performed either on the client side or the server side. When the web search is to be performed on the client side, it is denoted by websearch action type. When the web search is performed by the server itself, it is denoted by rss action type. The server searches the internet and using RSS feeds, returns an array of objects containing :

  • Title
  • Description
  • Link
  • Count

Each object is displayed as a result tile and all the results are rendered as swipeable tiles.

Lets visit SUSI WebChat and try it out.

Query : Google
Response: API response

SUSI WebChat uses the same code abstraction to render websearch and rss results as both are results of websearch, only difference being where the search is being performed i.e client side or server side.

How does the client know that it is a rss action type response?

"actions": [
  {
    "type": "answer",
    "expression": "I found this on the web:"
  },
  {
    "type": "rss",
    "title": "title",
    "description": "description",
    "link": "link",
    "count": 3
  }
],

The actions attribute in the JSON API response has information about the action type and the keys to be parsed for title, link and description.

  • The type attribute tells the action type is rss.
  • The title attribute tells that title for each result is under the key – title for each object in answers[0].data.
  • Similarly keys to be parsed for description and link are description and link respectively.
  • The count attribute tells the client how many results to display.

We then loop through the objects in answers,data[0] and from each object we extract title, description and link.

let rssKeys = Object.assign({}, data.answers[0].actions[index]);

delete rssKeys.type;

let count = -1;

if(rssKeys.hasOwnProperty('count')){
  count = rssKeys.count;
  delete rssKeys.count;
}

let rssTiles = getRSSTiles(rssKeys,data.answers[0].data,count);

We use the count attribute and the length of answers[0].data to fix the number of results to be displayed.

// Fetch RSS data

export function getRSSTiles(rssKeys,rssData,count){

  let parseKeys = Object.keys(rssKeys);
  let rssTiles = [];
  let tilesLimit = rssData.length;

  if(count > -1){
    tilesLimit = Math.min(count,rssData.length);
  }

  for(var i=0; i<tilesLimit; i++){
    let respData = rssData[i];
    let tileData = {};

    parseKeys.forEach((rssKey,j)=>{
      tileData[rssKey] = respData[rssKeys[rssKey]];
    });

    rssTiles.push(tileData);
  }

return rssTiles;

}

We now have our list of objects with the information parsed from the response.We then pass this list to our renderTiles function where each object in the rssTiles array returned from getRSSTiles function is converted into a Paper tile with the title and description and the entire tile is hyperlinked to the given link using Material UI Paper Component and few CSS attributes.

// Draw Tiles for Websearch RSS data

export function drawTiles(tilesData){

let resultTiles = tilesData.map((tile,i) => {

  return(
    <div key={i}>
      <MuiThemeProvider>
        <Paper zDepth={0} className='tile'>
          <a rel='noopener noreferrer'
          href={tile.link} target='_blank'
          className='tile-anchor'>
            {tile.icon &&
            (<div className='tile-img-container'>
               <img src={tile.icon}
               className='tile-img' alt=''/>
             </div>
            )}
            <div className='tile-text'>
              <p className='tile-title'>
                <strong>
                  {processText(tile.title,'websearch-rss')}
                </strong>
              </p>
              {processText(tile.description,'websearch-rss')}
            </div>
          </a>
        </Paper>
      </MuiThemeProvider>
    </div>
  );

});

return resultTiles;
}

The tile title and description is processed for HTML special entities and emojis too using the processText function.

case 'websearch-rss':{

let htmlText = entities.decode(text);
processedText = <Emojify>{htmlText}</Emojify>;
break;

}

We now display our result tiles as a carousel like swipeable display using react-slick. We initialise our slider with few default options specifying the swipe speed and the slider UI.

import Slider from 'react-slick';

// Render Websearch RSS tiles

export function renderTiles(tiles){

  if(tiles.length === 0){
    let noResultFound = 'NO Results Found';
    return(<center>{noResultFound}</center>);
  }

  let resultTiles = drawTiles(tiles);
  
  var settings = {
    speed: 500,
    slidesToShow: 3,
    slidesToScroll: 1,
    swipeToSlide:true,
    swipe:true,
    arrows:false
  };

  return(
    <Slider {...settings}>
      {resultTiles}
    </Slider>
  );
}

We finally add CSS attributes to style our result tile and add overflow for text maintaining standard width for all tiles.We also add some CSS for our carousel display to show multiple tiles instead of one by default. This is done by adding some margin for child components in the slider.

.slick-slide{
  margin: 0 10px;
}

.slick-list{
  max-height: 100px;
}

We finally have our swipeable display of rss data tiles each tile hyperlinked to the source of the data. When the user clicks on a tile, he is redirected to the link in a new window i.e the entire tile is hyperlinked. And when there are no results to display, we show a `NO Results Found` message.

The complete code can be found at SUSI WebChat Repo. Feel free to contribute

Resources

 

Continue ReadingHow SUSI WebChat Implements RSS Action Type

Versioning of SUSI Skills

This is a concept for the management of the skill repository aka The “SUSI Skill CMS.

With SUSI we are building a personal assistant where the users are able to write and edit skills in the easiest way that we can think of. To do that we have to develop a very simple skill language and a very simple skill editor

The skill editor should be done as a ‘wiki’-like content management system (cms). To create the wiki, we follow an API-centric approach. The SUSI server acts as an API server with a web front-end which acts as a client of the API and provides the user interface.

The skill editor will be ‘expert-centric’, an expert is a set of skills. That means if we edit one text file, that text file represents one expert, it may contain several skills which all belong together.

An ‘expert’ is stored within the following ontology:

model  >  group  >  language  >  expert  >  skill

To Implement the CMS wiki system we need versioning with a working AAA System. To implement versioning we used JGit. JGit is an EDL licensed, lightweight, pure Java library implementing the Git version control system.

So I included a Gradle dependency to add JGit to the SUSI Server project.

compile 'org.eclipse.jgit:org.eclipse.jgit:4.6.1.201703071140-r'

Now the task was to execute git commands when the authorised user makes changes in any of the expert. The possible changes in an expert can be

1. Creating an Expert
2. Modifying an existing Expert
3. Deleting an Expert

1. git add <filename>

2. git commit -m “commit message”

Real Example in SUSI Server

This is the code that every servlet shares. It defines the base user role set a URL endpoint to trigger the endpoint

public class ModifyExpertService extends AbstractAPIHandler implements APIHandler {
    @Override
    public String getAPIPath() {
        return "/cms/modifyExpert.json";
    }

This is the part where we do all the processing of the URL parameters and store their versions. This method takes the “Query call” and then extracts the “get” parameters from it.
For the functioning of this service, we need 5 things, “model”, “group”, “language”, “expert” and the “commit message”.

@Override
public ServiceResponse serviceImpl(Query call, HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) {

    String model_name = call.get("model", "general");
    File model = new File(DAO.model_watch_dir, model_name);
    String group_name = call.get("group", "knowledge");
    File group = new File(model, group_name);
    String language_name = call.get("language", "en");
    File language = new File(group, language_name);
    String expert_name = call.get("expert", null);
    File expert = new File(language, expert_name + ".txt");

Then we need to open your SUSI Skill DATA repository and commit the new file in it. Here we call the functions of JGit, which do the work of git add and git commit.

FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repository = null;
try {

    repository = builder.setGitDir((DAO.susi_skill_repo))
            .readEnvironment() // scan environment GIT_* variables
            .findGitDir() // scan up the file system tree
            .build();

    try (Git git = new Git(repository)) {

The code above opens our local git repository and creates an object “git”. Then we perform further operations on “git” object. Now we add our changes to “git”. This is similar to when we run “git add . ”

    git.add()
                .addFilepattern(expert_name)
                .call();

Finally, we commit the changes. This is similar to “git commit -m “message”.

    git.commit()
                .setMessage(commit_message)
                .call();

At last, we return the success object and set the “accepted” value as “true”.

    json.put("accepted", true);
        return new ServiceResponse(json);
    } catch (GitAPIException e) {
        e.printStackTrace();
    }

Resources

JGit documentation : https://eclipse.org/jgit/documentation/

SUSI Server : https://github.com/fossasia/susi_server

 

Continue ReadingVersioning of SUSI Skills

Servlets and Containers in SUSI Server

The core of SUSI Clients is the SUSI server that holds the “intelligence” and “personality” of SUSI AI. SUSI Server is in JAVA and it uses the concepts of Servlets heavily. While implementing server for SUSI, we have used JAVA as the backend language and hence are using servlets and Servlet containers heavily.

The problem that servlets are solving is regarding how to generate dynamic content on every request if the request comes with different parameters.  This is not possible by simply using HTML.

Servlets are widely used for dynamic content and have inbuilt support for HTTP (Hypertext Transfer Protocol).

In this blog post, along with basics of servlets and servlet containers, I am explaininghow to make classes for custom servlets. I am using an example of AbstractAPIHandler class that we have used in SUSI server for clear picture.

Web Server

Before we can understand a  servlet container, we need to understand what is a web server.

A web server uses HTTP protocol to transfer data. When a user types in a URL in a browser or any client, he first sees loading and then see the content of the page. So actually there is a server that is sending the content of the page to the user. The transfer of data is in HTTP protocol.

Servlet Container

From the example above, a user can request static pages from the server. Hence the pages will be very basic and we cannot do much with user inputs and dynamic data such as making a real time chat bot. So the idea of having Servlet container is using Java to dynamically generate the content of a web page from the server side.  

The flow of Control

The servlet container manages servlets through servlet life cycles. The container calls a servlet as soon as it receives an HTTP request. Then the servlet does the processing of the data which it has received from the request. the container sends the HTTP response back to the client and then the clients render it to show the final output.

The flowchart below explains how a request is processed from a browser to a servlet container and then to a servlet. It shows the usage of a generic Servlet.

Similarly in SUSI-Server we have a “AbstractAPIHandler” which is a generic servlet that every other servlet inherits.

Structure of a Servlet

public class ServletName extends HttpServlet {

public void init() throws ServletException {
// Servlet Initialization
}

protected void service(HttpServletRequest req,
HttpServletResponse resp)
throws ServletException, IOException {
// Code for the Service Method
}
/**
* Process a GET request
*/
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws IOException, ServletException {
// Code for the doGet() method
}
/**
* Process a POST request
*/
protected void doPost(HttpServletRequest request,
HttpServletResponse response)
throws IOException, ServletException {
// Code for the doPost() method

}
/**
* Process a PUT request
*/
protected void doPut(HttpServletRequest req,
HttpServletResponse resp)
throws ServletException, IOException {
//Code for the doPut() method

}
}

Abstract API Handler

public abstract class AbstractAPIHandler extends HttpServlet implements APIHandler {
}

The function below sets the Minimum Base Role of the SUSI user. We can set which servlet is accessible by which user.

@Override
public abstract BaseUserRole getMinimalBaseUserRole();

The function below gets the Default Permissions of a user Role in SUSI Server. The permissions for each SUSI user can be different.

@Override
public abstract JSONObject getDefaultPermissions(BaseUserRole baseUserRole);

The function below finally sets whatever output we want to show as a response when we query the SUSI Server.

   public abstract ServiceResponse serviceImpl(Query post,  HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) throws APIException;

Each Servlet in SUSI Server extends this class (AbstractAPIHandler.java). This also increases code reusability.

A Servlet is not a just a simple java class. You cannot run a servlet using javac or by running main() function. In order to run a servlet, you have to deploy this Servlet on a web server.

References

SUSI Server : https://github.com/fossasia/susi_server/

Docs: http://docs.oracle.com/javaee/6/tutorial/doc/bnafd.html

Continue ReadingServlets and Containers in SUSI Server

Creating and Maintaining User Sessions Using Universal-Cookies in SUSI Web Chat

If you login to SUSI Web Chat, and come back again after some days, you find that you didn’t have to login and all your previous sent messages are in there in the message pane. To achieve this, SUSI Web Chat uses cookies stored in your browser which is featured in this blog.  

In ReactJS, it’s highly misleading and extensive to use the conventional Javascript methodology of saving tokens and deleting them. However, universal-cookie, a node package allows you to store and get cookies with the least possible confusion. In the following examples, I have made use of the get, set and remove functions of the universal-cookie package which have documentations easily available at this link. The basic problem one finds while setting cookies and maintaining sessions is the time through which it should be valid and to secure the routes. We shall look at the steps below to figure out how was it implemented in SUSI Web Chat.

1. The first step is to install the packages by using the following command in your project’s root-

npm install universal-cookie --save

2. Import the Cookies Class, where-ever you want to create a Cookie object in your repository.

import Cookies from 'universal-cookie';

Create a Cookie object at the same time in the file you want to use it,

const cookies = new Cookies();

3. We make use of the set function of the package first, where we try to set the cookie while the User is trying to login to the account.

Note – The cookie value can be set to any value one wants, however, here I am setting it to the access token which is generated by the server so that I can access it throughout the application.

$.ajax({ options: options,
        success: function (response) {
//Get the response token generated from the server
                let accessToken = response.access_token;                       // store the current state
                 let state = this.state;
// set the time for which the session needs to be valid
            let time = response.valid_seconds;
//set the access token in the state
             state.accessToken = accessToken;
// set the time in the state
             state.time = time;           
// Pass the accessToken and the time through the binded function
             this.handleOnSubmit(accessToken, time);
            }.bind(this),
        error: function ( jqXHR, textStatus, errorThrown) {
                   // Handle errors
                   }
        });

Function –  handleOnSubmit()

// Receive the accessToken and the time for which it needs to be valid
handleOnSubmit = (loggedIn, time) => {
        let state = this.state;
        if (state.success) {
              // set the cookie of with the value of the access token at path ‘/’ and set the time using the parameter ‘maxAge’
            cookies.set('loggedIn', loggedIn, { path: '/', maxAge: time });
// Redirect the user to logged in state and reload
            this.props.history.push('/', { showLogin: false });
            window.location.reload();
        }
        else {
        // Handle errors
    }
}

4.  To access the value set to the cookie, we make use of the get function. To check the logged in state of the User we check if get method is returning a null value or an undefined value, this helps in maintaining the User behaviour at every point in the application.

if(cookies.get('loggedIn')===null||
    cookies.get('loggedIn')===undefined) {
    // Handle User behaviours do not send chat queries with access token if the cookie is null
    url = BASE_URL+'/susi/chat.json?q='+
          createdMessage.text+
          '&language='+locale;
  }
  else{
   //  Send the messages with User’s access token
    url = BASE_URL+'/susi/chat.json?q='
          +createdMessage.text+'&language='
          +locale+'&access_token='
          +cookies.get('loggedIn');
  }

5. To delete the cookies, we make use of the remove function, which deletes that cookie. This function is called while logging the user out of the application.

cookies.remove('loggedIn');
this.props.history.push('/');
window.location.reload();

Here’s the full code in the repository. Feel free to contribute:https://github.com/fossasia/chat.susi.ai

Resources

Continue ReadingCreating and Maintaining User Sessions Using Universal-Cookies in SUSI Web Chat

Using SUSI AI Accounting Object to Write User Settings

SUSI Server uses DAO in which accounting object is stored as JSONTray. SUSI clients are using this accounting object for user settings data. In this blogpost we will focus on how to use accounting JSONTray to write the user settings, so that a client can use such endpoint to store the user related settings in Susi server. The Susi server provides the required API endpoints to its web and mobile clients. Before starting with the implementation of servlet let’s take a look at Accounting.java file, to check how Susi server stores the accounting data.

public class Accounting {

        private JsonTray parent;
        private JSONObject json;
        private UserRequests requests;
        private ClientIdentity identity;
    ...
}

 

The JsonTray is class to hold the volume as <String,JsonObject> pairs as a Json file. The UserRequests  class holds all the user activities. The ClientIdentity class extend the base class Client and provides an Identification String for authentication of users. Now that we have understood about accounting in SUSI server let’s proceed for making an API endpoint to Store Webclient User settings. To make an endpoint we will use the HttpServlet class which provides methods, such as doGet and doPost, for handling HTTP-specific services. We will inherit our ChangeUserSettings class from AbstractAPIHandler yand implement APIhandler interface. In Susi Server the AbsrtactAPI handler extends a HTTPServlet which implements doGet and doPost ,all servlet in SUSI Server extends this class to increase code reusability.  

Since a User has to store its setting, set the minimum base role to access this endpoint to User. Apart from ‘User’ there are Admin and Anonymous roles too.

   @Override
    public BaseUserRole getMinimalBaseUserRole() {
        return BaseUserRole.USER;
    }

Next set the path for using this endpoint, by overriding getAPIPath method().

 @Override
    public String getAPIPath() {
        return "/aaa/changeUserSettings.json";
    }

We won’t be dealing with getdefault permissions so null can be return.

  @Override
    public JSONObject getDefaultPermissions(BaseUserRole baseUserRole) {
        return null;
    }

Next we implement serviceImpl method which takes four parameters the query, response, authorization and default permissions.

@Override
    public ServiceResponse serviceImpl(Query query, HttpServletResponse response, Authorization authorization, JsonObjectWithDefault permissions) throws APIException {
       String key = query.get("key", null);
       String value =query.get("value", null);
       if (key == null || value == null ) {
           throw new APIException(400, "Bad Service call, key or value parameters not provided");
       } else {
           if (authorization.getIdentity() == null) {
               throw new APIException(400, "Specified User Setting not found, ensure you are logged in");
           } else {
               Accounting accounting = DAO.getAccounting(authorization.getIdentity());
               JSONObject jsonObject = new JSONObject();
               jsonObject.put(key, value);
               if (accounting.getJSON().has("settings")) {
                   accounting.getJSON().getJSONObject("settings").put(key, value);
               } else {
                   accounting.getJSON().put("settings", jsonObject);
               }
               JSONObject result = new JSONObject();
               result.put("message", "You successfully changed settings to your account!");
               return new ServiceResponse(result);
           }
       }

    }

We will be storing the setting in Json object using key, value pairs. Take the values from user using query.get(“param”,”default value”) and set the default value to null. So that in case the parameters are not present the servlet can return “Bad service call”. To get the accounting object user identity string given by authorization.getIdentity() method is used. Now check if the same user settings is already present, if yes, overwrite it and if not append a new Json object with received key and value. And return the success message through ServiceResponse method.

Proceed to test the working of endpoint at http://127.0.0.1:4000/aaa/changeUserSettings.json?key=theme&value=dark and see if it’s stored using   http://127.0.0.1:4000/aaa/listUserSettings.json.

You have successfully created an endpoint to store user settings and  enhanced Susi Server, take a look and contribute to Susi Server.

Resources

Continue ReadingUsing SUSI AI Accounting Object to Write User Settings

Implementing Search Feature In SUSI Web Chat

SUSI WebChat now has a search feature. Users now have an option to filter or find messages. The user can enter a keyword or phrase in the search field and all the matched messages are highlighted with the given keyword and the user can then navigate through the results.

Lets visit SUSI WebChat and try it out.

  1. Clicking on the search icon on the top right corner of the chat app screen, we’ll see a search field expand to the left from the search icon.
  2. Type any word or phrase and you see that all the matches are highlighted in yellow and the currently focused message is highlighted in orange
  3. We can use the up and down arrows to navigate between previous and recent messages containing the search string.
  4. We can also choose to search case sensitively using the drop down provided by clicking on the vertical dots icon to the right of the search component.
  5. Click on the `X` icon or the search icon to exit from the search mode. We again see that the search field contracts to the right, back to its initial state as a search icon.

How does the search feature work?

We first make our search component with a search field, navigation arrow icon buttons and exit icon button. We then listen to input changes in our search field using onChange function, and on input change, we collect the search string and iterate through all the existing messages checking if the message contains the search string or not, and if present, we mark that message before passing it to MessageListItem to render the message.

let match = msgText.indexOf(matchString);

  if (match !== -1) {
    msgCopy.mark = {
    matchText: matchString,
    isCaseSensitive: isCaseSensitive
  };

}

We alse need to pass the message ID of the currently focused message to MessageListItem as we need to identify that message to highlight it in orange instead of yellow differentiating between all matches and the current match.

function getMessageListItem(messages, markID) {
  if(markID){
    return messages.map((message) => {
      return (
        <MessageListItem
          key={message.id}
          message={message}
          markID={markID}
        />
      );
    });
  }
}

We also store the indices of the messages marked in the MessageSection Component state which is later used to iterate through the highlighted results.

searchTextChanged = (event) => {

  let matchString = event.target.value;
  let messages = this.state.messages;
  let markingData = searchMsgs(messages, matchString,
    this.state.searchState.caseSensitive);

  if(matchString){

    let searchState = {
      markedMsgs: markingData.allmsgs,
      markedIDs: markingData.markedIDs,
      markedIndices: markingData.markedIndices,
      scrollLimit: markingData.markedIDs.length,
      scrollIndex: 0,
      scrollID: markingData.markedIDs[0],
      caseSensitive: this.state.searchState.caseSensitive,
      open: false,
      searchText: matchString
    };

    this.setState({
      searchState: searchState
    });

  }
}

After marking the matched messages with the search string, we pass the messages array into MessageListItem Component where the messages are processed and rendered. Here, we check if the message being received from MessageSection is marked or not and if marked, we then highlight the message. To highlight all occurrences of the search string in the message text, I used a module called react-text-highlight.

import TextHighlight from 'react-text-highlight';

if(this.props.message.id === markMsgID){
  markedText.push(
    <TextHighlight
      key={key}
      highlight={matchString}
      text={part}
      markTag='em'
      caseSensitive={isCaseSensitive}
    />
  );
}
else{
  markedText.push(
    <TextHighlight
      key={key}
      highlight={matchString}
      text={part}
      caseSensitive={isCaseSensitive}/>
  );
}

Here, we are using the message ID of the currently focused message, sent as props to MessageListItem to identify the currently focused message and highlight it specifically in orange instead of the default yellow color for all other matches.

I used ‘em’ tag to emphasise the currently highlighted message and colored it orange using CSS attributes.

em{
  background-color: orange;
}

We next need to add functionality to navigate through the matched results. The arrow buttons are used to navigate. We stored all the marked messages in the MessageSection state as `markedIDs` and their corresponding indices as `markedIndices`. Using the length of this array, we get the `scrollLimit` i.e we know the bounds to apply while navigating through the search results.

On clicking the up or down arrows, we update the currently highlighted message through `scrollID` and `scrollIndex`, and also check for bounds using `scrollLimit`  in the searchState. Once these are updated, the chat app must automatically scroll to the new currently highlighted message. Since findDOMNode is being deprecated, I used the custom scrollbar to find the node of the currently highlighted message without using findDOMNode. The custom scrollbar was implemented using the module react-custom-scrollbars. Once the node is found, we use the inbuilt HTML DOM method, scrollIntoView()  to automatically scroll to that message.

if(this.state.search){
  if (this.state.searchState.scrollIndex === -1
      || this.state.searchState.scrollIndex === null) {
      this._scrollToBottom();
  }
  else {
    let markedIDs = this.state.searchState.markedIDs;
    let markedIndices = this.state.searchState.markedIndices;
    let limit = this.state.searchState.scrollLimit;
    let ul = this.messageList;

    if (markedIDs && ul && limit > 0) {
      let currentID = markedIndices[this.state.searchState.scrollIndex];
      this.scrollarea.view.childNodes[currentID].scrollIntoView();
    }
  }
}

Let us now see how the search field was animated. I used a CSS transition property along width to get the search field animation to work. This gives the animation when there is a change of width for the search field. I fixed the width to be zero when the search mode is not activated, so only the search icon is displayed. When the search mode is activated i.e the user clicks on the search field, I fixed the width as 125px. Since the width has changed, the increase in width is displayed as an expanding animation due to the CSS transition property.

const animationStyle = {
  transition: 'width 0.75s cubic-bezier(0.000, 0.795, 0.000, 1.000)'
};

const baseStyles = {
  open: { width: 125 },
  closed: { width: 0 },
}

We also have a case sensitive option which is displayed on clicking the rightmost button i.e the three vertical dots button. We can toggle between case sensitive option, whose value is stored in MessageSection searchState and is passed along with the messages to MessageListItem where it is used by react-text-highlight to highlight text accordingly and render the highlighted messages.

This is how the search feature was implemented in SUSI WebChat. You can find the complete code at SUSI WebChat.

Resources
Continue ReadingImplementing Search Feature In SUSI Web Chat

Processing Text Responses in SUSI Web Chat

SUSI Web Chat client now supports emojis, images, links and special characters. However, these aren’t declared as separate action types i.e the server doesn’t explicitly tell the client that the response contains any of the above features when it sends the JSON response. So the client must parse the text response from server and add support for each of the above mentioned features instead of rendering the plain text as is, to ensure good UX.

SUSI Web Chat client parses the text responses to support :

  • HTML Special Entities
  • Images and GIFs
  • URLs and Mail IDs
  • Emojis and Symbols
// Proccess the text for HTML Spl Chars, Images, Links and Emojis

function processText(text){

  if(text){
    let htmlText = entities.decode(text);
    let imgText = imageParse(htmlText);
    let replacedText = parseAndReplace(imgText);

    return <Emojify>{replacedText}</Emojify>;

  };
  return text;
}

Let us write sample skills to test these out. Visit http://dream.susi.ai/ and enter textprocessing.

You can then see few sample queries and responses at http://dream.susi.ai/p/textprocessing.

Lets visit SUSI WebChat and try it out.

Query : dream textprocessing

Response: dreaming enabled for textprocessing

Query : text with special characters

Response:  &para; Here are few “Special Characters&rdquo;!

All the special entities notations have been parsed and rendered accordingly!

Sometimes we might need to use HTML special characters due to reasons like

  • You need to escape HTML special characters like <, &, or .
  • Your keyboard does not support the required character. For example, many keyboards do not have em-dash or the copyright symbol.

You might be wondering why the client needs to handle this separately as it is generally, automatically converted to relevant HTML character while rendering the HTML. SUSI Web Chat client uses reactjs which has JSX and not HTML. So JSX doesn’t support HTML special characters i.e they aren’t automatically converted to relevant characters while rendering. Hence, the client needs to handle this explicitly.

We used the module, html-entities to decode all types of special HTML characters and entities. This module parses the text for HTML entities and replaces them with the relevant character for rendering when used to decode text.

import {AllHtmlEntities} from 'html-entities';
const entities = new AllHtmlEntities();

let htmlText = entities.decode(text);

Now that the HTML entities are processed, the client then processes the text for image links. Let us now look at how images and gifs are handled.

Query : random gif

Response: https://media1.giphy.com/media/AAKZ9onKpXog8/200.gif

Sometimes, the text contains links for images or gifs and the user would be expecting a media type like image or gif instead of text. So we need to replace those image links with actual images to ensure good UX. This is handled using regular expressions to match image type urls and correspondingly replace them with html img tags so that the response is a image and not URL text.

// Parse text for Image URLs

function imageParse(stringWithLinks){

  let replacePattern = new RegExp([
    '((?:https?:\\/\\/)(?:[a-zA-Z]{1}',
    '(?:[\\w-]+\\.)+(?:[\\w]{2,5}))',
    '(?::[\\d]{1,5})?\\/(?:[^\\s/]+\\/)',
    '*(?:[^\\s]+\\.(?:jpe?g|gif|png))',
    '(?:\\?\\w+=\\w+(?:&\\w+=\\w+)*)?)'
  ].join(''),'gim');

  let splits = stringWithLinks.split(replacePattern);

  let result = [];

  splits.forEach((item,key)=>{
    let checkmatch = item.match(replacePattern);

    if(checkmatch){
      result.push(
        <img key={key} src={checkmatch}
        style={{width:'95%',height:'auto'}} alt=''/>)
    }
    else{
      result.push(item);
    }
  });

  return result;
}

The text is split using the regular expression and every matched part is replaced with the corresponding image using the img tag with source as the URL contained in the text.

The client then parses URLs and Mail IDs.

Query: search internet

Response: Internet The global system of interconnected computer networks that use the Internet protocol suite to… https://duckduckgo.com/Internet

The link has been parsed from the response text and has been successfully hyperlinked. Clicking the links opens the respective url in a new window.

We used react-linkify module to parse links and email IDs. The module parses the text and hyperlinks all kinds of URLs and Mail IDs.

import Linkify from 'react-linkify';

export const parseAndReplace = (text) => {return <Linkify properties={{target:"_blank"}}>{text}</Linkify>;}

Finally, let us see, how emojis are parsed.

Query : dream textprocessing

Response: dreaming enabled for textprocessing

Query : susi, do you use emojis?

Response: Ofcourse ^__^ 😎 What about you!? 😉 😛

All the notations for emojis have been parsed and rendered as emojis instead of text!

We used react-emojine module to emojify the text.

import Emojify from 'react-emojione';

<Emojify>{text}</Emojify>;

This is how text is processed to support special characters, images, links and emojis, ensuring a rich user experience. You can find the complete code at SUSI WebChat.

Resources

Continue ReadingProcessing Text Responses in SUSI Web Chat