Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners

SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop.

Any electron app essentially comprises of the following components

    • Main Process (Managing windows and other interactions with the operating system)
    • Renderer Process (Manage the view inside the BrowserWindow)

Steps to setup development environment

      • Clone the repo locally.
$ git clone https://github.com/fossasia/susi_desktop.git
$ cd susi_desktop
      • Install the dependencies listed in package.json file.
$ npm install
      • Start the app using the start script.
$ npm start

Structure of the project

The project was restructured to ensure that the working environment of the Main and Renderer processes are separate which makes the codebase easier to read and debug, this is how the current project is structured.

The root directory of the project contains another directory ‘app’ which contains our electron application. Then we have a package.json which contains the information about the project and the modules required for building the project and then there are other github helper files.

Inside the app directory-

  • Main – Files for managing the main process of the app
  • Renderer – Files for managing the renderer process of the app
  • Resources – Icons for the app and the tray/media files
  • Webview Tag

    Display external web content in an isolated frame and process, this is used to load chat.susi.ai in a BrowserWindow as

    <webview src="https://chat.susi.ai/"></webview>
    

    Adding event listeners to the app

    Various electron APIs were used to give a native feel to the application.

  • Send focus to the window WebContents on focussing the app window.
  • win.on('focus', () => {
    	win.webContents.send('focus');
    });
    
  • Display the window only once the DOM has completely loaded.
  • const page = mainWindow.webContents;
    ...
    page.on('dom-ready', () => {
    	mainWindow.show();
    });
    
  • Display the window on ‘ready-to-show’ event
  • win.once('ready-to-show', () => {
    	win.show();
    });
    

    Resources

    1. A quick article to understand electron’s main and renderer process by Cameron Nokes at Medium link
    2. Official documentation about the webview tag at https://electron.atom.io/docs/api/webview-tag/
    3. Read more about electron processes at https://electronjs.org/docs/glossary#process
    4. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.

    Link Preview Holder on SUSI.AI Android Chat

    SUSI Android contains several view holders which binds a view based on its type, and one of them is LinkPreviewHolder. As the name suggests it is used for previewing links in the chat window. As soon as it receives an input as of link it inflates a link preview layout. The problem which exists was that whenever a user inputs a link as an input to app, it crashed. It crashed because it tries to inflate component that doesn’t exists in the view that is given to ViewHolder. So it gave a Null pointer Exception, due to which the app crashed. The work around for fixing this bug was that based on the type of user it will inflate the layout and its components. Let’s see how all functionalities were implemented in the LinkPreviewHolder class.

    Components of LinkPreviewHolder

    @BindView(R.id.text)
    public TextView text;
    @BindView(R.id.background_layout)
    public LinearLayout backgroundLayout;
    @BindView(R.id.link_preview_image)
    public ImageView previewImageView;
    @BindView(R.id.link_preview_title)
    public TextView titleTextView;
    @BindView(R.id.link_preview_description)
    public TextView descriptionTextView;
    @BindView(R.id.timestamp)
    public TextView timestampTextView;
    @BindView(R.id.preview_layout)
    public LinearLayout previewLayout;
    @Nullable @BindView(R.id.received_tick)
    public ImageView receivedTick;
    @Nullable
    @BindView(R.id.thumbs_up)
    protected ImageView thumbsUp;
    @Nullable
    @BindView(R.id.thumbs_down)
    protected ImageView thumbsDown;

    Currently in this it binds the view components with the associated id using declarator @BindView(id)

    Instantiates the class with a constructor

    public LinkPreviewViewHolder(View itemView , ClickListener listener) {
       super(itemView, listener);
       realm = Realm.getDefaultInstance();
       ButterKnife.bind(this,itemView);
    }

    Here it binds the current class with the view passed in the constructor using ButterKnife and initiates the ClickListener.

    Now it is to set the components described above in the setView function:

    Spanned answerText;
    text.setLinksClickable(true);
    text.setMovementMethod(LinkMovementMethod.getInstance());
    if (android.os.Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
    answerText = Html.fromHtml(model.getContent(), Html.FROM_HTML_MODE_COMPACT);
    } else {
    answerText = Html.fromHtml(model.getContent());
    }

    Sets the textView inside the view with a clickable link. Version checking also has been put for checking the version of Android (Above Nougat or not) and implement the function accordingly.

    This ViewHolder will inflate different components based on the thing that who has requested the output. If the query wants to inflate the LinkPreviewHolder them some extra set of components will get inflated which need not be inflated for the response apart from the basic layout.

    if (viewType == USER_WITHLINK) {
       if (model.getIsDelivered())
           receivedTick.setImageResource(R.drawable.ic_check);
       else
           receivedTick.setImageResource(R.drawable.ic_clock);
    }

    In the above code  received tick image resource is set according to the attribute of message is delivered or not for the Query sent by the user. These components will only get initialised when the user has sent some links.

    Now comes the configuration for the result obtained from the query.  Every skill has some rating associated to it. To mark the ratings there needs to be a counter set for rating the skills, positive or negative. This code should only execute for the response and not for the query part. This is the reason for crashing of the app because the logic tries to inflate the contents of the part of response but the view that is passed belongs to query. So it gives NullPointerException there, so there is a need to separate the logic of Response from the Query.

    if (viewType != USER_WITHLINK) {
       if(model.getSkillLocation().isEmpty()){
           thumbsUp.setVisibility(View.GONE);
           thumbsDown.setVisibility(View.GONE);
       } else {
           thumbsUp.setVisibility(View.VISIBLE);
           thumbsDown.setVisibility(View.VISIBLE);
       }
    
       if(model.isPositiveRated()){
           thumbsUp.setImageResource(R.drawable.thumbs_up_solid);
       } else {
           thumbsUp.setImageResource(R.drawable.thumbs_up_outline);
       }
    
       if(model.isNegativeRated()){
           thumbsDown.setImageResource(R.drawable.thumbs_down_solid);
       } else {
           thumbsDown.setImageResource(R.drawable.thumbs_down_outline);
       }
    
       thumbsUp.setOnClickListener(new View.OnClickListener() {
           @Override
           public void onClick(View view) { . . . }
       });
    
    
    
       thumbsDown.setOnClickListener(new View.OnClickListener() {
           @Override
           public void onClick(View view) { . . . }
       });
    
    }

    As you can see in the above code  it inflates the rating components (thumbsUp and thumbsDown) for the view of the SUSI.AI response and set on the clickListeners for the rating buttons. Them in the below code it previews the link and commit the data using Realm in the database through WebLink class.

    LinkPreviewCallback linkPreviewCallback = new LinkPreviewCallback() {
       @Override
       public void onPre() { . . . }
    
       @Override
       public void onPos(final SourceContent sourceContent, boolean b) { . . . }
    }

    This method calls the api and set the rating of that skill on the server. On successful result it made the thumb Icon change and alter the rating method and commit those changes in the databases using Realm.

    private void rateSusiSkill(final String polarity, String locationUrl, final Context context) {..}

    References

    SUSI.AI Chrome Bot and Web Speech: Integrating Speech Synthesis and Recognition

    Susi Chrome Bot is a Chrome extension which is used to communicate with Susi AI. The advantage of having chrome extensions is that they are very accessible for the user to perform certain tasks which sometimes needs the user to move to another tab/site.

    In this blog post, we will be going through the process of integrating the web speech API to SUSI Chromebot.

    Web Speech API

    Web Speech API enables web apps to be able to use voice data. The Web Speech API has two components:

    Speech Recognition:  Speech recognition gives web apps the ability to recognize voice data from an audio source. Speech recognition provides the speech-to-text service.

    Speech Synthesis: Speech synthesis provides the text-to-speech services for the web apps.

    Integrating speech synthesis and speech recognition in SUSI Chromebot

    Chrome provides the webkitSpeechRecognition() interface which we will use for our speech recognition tasks.

    var recognition = new webkitSpeechRecognition();
    

     

    Now, we have a speech recognition instance recognition. Let us define necessary checks for error detection and resetting the recognizer.

    var recognizing;
    
    function reset() {
    recognizing = false;
    }
    
    recognition.onerror = function(e){
    console.log(e.error);
    };
    
    recognition.onend = function(){
    reset();
    };
    

     

    We now define the toggleStartStop() function that will check if recognition is already being performed in which case it will stop recognition and reset the recognizer, otherwise, it will start recognition.

    function toggleStartStop() {
        if (recognizing) {
          recognition.stop();
          reset();
        } else {
          recognition.start();
          recognizing = true;
        }
    }
    

     

    We can then attach an event listener to a mic button which calls the toggleStartStop() function to start or stop our speech recognition.

    mic.addEventListener("click", function () {
        toggleStartStop();
    });
    

     

    Finally, when the speech recognizer has some results it calls the onresult event handler. We’ll use this event handler to catch the results returned.

    recognition.onresult = function (event) {
        for (var i = event.resultIndex; i < event.results.length; ++i) {
          if (event.results[i].isFinal) {
            textarea.value = event.results[i][0].transcript;
            submitForm();
          }
        }
    };
    

     

    The above code snipped tests for the results produced by the speech recognizer and if it’s the final result then it sets textarea value with the result of speech recognition and then we submit that to the backend.

    One problem that we might face is the extension not being able to access the microphone. This can be resolved by asking for microphone access from an external tab/window/iframe. For SUSI Chromebot this is being done using an external tab. Pressing on the settings icon makes a new tab which then asks for microphone access from the user. This needs to be done only once, so that does not cause a lot of trouble.

    setting.addEventListener("click", function () {
    chrome.tabs.create({
    url: chrome.runtime.getURL("options.html")
    });
    });navigator.webkitGetUserMedia({
    audio: true
    }, function(stream) {
    stream.stop();
    }, function () {
    console.log('no access');
    });
    

     

    In contrast to speech recognition, speech synthesis is very easy to implement.

    function speakOutput(msg){
        var voiceMsg = new SpeechSynthesisUtterance(msg);
        window.speechSynthesis.speak(voiceMsg);
    }
    

     

    This function takes a message as input, declares a new SpeechSynthesisUtterance instance and then calls the speak method to convert the text message to voice.

    There are many properties and attributes that come with this speech recognition and synthesis interface. This blog post only introduces the very basics.

    Resources

     

    Enhancing SUSI Desktop to Display a Loading Animation and Auto-Hide Menu Bar by Default

    SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop. The benefits of using chat.susi.ai as a submodule is that it inherits all the features that the webapp offers and thus serves them in a nicely build native application.

    Display a loading animation during DOM load.

    Electron apps should give a native feel, rather than feeling like they are just rendering some DOM, it would be great if we display a loading animation while the web content is actually loading, as depicted in the gif below is how I implemented that.
    Electron provides a nice, easy to use API for handling BrowserWindow, WebContent events. I read through the official docs and came up with a simple solution for this, as depicted in the below snippet.

    onload = function () {
    	const webview = document.querySelector('webview');
    	const loading = document.querySelector('#loading');
    
    	function onStopLoad() {
    		loading.classList.add('hide');
    	}
    
    	function onStartLoad() {
    		loading.classList.remove('hide');
    	}
    
    	webview.addEventListener('did-stop-loading', onStopLoad);
    	webview.addEventListener('did-start-loading', onStartLoad);
    };
    

    Hiding menu bar as default

    Menu bars are useful, but are annoying since they take up space in main window, so I hid them by default and users can toggle their display on pressing the Alt key at any point of time, I used the autoHideMenuBar property of BrowserWindow class while creating an object to achieve this.

    const win = new BrowserWindow({
    	
    	show: false,
    	autoHideMenuBar: true
    });
    

    Resources

    1. More information about BrowserWindow class in the official documentation at electron.atom.io.
    2. Follow a quick tutorial to kickstart creating apps with electron at https://www.youtube.com/watch?v=jKzBJAowmGg.
    3. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.

    Showing “Get started” button in SUSI Viber bot

    When we start a chat with SUSI.AI on Viber i.e. SUSI Viberbot, there should be an option on how to get started with the bot. The response to it are some options like “Visit repository”, “How to contribute” which direct the user to check how SUSI.AI bot is made and prompts him/her to contribute to it. Along with that an option of “start chatting” can be shown to add up some sample queries for the user to try.

    To accomplish the task at hand, we will accomplish these sub tasks:

    1. To show the “Get started” button.
    2. To show the reply to “Get started” query.
    3. To respond to the queries, nested in the response of “Get started”

    Showing “Get started”:

    The Viber developers platform notifies us when a user starts a conversation with our bot. To be exact, a conversation_started event is sent to our webhook and can be handled accordingly. The Viberbot shows a welcome message to the user along with a Get started button to help him/her start.

    To send just the welcome message:

    if (req.body.event === 'conversation_started') {
           // Welcome Message
           var options = {
               method: 'POST',
               url: 'https://chatapi.viber.com/pa/send_message',
               headers: headerBody,
               body: {
                   // some required body properties here
                   text: 'Welcome to SUSI.AI!, ' + req.body.user.name + '.',
                   // code for showing the get started button here.
            }
               json: true
           };
     
           request(options, function(error, res, body) {
               // handle error
           });
       }

    The next step is to show the “Get started” button. To show that we use a keyboard tool, provided by Viber developers platform. So after the “text” key we have the “keyboard” key and a value for it:

    keyboard: {
                 "Type": "keyboard",
                 "DefaultHeight": true,
                 "Buttons": [{
                     "ActionType": "reply",
                     "ActionBody": "Get started",
                 }]
             }

    The action type as shown in the code, can be “reply” or “open-url”. The “reply” action type, triggers an automatic query sent back with “Get started” (i.e. the value of “ActionBody” key), when that button gets clicked.

    Hence, this code helps us tackle our first sub task:

    Reply to “Get started”:

    We target to make each SUSI.AI bot generic. The SUSI FBbot and SUSI Tweetbot shows some options like “Visit repository”, “Start chatting” and “How to contribute?” for the “Get started” query. We render the same answer structure in Viberbot.

    The “rich_media” type helps us send buttons in our reply message. As we ought to use three buttons in our message, the button rows are three in the body object:

    if(message === "Get started"){
                       var options = {
                           method: 'POST',
                           url: 'https://chatapi.viber.com/pa/send_message',
                           headers: headerBody,
                           body: {
                               // some body object properties here
                               type: 'rich_media',
                               rich_media: {
                                   Type: "rich_media",
                                   ButtonsGroupColumns: 6,
                                   ButtonsGroupRows: 3,
                                   BgColor: "#FFFFFF",
                                   Buttons: buttons
                               }
                           },
                           json: true
                       };
     
                       request(options, function(error, res, body) {
                           if (error) throw new Error(error);
                           console.log(body);
                       });

    As said before, 2 type of Action types are available – “open-url” and “reply”. “Visit repository” button has an “open-url” action type and “How to contribute?” or “start chatting” has a “reply” action type.

    Example of “Visit repository” button:

    var buttons = [{
                    Columns: 6,
                    Rows: 1,
                    Text: "Visit repository",
                    "ActionType": "open-url",
                    "ActionBody": "https://www.github.com/fossasia/susi_server",
                    // some text styling properties here
                 }];

    To respond to the “reply” action type queries:

    When the “reply” action type button gets clicked, it triggers an automatic query sent back to the bot with the value same as that of the “ActionBody” key. So we just need to apply a check if the message string recieved is “Start chatting” or “How to contribute?”

    For the response to “Start chatting”, we plan to show sample queries for the user to try. This can be shown by using buttons with the action type as “reply”.

    Code snippet to show a button with the text as “What is FOSSASIA?”:

    var buttons = [{
                            Columns: 6,
                            Rows: 1,
                            Text: "What is FOSSASIA? ",
                            "ActionType": "reply",
                            "ActionBody": "What is FOSSASIA?",
                            // text styling here
                        }];

    For the response to “How to contribute”, we show some messages to help the user contribute to SUSI.AI. These messages also just need buttons with it, to be able to apply the necessary action.

    We respond with 2 messages to the user, both the messages have a button.

    For example, a button to visit the SUSI.AI Gitter channel:

    var buttons = [{
                        Columns: 6,
                        Rows: 1,
                           Text: "<font color=#323232><b>Chat on Gitter</b></font>",
                          ActionType: "open-url",
                          ActionBody: "https://www.gitter.im/fossasia/susi_server",
                          // text styling here
                }];

    This way we have successfully added the “Get started” option to our Viberbot and handled all the subsequent steps.

    Resources:

    1. Viber video managing chat extensions by Ingrid Lunden from Tech crunch.
    2. Develop a chat bot with node js by Slobodan Stojanović from smashing magazine.

    Creating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin

    An Android application often includes settings that allow the user to modify features of the app. For example, SUSI Android app allows users to specify whether they want to use in built mic to give speech input or not. Different settings in SUSI Android app and their purpose are given below

    Setting                                        Purpose
    Enter As Send It allows users to specify whether they want to use enter key to send message or to add new line.
    Mic Input It allows users to specify whether they want to use in built mic to give speech input or not.
    Speech Always It allows users to specify whether they want voice output in case of speech input or not.
    Speech Output It allows users to specify whether they want speech output irrespective of input type or not.
    Language It allows users to set different query language.
    Reset Password It allows users to change password.
    Select Server It allows users to specify whether they want to use custom server or not.

    Android provides a powerful framework, Preference framework, that allows us to define the way we want preferences. In this blog post, I will show you how Settings UI is created using Preference framework and Kotlin in SUSI Android.

    Advantages of using Preference are:

    • It has own UI so we don‘t have to develop our own UI for it
    • It stores the string into the SharedPreferences so we don’t need to manage the values in SharedPreference.

    First, we will add the dependency in build.gradle(project) file as shown below.

    compile ‘com.takisoft.fix:preference-v7:25.4.0.3’

    To create the custom style for our Settings Activity screen we can set

    android:theme=“@style/PreferencesThemeLight”

    as the base theme and can apply various other modifications and colour over this. By default, it has the usual Day and Night theme with NoActionBar extension.

    Layout Design

    I used PreferenceScreen as the main container to create UI of Settings and filled it with the other components. Different components used are following:

    • SwitchPreferenceCompat: This gives us the Switch Preference which we can use to toggle between two different modes in the setting.
    <com.takisoft.fix.support.v7.preference.SwitchPreferenceCompat

    android:defaultValue=”true”

    • PreferenceCategory: It is used for grouping the preference. For example, Chat Settings, Mic Settings, Speech Settings etc are different groups in settings.

    • ListPreference: This preference display list of values and help in selecting one. For example in setLanguage option ListPreference is used to show a list of query language. List of query language is provided via xml file array.xml (res/values). Attribute android:entries point to arrays languagentries and android:entryValue holds the corresponding value defined for each of the languages.
    <ListPreference

    android:title=“@string/Language”
    android:key=“Lang_Select”

    android:entries=“@array/languagentries”

    android:entryValues=“@array/languagentry”

    </ListPreference>

    Implementation in SUSI Android

    All the logic related to Preferences and their action is written in ChatSettingsFragment class. ChatSettingsFragment extends PreferenceFragmentCompat class.

    class ChatSettingsFragment : PreferenceFragmentCompat()

    Fragment populate the preferences when created. addPreferencesFromResource method is used to inflate view from xml.

    addPreferencesFromResource(R.xml.pref_settings)

    Reference

    Implement Internationalization in SUSI Android With Weblate

    When you build an Android app, you must consider about users for whom you are building an app. It may be possible that you users are from the different region. To support the most users your app should show text in locale language so that user can use your app easily. Our app SUSI Android is also targeting users from different regions. Internationalization is a way that ensures our app can be adapted to various languages without requiring any change to source code. This also allows projects to collaborate with non-coders more easily and plugin translation tools like Weblate.

    Benefits of using Internationalization are:

    • It reduces the time for localization i.e it will localize your app automatically.
    • It helps us to keep and maintain only single source code for different regions.

    To achieve Internationalization in Android app you must follow below steps:

    • Move all the required contents of your app’s user interface into the resource file.
    • Create new directories inside res to add support for Internationalization. Each directory’s name should follow rule <resource type>-(language code). For example values-es contains string resource for es language code i.e Spanish.
    • Now add different locale content in the respective folder.

    We need to create separate directories for different locale because to show locale specific content, Android check specific folder i.e res/<resource type>-(language code) like res/values-de and show content from that folder. That’s why we need to move all the required content into resource file so that each required content can be shown in the specific locale.

    How Internationalization is implemented in SUSI Android

    In SUSI Android there is not any locale specific image but only string. So I created only locale specific value resource folder to add locale specific strings. To create locale specific values folder I follow the above-mentioned rule i.e <resource type>-(language code).

    After that, I added specific language string in the respective folder.

    Instead of hard-coded strings, we used strings from string.xml file so that it will change automatically according to the region.

    android:text=“@string/reset”

    and

    showToast(getString(R.string.wrong_password))

    In absence of resource directory for any specific locale, Android use default resource directory.

    Integrate Weblate in SUSI Android

    Weblate is a web based translation tool. The best part of Weblate is its tight version control integration which makes it easy for translators to contribute because translator does not need to fork your repo and send pull request for each change but Weblate handle this part i.e translator translate strings of the project in Weblate site and Weblate will send pull request for those changes.

    Weblate can host your free software projects for free but it depends on them. Here is the link of SUSI Android project hosted on Weblate. If your project is good then they can host your project for free. But for that, you have to apply from this link and select ask for hosting. Now fill up form as shown in below picture.

    Once your project is hosted on Weblate, they will email you about it. After that, you have to integrate Weblate in your project so that Weblate can automatically push translated strings to your project and also Weblate get notify about changes in your repository. Here is the link on how to add Weblate service and Weblate user to your project.

    If it is not possible to host your project on Weblate for free then you can host it by own. You can follow below steps:

    • First, we deploy Weblate on our localhost using the installation guide given on Weblate site. I install Weblate from git. I cloned latest source code using Git
    git clone https://github.com/WeblateOrg/weblate.git
    • Now change directory to where you cloned weblate source code and install all the required dependencies and optional dependencies using code
    pip install -r requirements.txt

    and

    pip install -r requirements-optional.txt
    • After doing that we copy weblate/settings_example.py to weblate/settings.py. Then we configure settings.py and use the following command to migrate the settings.
    ./manage.py migrate
    • Now create an admin using following command.
    ./manage.py createadmin
    • After that add a project from your Admin dashboard (Web translations-> Projects-> Add Project) by filling all details.
    • Once the project is added, we add the component (Web translations-> Components-> Add Component) to link our Translation files.
    • To change any translation we make changes and push it to the repository where our SSH key generated from Weblate is added. A full guide to do that is mentioned in this link.

    Reference

    How to use Realm in SUSI Android to Save Data

    Sometimes we need to store information on the device locally so that we can use information offline and also query data faster. Initially, SQLite was only option to store information on the device. But working with SQLite is difficult sometimes and also it makes code difficult to understand. Also, SQL queries take a long time. But now we have realm a better alternative of SQLite. The Realm is a lightweight mobile database and better substitute of SQLite. The Realm has own C++ core and store data in a universal, table-based format by a C++ core. This allows Realm to allow data access from multiple languages as well as a range of queries. In this blog post, I will show you why we used Realm and how we save data in SUSI Android using Realm.

    “How about performance? Well, we’re glad you asked 🙂 For all the API goodness & development productivity we give you, we’re still up to 100x faster than some SQLite ORMs and on average ~10x faster than raw SQLite and common ORMs for typical operations.” (compare: https://blog.realm.io/realm-for-android/)

    Advantages of Realm over SQLite are following:

    • It is faster than SQLite as explained on the Realm blog. One of the reasons realm is faster than SQLite is, the traditional SQLite + ORM abstraction is leaky because ORM simply converts  Objects and their methods into SQL statements. Realm, on the other hand, is an object database, meaning your objects directly reflect your database.
    • It is easier to use as it uses objects for storing data. When we use SQLite we need boilerplate code to convert values to and from the database, setting up mappings between classes and tables, fields and columns, foreign keys, etc. Whereas in Realm data is directly exposed as objects and can be queried without any conversion.

    Prerequisites

    To include this library in your project you need

    • Android studio version 1.5.1 or higher.
    • JDK version 7.0 or higher.
    • Android API level 9 or higher.

    How to use realm in Android

    To use Realm in your project we must add the dependency of the library in build.gradle(project) file 

     dependencies {
           classpath “io.realm:realm-gradle-plugin:3.3.1”
       }

    and build.gradle(module) file.

    apply plugin: realm-android
    dependencies {
    compile io.realm:android-adapters:1.3.0
    }

    Now you have to instantiate Realm in your application class. Setting a default configuration in your Application class, will ensure that it is available in the rest of your code.

    RealmConfiguration realmConfiguration = new RealmConfiguration.Builder(this)
                                                                  .deleteRealmIfMigrationNeeded().build();
    Realm.setDefaultConfiguration(realmConfiguration);

    Now we need to create a model class. A model class is use to save data in Realm and retrieve saved data and it must extend RealmObject class. For eg.

    public class Person extends RealmObject {
       private String name;
       public String getName() {
           return name;
       }
       public void setName(String name) {
           this.name = name;
       }
    }

    Field in the model class uses to define columns. For eg. ‘name’ is a column name. Method like setName() use to save data  and getName() use to retrieve saved data.

    Now create an instance of the Realm in the activity where you want to use it. It will be used to read data from the Realm and write data to the Realm.

    Realm realm = Realm.getInstance(this);

    Before you start a new transaction you must call beginTransaction(). It will open database.

    realm.beginTransaction();

    To write data to the Realm you need to create an instance of the model class. createObject used to create an instance of RealmObject class. Our model class is RealmObject type so we use createObject() to create an instance of the model class.

    Person person = realm.createObject(Person.class);

    Write data to realm.

    person.setName(“MSDHONI”);

    And after it you must call commitTransaction(). commitTransaction() use to end transaction.

    realm.commitTransaction();

    Reading data from Realm is easier than writing data to it. You need to create an instance of the Realm.

    Realm realm = Realm.getInstance(this);

    To create query use where the method and pass the class of object you want to query. After creating query you can fetch all data using findAll() method.

    realm.where(Person.class).findAll();

    Reference

    List all the Users Registered on SUSI.AI

    In this blog, I’ll be telling on how SUSI admins can access list of all the registered users from SUSI-server. Following this, they may modify/edit user role of any registered user.

    What is User Role?

    A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely :

    • BOT
    • ANONYMOUS
    • USER  
    • REVIEWER
    • ACCOUNTCREATOR
    • ADMIN
    • BUREAUCRAT

    * Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely.

    All the users who are not logged in but interacting with SUSI are anonymous users. These are only subject to chat with SUSI, login, signup or may use forgot password service. Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them. Privileged users are those who have special rights with them. These are more like moderators with much special rights than any other user. At the top level of the hierarchy are the admins. These users have more rights than anyone. They can change role of any other user, override decision of any privileged user as well.

    Let us now look at the control flow of this.

    First things first, make a component of User List in the project. Let us name it ListUsers and since it has to be accessible by those users who possess ADMIN rights, you will find it enclosed in Admin package in components folder. Open up

    index.js file, import Listusers component  and add route to it in the following way :

    ...//other import statements
    import ListUser from "./components/Admin/ListUser/ListUser";
    ...//class definition and other methods
    <Route path="/listUser" component={ListUser}/>
    //other routes defined
    

    Find a suitable image for “List Users” option and add the option for List Users in static appbar component along with the image. We have used Material UI’s List image in our project.

    ...// other imports
    
    import List from 'material-ui/svg-icons/action/list';
    
    Class and method definition
    
    <MenuItem primaryText="List Users"
              onTouchTap={this.handleClose}
              containerElement={<Link to="/listUser" />}
                    rightIcon={<List/>}
          />
    
    ...//other options in top right corner menu
    

    Above code snippet will add an option to redirect admins to ‘/listUsers’ route. Let us now have a closer look at functionality of both client and server. By now you must have known what ComponentDidMount does. {If not, I’ll tell you. This is a method which is given first execution after the page is rendered. For more information, visit this link}. As mentioned earlier as well that this list will be available only for admins and may be even extended for privileged users but not for anonymous or any other user, an AJAX call is made to server in ComponentDidMount of ‘listuser’ route which returns the base user role of current user. If user is an Admin, another method, fetchUsers() is called.

    let url;
            url = "http://api.susi.ai/aaa/account-permissions.json";
            $.ajax({
                url: url,
                dataType: 'jsonp',
                jsonpCallback: 'py',
                jsonp: 'callback',
                crossDomain: true,
                success: function (response) {
                    console.log(response.userRole)
                    if (response.userRole !== "admin") {
                        console.log("Not an admin")
                    } else {
                        this.fetchUsers();
                        console.log("Admin")
                    }
                }.bind(this),
    });
    

    In fetchUsers method, an AJAX call is made to server which returns username in JSONArray. The response looks something likes this :

    {
    	"users" : {
    		"email:""[email protected]",
    ...
    },
    "Username":["[email protected]", "[email protected]"...]
    }
    

    Now, only rendering this data in a systematic form is left. To give it a proper look, we have used material-ui’s table. Import Table, TableBody, TableHeader,

       TableHeaderColumn, TableRow, TableRowColumn from material-ui/table.

    In fetchUsers method, response is catched in data Oblect. Now the keys are extracted from the JSON response and mapped with an array. Iterating through array received as username array, we get list of all the registered users. Now, popuulate the data in the table you generated.

    return (
                            <TableRow key={i}>
                                <TableRowColumn>{++i}>
                                <TableRowColumn>{name}</TableRowColumn>
                                <TableRowColumn> </TableRowColumn>
                                <TableRowColumn> </TableRowColumn>
                                <TableRowColumn> </TableRowColumn>
                                <TableRowColumn> </TableRowColumn>
                            </TableRow>
                        )
    

    Above piece of code may help you while populating the table. These details are returned from susi server which gets a list of all the registered in the following manner. First, it checks if base url of this user is something apart from admin. If not, it returns error which may look like this :

    Failed to load resource: the server responded with a status of 401 (Base user role not sufficient. Your base user role is 'ANONYMOUS', your user role is 'anonymous')
    

    Otherwise, it will generate a client identity, use to to get an authorization object which will loop through authorization.json file and return all the users encoded as JSONArray.

    Additional Resources

    1. Official Material UI Documentation on Tables from marterial-ui
    2. Answer by Marco Bonelli on Stackoverflow on How to map JSON response in JavaScript?
    3. Answer by janpieter_z on Stackoverflow – on Render JSON data in ReactJS table

    SUSI.AI User Roles and How to Modify Them

    In this blog, I discuss what is ‘user-role’ in SUSI.AI, what are the various roles and how SUSI admins can modify/update a user’s roles.

    What is User Role?

    A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely :

    • BOT
    • ANONYMOUS
    • USER  
    • REVIEWER
    • ACCOUNTCREATOR
    • ADMIN
    • BUREAUCRAT

    * Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely.

    If SUSI is active as a bot on some bot integrated platform (like line or kik), the user role assigned to it will be that of BOT. This user role just has technical access to the server.

    All the users who are not logged in but interacting with SUSI are ANONYMOUS users. These are only subject to chat, login and signup. They may use forgot password service and reset password services as well.

    Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them as USER(s).

    Users with role assigned as “REVIEWERS” are expected to manage the Skill CMS. There might be some dispute or conflict in a skill. REVIEWERS then take the access of skill data and finalise the conflict there itself for smooth functionality.

    ADMIN users are those who have special rights with them. These are more like moderators with much special rights than any other user.

    At the top level of the hierarchy are the BUREAUCRATS. These users have more rights than anyone. They can change role of any other user, override decision of any ADMIN user as well. Both admins and bureaucrats have the access to all the settings file on the server. They not only can look at the list, but also download and upload them. Now these users also have right to upgrade or downgrade any other user as well.

    All these user roles are defined in UserRole.java file.

    In each request received by the server, the user role of user making the request is compared with the minimal user role in getMinimalUserRole() method. This method is defined in AbstractAPIHandler which validates if a user is allowed to access a particular servlet or not.

    private void process(HttpServletRequest request, HttpServletResponse response, Query query) throws ServletException, IOException {
    	// object initialisation and comparsions
    // user authorization: we use the identification of the user to get the assigned authorization
            Authorization authorization = DAO.getAuthorization(identity);
    
            if (authorization.getUserRole().ordinal() < minimalUserRole.ordinal()) {
            	response.sendError(401, "Base user role not sufficient. Your base user role is '" + authorization.getUserRole().name() + "', your user role is '" + authorization.getUserRole().getName() + "'");
    			return;
            }
    // evaluations based on other request parameters.
    }
    

    Now that we know about what User Roles actually are, let us look at how the servlet which allows the users {with at least ADMIN login} to change user role of some other user works.

    In the request, 2 parameters are expected. These are :

    • user : email id of the user whose role has to be changed.
    • role : new role which will be assigned to this user.

    Using a switch case, we identify the user role which is requested. If role is found to be null or any other value apart from “bot”, “anonymous”, “user”, “reviewer”, “accountcreator”, “admin” or “bureaucrat”, an error with error code 400 and error message “Bad User role” is thrown.

    In the next steps, server generates client identity in order to get the corresponding Authorization object. If the user is not found in the database, again an error is thrown with error code 400 and error message “role not found

    ClientCredential credential = new ClientCredential(ClientCredential.Type.passwd_login, userTobeUpgraded);
                ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, credential.getName());
                if (!DAO.hasAuthorization(identity)) {
                    throw new APIException(400, "Username not found");
                }
    

    By now, server is clear with the user identity and new role to be assigned. Since the user role is defined in authorization.json file, we overwrite the existing user role and finally server sends back the new user role of the use

    Authorization auth = DAO.getAuthorization(identity);
                try {
                    auth.setUserRole(userRole);
                } catch (IllegalArgumentException e) {
                    throw new APIException(400, "role not found");
                }
    
                // Print Response
                result.put("newDetails", auth.getJSON());
                result.put("accepted", true);
                result.put("message", "User role changed successfully!!");
                return new ServiceResponse(result);