Read more about the article Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners
SUSI Desktop

Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners

SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop.

Any electron app essentially comprises of the following components

    • Main Process (Managing windows and other interactions with the operating system)
    • Renderer Process (Manage the view inside the BrowserWindow)

Steps to setup development environment

      • Clone the repo locally.
$ git clone https://github.com/fossasia/susi_desktop.git
$ cd susi_desktop
      • Install the dependencies listed in package.json file.
$ npm install
      • Start the app using the start script.
$ npm start

Structure of the project

The project was restructured to ensure that the working environment of the Main and Renderer processes are separate which makes the codebase easier to read and debug, this is how the current project is structured.

The root directory of the project contains another directory ‘app’ which contains our electron application. Then we have a package.json which contains the information about the project and the modules required for building the project and then there are other github helper files.

Inside the app directory-

  • Main – Files for managing the main process of the app
  • Renderer – Files for managing the renderer process of the app
  • Resources – Icons for the app and the tray/media files
  • Webview Tag

    Display external web content in an isolated frame and process, this is used to load chat.susi.ai in a BrowserWindow as

    <webview src="https://chat.susi.ai/"></webview>
    

    Adding event listeners to the app

    Various electron APIs were used to give a native feel to the application.

  • Send focus to the window WebContents on focussing the app window.
  • win.on('focus', () => {
    	win.webContents.send('focus');
    });
    
  • Display the window only once the DOM has completely loaded.
  • const page = mainWindow.webContents;
    ...
    page.on('dom-ready', () => {
    	mainWindow.show();
    });
    
  • Display the window on ‘ready-to-show’ event
  • win.once('ready-to-show', () => {
    	win.show();
    });
    

    Resources

    1. A quick article to understand electron’s main and renderer process by Cameron Nokes at Medium link
    2. Official documentation about the webview tag at https://electron.atom.io/docs/api/webview-tag/
    3. Read more about electron processes at https://electronjs.org/docs/glossary#process
    4. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.

    Continue ReadingSetting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners

    Showing “Get started” button in SUSI Viber bot

    When we start a chat with SUSI.AI on Viber i.e. SUSI Viberbot, there should be an option on how to get started with the bot. The response to it are some options like “Visit repository”, “How to contribute” which direct the user to check how SUSI.AI bot is made and prompts him/her to contribute to it. Along with that an option of “start chatting” can be shown to add up some sample queries for the user to try.

    To accomplish the task at hand, we will accomplish these sub tasks:

    1. To show the “Get started” button.
    2. To show the reply to “Get started” query.
    3. To respond to the queries, nested in the response of “Get started”

    Showing “Get started”:

    The Viber developers platform notifies us when a user starts a conversation with our bot. To be exact, a conversation_started event is sent to our webhook and can be handled accordingly. The Viberbot shows a welcome message to the user along with a Get started button to help him/her start.

    To send just the welcome message:

    if (req.body.event === 'conversation_started') {
           // Welcome Message
           var options = {
               method: 'POST',
               url: 'https://chatapi.viber.com/pa/send_message',
               headers: headerBody,
               body: {
                   // some required body properties here
                   text: 'Welcome to SUSI.AI!, ' + req.body.user.name + '.',
                   // code for showing the get started button here.
            }
               json: true
           };
     
           request(options, function(error, res, body) {
               // handle error
           });
       }

    The next step is to show the “Get started” button. To show that we use a keyboard tool, provided by Viber developers platform. So after the “text” key we have the “keyboard” key and a value for it:

    keyboard: {
                 "Type": "keyboard",
                 "DefaultHeight": true,
                 "Buttons": [{
                     "ActionType": "reply",
                     "ActionBody": "Get started",
                 }]
             }

    The action type as shown in the code, can be “reply” or “open-url”. The “reply” action type, triggers an automatic query sent back with “Get started” (i.e. the value of “ActionBody” key), when that button gets clicked.

    Hence, this code helps us tackle our first sub task:

    Reply to “Get started”:

    We target to make each SUSI.AI bot generic. The SUSI FBbot and SUSI Tweetbot shows some options like “Visit repository”, “Start chatting” and “How to contribute?” for the “Get started” query. We render the same answer structure in Viberbot.

    The “rich_media” type helps us send buttons in our reply message. As we ought to use three buttons in our message, the button rows are three in the body object:

    if(message === "Get started"){
                       var options = {
                           method: 'POST',
                           url: 'https://chatapi.viber.com/pa/send_message',
                           headers: headerBody,
                           body: {
                               // some body object properties here
                               type: 'rich_media',
                               rich_media: {
                                   Type: "rich_media",
                                   ButtonsGroupColumns: 6,
                                   ButtonsGroupRows: 3,
                                   BgColor: "#FFFFFF",
                                   Buttons: buttons
                               }
                           },
                           json: true
                       };
     
                       request(options, function(error, res, body) {
                           if (error) throw new Error(error);
                           console.log(body);
                       });

    As said before, 2 type of Action types are available – “open-url” and “reply”. “Visit repository” button has an “open-url” action type and “How to contribute?” or “start chatting” has a “reply” action type.

    Example of “Visit repository” button:

    var buttons = [{
                    Columns: 6,
                    Rows: 1,
                    Text: "Visit repository",
                    "ActionType": "open-url",
                    "ActionBody": "https://www.github.com/fossasia/susi_server",
                    // some text styling properties here
                 }];

    To respond to the “reply” action type queries:

    When the “reply” action type button gets clicked, it triggers an automatic query sent back to the bot with the value same as that of the “ActionBody” key. So we just need to apply a check if the message string recieved is “Start chatting” or “How to contribute?”

    For the response to “Start chatting”, we plan to show sample queries for the user to try. This can be shown by using buttons with the action type as “reply”.

    Code snippet to show a button with the text as “What is FOSSASIA?”:

    var buttons = [{
                            Columns: 6,
                            Rows: 1,
                            Text: "What is FOSSASIA? ",
                            "ActionType": "reply",
                            "ActionBody": "What is FOSSASIA?",
                            // text styling here
                        }];

    For the response to “How to contribute”, we show some messages to help the user contribute to SUSI.AI. These messages also just need buttons with it, to be able to apply the necessary action.

    We respond with 2 messages to the user, both the messages have a button.

    For example, a button to visit the SUSI.AI Gitter channel:

    var buttons = [{
                        Columns: 6,
                        Rows: 1,
                           Text: "<font color=#323232><b>Chat on Gitter</b></font>",
                          ActionType: "open-url",
                          ActionBody: "https://www.gitter.im/fossasia/susi_server",
                          // text styling here
                }];

    This way we have successfully added the “Get started” option to our Viberbot and handled all the subsequent steps.

    Resources:

    1. Viber video managing chat extensions by Ingrid Lunden from Tech crunch.
    2. Develop a chat bot with node js by Slobodan Stojanović from smashing magazine.
    Continue ReadingShowing “Get started” button in SUSI Viber bot

    Creating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin

    An Android application often includes settings that allow the user to modify features of the app. For example, SUSI Android app allows users to specify whether they want to use in built mic to give speech input or not. Different settings in SUSI Android app and their purpose are given below

    Setting                                        Purpose
    Enter As Send It allows users to specify whether they want to use enter key to send message or to add new line.
    Mic Input It allows users to specify whether they want to use in built mic to give speech input or not.
    Speech Always It allows users to specify whether they want voice output in case of speech input or not.
    Speech Output It allows users to specify whether they want speech output irrespective of input type or not.
    Language It allows users to set different query language.
    Reset Password It allows users to change password.
    Select Server It allows users to specify whether they want to use custom server or not.

    Android provides a powerful framework, Preference framework, that allows us to define the way we want preferences. In this blog post, I will show you how Settings UI is created using Preference framework and Kotlin in SUSI Android.

    Advantages of using Preference are:

    • It has own UI so we don‘t have to develop our own UI for it
    • It stores the string into the SharedPreferences so we don’t need to manage the values in SharedPreference.

    First, we will add the dependency in build.gradle(project) file as shown below.

    compile ‘com.takisoft.fix:preference-v7:25.4.0.3’

    To create the custom style for our Settings Activity screen we can set

    android:theme=“@style/PreferencesThemeLight”

    as the base theme and can apply various other modifications and colour over this. By default, it has the usual Day and Night theme with NoActionBar extension.

    Layout Design

    I used PreferenceScreen as the main container to create UI of Settings and filled it with the other components. Different components used are following:

    • SwitchPreferenceCompat: This gives us the Switch Preference which we can use to toggle between two different modes in the setting.
    <com.takisoft.fix.support.v7.preference.SwitchPreferenceCompat

    android:defaultValue=”true”

    • PreferenceCategory: It is used for grouping the preference. For example, Chat Settings, Mic Settings, Speech Settings etc are different groups in settings.

    • ListPreference: This preference display list of values and help in selecting one. For example in setLanguage option ListPreference is used to show a list of query language. List of query language is provided via xml file array.xml (res/values). Attribute android:entries point to arrays languagentries and android:entryValue holds the corresponding value defined for each of the languages.
    <ListPreference

    android:title=“@string/Language”
    android:key=“Lang_Select”

    android:entries=“@array/languagentries”

    android:entryValues=“@array/languagentry”

    </ListPreference>

    Implementation in SUSI Android

    All the logic related to Preferences and their action is written in ChatSettingsFragment class. ChatSettingsFragment extends PreferenceFragmentCompat class.

    class ChatSettingsFragment : PreferenceFragmentCompat()

    Fragment populate the preferences when created. addPreferencesFromResource method is used to inflate view from xml.

    addPreferencesFromResource(R.xml.pref_settings)

    Reference

    Continue ReadingCreating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin

    Implement Internationalization in SUSI Android With Weblate

    When you build an Android app, you must consider about users for whom you are building an app. It may be possible that you users are from the different region. To support the most users your app should show text in locale language so that user can use your app easily. Our app SUSI Android is also targeting users from different regions. Internationalization is a way that ensures our app can be adapted to various languages without requiring any change to source code. This also allows projects to collaborate with non-coders more easily and plugin translation tools like Weblate.

    Benefits of using Internationalization are:

    • It reduces the time for localization i.e it will localize your app automatically.
    • It helps us to keep and maintain only single source code for different regions.

    To achieve Internationalization in Android app you must follow below steps:

    • Move all the required contents of your app’s user interface into the resource file.
    • Create new directories inside res to add support for Internationalization. Each directory’s name should follow rule <resource type>-(language code). For example values-es contains string resource for es language code i.e Spanish.
    • Now add different locale content in the respective folder.

    We need to create separate directories for different locale because to show locale specific content, Android check specific folder i.e res/<resource type>-(language code) like res/values-de and show content from that folder. That’s why we need to move all the required content into resource file so that each required content can be shown in the specific locale.

    How Internationalization is implemented in SUSI Android

    In SUSI Android there is not any locale specific image but only string. So I created only locale specific value resource folder to add locale specific strings. To create locale specific values folder I follow the above-mentioned rule i.e <resource type>-(language code).

    After that, I added specific language string in the respective folder.

    Instead of hard-coded strings, we used strings from string.xml file so that it will change automatically according to the region.

    android:text=“@string/reset”

    and

    showToast(getString(R.string.wrong_password))

    In absence of resource directory for any specific locale, Android use default resource directory.

    Integrate Weblate in SUSI Android

    Weblate is a web based translation tool. The best part of Weblate is its tight version control integration which makes it easy for translators to contribute because translator does not need to fork your repo and send pull request for each change but Weblate handle this part i.e translator translate strings of the project in Weblate site and Weblate will send pull request for those changes.

    Weblate can host your free software projects for free but it depends on them. Here is the link of SUSI Android project hosted on Weblate. If your project is good then they can host your project for free. But for that, you have to apply from this link and select ask for hosting. Now fill up form as shown in below picture.

    Once your project is hosted on Weblate, they will email you about it. After that, you have to integrate Weblate in your project so that Weblate can automatically push translated strings to your project and also Weblate get notify about changes in your repository. Here is the link on how to add Weblate service and Weblate user to your project.

    If it is not possible to host your project on Weblate for free then you can host it by own. You can follow below steps:

    • First, we deploy Weblate on our localhost using the installation guide given on Weblate site. I install Weblate from git. I cloned latest source code using Git
    git clone https://github.com/WeblateOrg/weblate.git
    • Now change directory to where you cloned weblate source code and install all the required dependencies and optional dependencies using code
    pip install -r requirements.txt

    and

    pip install -r requirements-optional.txt
    • After doing that we copy weblate/settings_example.py to weblate/settings.py. Then we configure settings.py and use the following command to migrate the settings.
    ./manage.py migrate
    • Now create an admin using following command.
    ./manage.py createadmin
    • After that add a project from your Admin dashboard (Web translations-> Projects-> Add Project) by filling all details.
    • Once the project is added, we add the component (Web translations-> Components-> Add Component) to link our Translation files.
    • To change any translation we make changes and push it to the repository where our SSH key generated from Weblate is added. A full guide to do that is mentioned in this link.

    Reference

    Continue ReadingImplement Internationalization in SUSI Android With Weblate

    How to use Realm in SUSI Android to Save Data

    Sometimes we need to store information on the device locally so that we can use information offline and also query data faster. Initially, SQLite was only option to store information on the device. But working with SQLite is difficult sometimes and also it makes code difficult to understand. Also, SQL queries take a long time. But now we have realm a better alternative of SQLite. The Realm is a lightweight mobile database and better substitute of SQLite. The Realm has own C++ core and store data in a universal, table-based format by a C++ core. This allows Realm to allow data access from multiple languages as well as a range of queries. In this blog post, I will show you why we used Realm and how we save data in SUSI Android using Realm.

    “How about performance? Well, we’re glad you asked 🙂 For all the API goodness & development productivity we give you, we’re still up to 100x faster than some SQLite ORMs and on average ~10x faster than raw SQLite and common ORMs for typical operations.” (compare: https://blog.realm.io/realm-for-android/)

    Advantages of Realm over SQLite are following:

    • It is faster than SQLite as explained on the Realm blog. One of the reasons realm is faster than SQLite is, the traditional SQLite + ORM abstraction is leaky because ORM simply converts  Objects and their methods into SQL statements. Realm, on the other hand, is an object database, meaning your objects directly reflect your database.
    • It is easier to use as it uses objects for storing data. When we use SQLite we need boilerplate code to convert values to and from the database, setting up mappings between classes and tables, fields and columns, foreign keys, etc. Whereas in Realm data is directly exposed as objects and can be queried without any conversion.

    Prerequisites

    To include this library in your project you need

    • Android studio version 1.5.1 or higher.
    • JDK version 7.0 or higher.
    • Android API level 9 or higher.

    How to use realm in Android

    To use Realm in your project we must add the dependency of the library in build.gradle(project) file 

     dependencies {
           classpath “io.realm:realm-gradle-plugin:3.3.1”
       }

    and build.gradle(module) file.

    apply plugin: realm-android
    dependencies {
    compile io.realm:android-adapters:1.3.0
    }

    Now you have to instantiate Realm in your application class. Setting a default configuration in your Application class, will ensure that it is available in the rest of your code.

    RealmConfiguration realmConfiguration = new RealmConfiguration.Builder(this)
                                                                  .deleteRealmIfMigrationNeeded().build();
    Realm.setDefaultConfiguration(realmConfiguration);

    Now we need to create a model class. A model class is use to save data in Realm and retrieve saved data and it must extend RealmObject class. For eg.

    public class Person extends RealmObject {
       private String name;
       public String getName() {
           return name;
       }
       public void setName(String name) {
           this.name = name;
       }
    }

    Field in the model class uses to define columns. For eg. ‘name’ is a column name. Method like setName() use to save data  and getName() use to retrieve saved data.

    Now create an instance of the Realm in the activity where you want to use it. It will be used to read data from the Realm and write data to the Realm.

    Realm realm = Realm.getInstance(this);

    Before you start a new transaction you must call beginTransaction(). It will open database.

    realm.beginTransaction();

    To write data to the Realm you need to create an instance of the model class. createObject used to create an instance of RealmObject class. Our model class is RealmObject type so we use createObject() to create an instance of the model class.

    Person person = realm.createObject(Person.class);

    Write data to realm.

    person.setName(“MSDHONI”);

    And after it you must call commitTransaction(). commitTransaction() use to end transaction.

    realm.commitTransaction();

    Reading data from Realm is easier than writing data to it. You need to create an instance of the Realm.

    Realm realm = Realm.getInstance(this);

    To create query use where the method and pass the class of object you want to query. After creating query you can fetch all data using findAll() method.

    realm.where(Person.class).findAll();

    Reference

    Continue ReadingHow to use Realm in SUSI Android to Save Data

    Showing location response in SUSI.AI bots

    SUSI.AI has a capability to tell the basic information about a location, it is asked for. Along with the basic information about that place, it shows a map (i.e. open street map) pointing to that location. The task at hand is to inculcate this “location” feature to the SUSI.AI messenger bots. The SUSI Tweetbot and SUSI Fbbot are used as examples in this blog.

    Let’s first check on what response do we get, when a person asks a query like “where is london” to the SUSI API. Along with the basic information about that location, the result as shown below has the type of reply (i.e. map), latitude, longitude and a link to the open street map.

    "actions": [
          {
            "type": "answer",
            "language": "en",
            "expression": "Ludhiana is a city and a municipal corporation in Ludhiana district in the Indian state of Punjab, and is the largest city north of Delhi."
          },
          {
            "type": "anchor",
            "link": "https://www.openstreetmap.org/#map=13/30.912040570244187/75.85379021980509",
            "text": "Here is a map",
            "language": "en"
          },
          {
            "type": "map",
            "latitude": "30.912040570244187",
            "longitude": "75.85379021980509",
            "zoom": "13",
            "language": "en"
          }
        ]

    The response for a location type query has these 3 main parts:

    1. Clickable static map image.
    2. A basic information of the place asked about.
    3. The link i.e. when the static map image is clicked it should redirect to the corresponding map location.

    Let’s try to make up with the 1st part of the response i.e. Static map image.

    The map quest API is used to result in a static map image of the location. We need an API key to access the map quest API, which can be requested from their developer site.

    Along with that we need the latitude and longitude of the location at hand.

    The SUSI API’s response helps us to get the latitude value:

    // if body represents the response object 
    var lat = body.answers[0].actions[2].latitude;
    

    And the longitude value:

    var lon = body.answers[0].actions[2].longitude;
    

    Using the three values that are API key, latitude and longitude, the static image is rendered by this link:

    var static_image_url = "https://open.mapquestapi.com/staticmap/v4/getmap?key=API_KEY&size=600,400&zoom=13&center="+lat+","+lon;
    

    The second part is, basic information about the place asked, can be fetched from:

    // if body is the JSON response object from SUSI API
    var mapMessage = body.answers[0].actions[0].expression;
    

    The link to the map location can be easily fetched from the SUSI API’s response:

    var link = body.answers[0].actions[1].link;
    

    As all the three parts are ready, let’s look on how to render them on the SUSI.AI bot’s screen.

    Facebook:

    Sending a generic template message object:

    message = {
            "type":"template",
            "payload":{
                        "template_type":"generic",
                        "elements":[{
                            "title": mapMessage,
                           "image_url": static_image_url,
                           "Item_url": link
                        }]
            }
    };
    
    sendTextMessage(sender, message, 1);
    

    Twitter:

    The Twitter API does not need a static image of the map to be rendered. It does that work for us. We just need to send an event to the Twitter API with the message data object constituting of the map message, the latitude and longitude values:

    "message_data": {
                "text": mapMessage,
                "attachment": {
                    "type": "location",
                    "location": {
                        "type": "shared_coordinate",
                        "shared_coordinate": {
                            "coordinates": {
                                "type": "Point",
                                "coordinates": [lon, lat]
                            }
                        }
                    }
                }
    }

    Resources:

    1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
    2. Drive discovery of bots and other customer experiences in direct messages by By Travis Lull from Twitter blog.
    3. By Seth Rosenberg from Facebook developers blogLink Ads to Messenger, Enhanced Mobile Websites, Payments and More.
    Continue ReadingShowing location response in SUSI.AI bots

    Implementation of Set Different Language for Query in SUSI Android

    SUSI.AI has many skills. Some of which are displaying web search of a certain query, provide a list of relevant information of a topic, displaying a map of the certain position and simple text message of any query. Previously SUSI.AI reply all query in English language but now one important feature is added in SUSI.AI and that feature is, reply query of the user in the language that user wants. But to get the reply in different language user has to send language code of that language along with query to SUSI Server. In this blog post, I will show you how it is implemented in SUSI Android app.

    Different language supported in SUSI Android

    Currently, different languages added in option in SUSI Android and their language code are:

    Language Language Code
    English en
    German de
    Spanish es
    French fr
    Italian it
    Default Default language of the device.

    Layout design

    I added an option for choosing the different language in settings. When the user clicks on Language option a dialog box pops up. I used listpreference to show the list of different languages.

    <ListPreference

      android:title=“@string/Language”

      android:key=“Lang_Select”

      android:entries=“@array/languagentries”

      android:entryValues=“@array/languagentry”>

    </ListPreference>

    “title” is used to show the title of setting, “entries” is used to show the list of entry to the user and “entryValue” is the value corresponding to each entry. I used listpreference because it has own UI so we don‘t have to develop our own UI for it and also it stores the string into the SharedPreferences so we don’t need to manage the values in SharedPreference. SharedPreference needed to set value in Language in settings so that once user close app and again open it setting will show same value otherwise it will show default value. We used an array of string to show the list of languages.

    <string-array name=“languagentries”>

      <item>Default</item>

      <item>English</item>

      <item>German</item>

      <item>Spanish</item>

      <item>French</item>

      <item>Italian</item>

    </string-array>

    SetLanguage implementation

    To set language user must choose Language option in setting.

    On clicking Language option a dialog box pop up with the list of different languages. When the user chooses a language then we save corresponding language code in preference with key “prefLanguage” so that we can use it later to send it to the server along with the query. It also uses to send language to the server to store user language on the server, so that user can use the same language in the different client.

    querylanguage.setOnPreferenceChangeListener { _, newValue ->

      PrefManager.putString(Constant.LANGUAGE, newValue.toString())

      if(!settingsPresenter.getAnonymity()) {

          settingsPresenter.sendSetting(Constant.LANGUAGE, newValue.toString(), 1)

      }

    }

    newValue.toString() is the value i.e language code of corresponding language.

    Now when we query anything from SUSI.AI we send language code along with query to the server. Default language is default language of the device. Before sending language to the server we check language is default language or any specific language.

    val language = if (PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT).equals(Constant.DEFAULT))

    Locale.getDefault().language

    else PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT)

    And after that, we send the corresponding language along with query to the server.

    clientBuilder.susiApi.getSusiResponse(timezoneOffset, longitude, latitude, source, language, query)

    Reference

    Continue ReadingImplementation of Set Different Language for Query in SUSI Android

    Using Vector Images in SUSI Android

    Android designed to run across many devices with different screen sizes and display resolutions. One of the things that confuse many new Android developers is how to support multiple screen sizes. For making  SUSI Android app more user-friendly and interactive we used a lot of images in the form of drawable resources. Most of these drawables are in the form of PNG (Portable Network Graphic) and to support multiple screen size we have to include separate images of varied dimensions because we can’t scale PNG images without losing quality.

    Other disadvantages of using PNG images are

    • PNG images take large disk space.
    • PNG images have fixed colour and dimensions which cannot be changed.

    To overcome these shortcomings of PNG image we decided to use vector images in SUSI Android.

    Advantages of using vector image are

    • It is smaller in size as compared to PNG i.e it takes less disk space than PNG.
    • It can be scaled without losing quality. So we need to include only a single image for all screen size.

    Disadvantages of using vector image are

    • Reconstruction of vector data may take considerably longer than that contained in a bitmap file of equivalent complexity because each image element must be drawn individually and in sequence.
    • Vector image can’t be used to display complex photographs.
    • If the object consists of a large number of small elements then file size grow very fast.

    Vector produce smaller size file than bitmap only for simple stuff. When it comes to creating a photographic quality vector where colour information is paramount and may vary on a pixel-by-pixel basis, the file can be much larger than its bitmap version, that’s why vector image is only suitable for small images like logo designs, icons etc.

    In this blog post, I will show you how we used vector images in SUSI Android.

    How to create vector image using Asset Studio tool

    Asset Studio is inbuilt Android Studio tool which we use to create a vector image from default material icon or local SVG images. To use vector asset in Android Studio

    • Go to File> New > Vector Asset
    • Vector Asset Studio Tool Window is prompted

    • Create vector image from default material icon or local SVG image.

    Asset Studio will add icon in drawable. It can be used anywhere in the project.

    Vector images in SUSI Android

    In SUSI Android all drawable images like search, mic, arrow, check etc are vector images,  even the logo of the app is a vector image.

    This vector image is the result of code you can find here.

    Now I will describe some special attributes used here

    • fillColor: It defines the colour of the image. So the colour of your image depends on the colour you provide in fillColor. We used white color(#ffffff) as fillColor that’s why the image is white in colour.
    • strokeColour: It defines the colour of boundary of the image.
    • pathData: Shape of your image depends on pathData. Different commands used in pathData are
    1. M: It is used to define the starting point. For example, M10,10 means starting     Coordinate of the image will be (10,10). M used for absolute position and m   for relative position.
    2. L: L means to draw a line from the current position to given position. For example, M10,10 L20,20 means a straight line will be drawn from coordinate (10,10) to coordinate (20,20. L Use for absolute position and l for relative position.
    3. C: C means to draw an arc. For example, Cx1,x2 x3,x4 x5,x6 means draw a curve  (cubic Bezier curve) from coordinate  (x1,x2) using (x3,x4) as control point at the beginning and (x5,x6) as control point at the end.
    4. Z: Z means to close the path using the straight line between the current position and starting position.

      Example

    Let’s consider a curve

    M9,16.17L4.83,12l-1.42,1.41L9,19 21,7l-1.41,-1.41z

    It starts from point (9,16.17). From (9,16.17) to (4.83,12) a straight line will be drawn. Now again a straight will be drawn from point (4.83,12) to the relative point (-1.42,1.41). After that from point (-1.42, 1.41) it will draw a polyline like an upward arrow(^) with the end point at (21,7). Again it will draw a straight line between point (21,7) and relative point (-1.41, -1.41). At the last z will close curve using the straight line between current point and starting point).

    Resources

    Tags

     

    Continue ReadingUsing Vector Images in SUSI Android

    Shift from Java to Kotlin in SUSI Android

    Previously SUSI Android was written in JAVA. But recently Google announced that it will officially support Kotlin on Android as a first class language so we decided to shift from Java to Kotlin. Kotlin runs on Java Virtual Machine and hence we can use Kotlin for Android app development. Kotlin is new programming language developed by JetBrains, which also developed IntelliJ-the IDE that Android Studio is based on. So Android Studio has excellent support for Kotlin.

    Advantages of Kotlin over Java

    • Kotlin is a null safe language. It changes all the instances used in the code to non nullable type thus it ensures that the developers don’t get any nullPointerException.
    • Good Android Studio support. Kotlin tool is included with Android Studio 3.0 by default. But for Android Studio 2.x we need to install the required plugin for Kotlin.
    • Kotlin also provides support for Lambda function and other high order functions.
    • One of Kotlin’s greatest strengths as a potential alternative to Java is interoperability between Java and Kotlin. You can even have Java and Kotlin code existing side by side in the same project.
    • Kotlin provides the way to declare Extension function similar to that of C# and Gosu. We can use this function in the same way as we use the member functions of our class.

    After seeing the above points it is now clear that Kotlin is much more effective than Java. In this blog post, I will show you how ForgotPasswordActivity is shifted from JAVA to Kotlin in SUSI Android.

    How to install Kotlin plugin in Android Studio 2.x

    We are using Android Studio 2.3.0 in our android project. To use Kotlin in Android Studio 2.x we need to install Kotlin plugin.

    Steps to include Kotlin plugin in Android Studio 2.x are:

    • Go to File
    • After that select Settings
    • Then select Plugin Option from the left sidebar.
    • After that click on Browse repositories.
    • Search for Kotlin and install it.

    Once it is installed successfully, restart Android Studio. Now configure Kotlin in Android Studio.

    Shift  SUSI Android from JAVA to Kotlin

    Kotlin is compatible with JAVA. Thus it allows us to change the code bit by bit instead of all at once. In SUSI Android app we implemented MVP architecture with Kotlin. We converted the code by one activity each time from JAVA to Kotlin. I converted the ForgotPassword of SUSI Android from JAVA to Kotlin with MVP architecture. I will discuss only shifting of SUSI Android from JAVA to Kotlin and not about MVP architecture implementation.

    First thing is how to extend parent class. In JAVA we need to use extend keyword

    public class ForgotPasswordActivity extends AppCompatActivity

    but in Kotlin parent class is extended like

    class ForgotPasswordActivity : AppCompatActivity ()

    Second is no need to bind view in Kotlin. In JAVA we bind view using Butter Knife.

    @BindView(R.id.forgot_email)

    protected TextInputLayout email;

    and

    email.setError(getString(R.string.email_invalid_title));

    but in Kotlin we can directly use view using id.

    forgot_email.error = getString(R.string.email_invalid_title)

    Another important thing is that instead of using setError and getError we can use only error for both purposes.

    forgot_email.error

    It can be used to get error in TextInputLayout ‘forgot_email’ and

    forgot_email.error = getString(R.string.email_invalid_title)

    can be use to set error in TextInputLayout ‘forgot_email’.

    Third one is in Kotlin we don’t need to define variable type always

    val notSuccessAlertboxHelper = AlertboxHelper(this@ForgotPasswordActivity, title, message, null, null, button, null, color)

    but in JAVA we need to define it always.

    AlertboxHelper notSuccessAlertboxHelper = new AlertboxHelper(ForgotPasswordActivity.this, title, message, null, null, positiveButtonText, null, colour);

    And one of the most important feature of Kotlin null safety. So you don’t need to worry about NullPointerException. In java we need to check for null otherwise our app crash

    if (url.getEditText() == null)

          throw new IllegalArgumentException(“No Edittext hosted!”);

    But in Kotlin there is no need to check for null. You need to use ’?’ and Kotlin will handle it itself.  

    input_url.editText?.text.toString()

    You can find the previous ForgotPasswordActivity here. You can compare it with new ForgotPaswordActivity.

    Reference

    Continue ReadingShift from Java to Kotlin in SUSI Android

    Full-Screen Speech Input Implementation in SUSI Android App

    SUSI Android has some very good features and one of them is, it can take input in speech format from user i.e if the user says anything then it can detect it and convert it to text. This feature is implemented in SUSI Android app using Android’s built-in speech-to-text functionality. You can implement Android’s built-in speech-to-text functionality using either only RecognizerIntent class or SpeechRecognizerIntent class, RecognitionListner interface and RecognizerIntent class. Using former method has some disadvantages:

    • During speech input, it shows a dialog box (as shown here) and it breaks the connection between user and app.
    • We can’t show partial result i.e text to the user but using the later method we can show it.

    We used  SpeechRecognizerIntent class, RecognitionListner interface and RecognizerIntent class to implement Android’s built-in speech-to-text functionality in SUSI Android and you know the reason for that. In this blog post, I will show you how I implemented this feature in SUSI Android with new UI.

    Layout design

    You can give speech input to SUSI Android either by clicking mic button

    or using ‘Hi SUSI’ hotword. When you click on mic button or use ‘Hi SUSI’ hotword, you can see a screen where you will give speech input.

    Two important part of this layout are:

    <TextView

      android:id=“@+id/txtchat”

      android:layout_width=“wrap_content”

      android:layout_height=“wrap_content”

      

     />

    • TextView: It used to show the partial result of speech input i.e it will show converted text (partial) of your speech.
    <org.fossasia.susi.ai.speechinputanimation.SpeechProgressView

      android:id=“@+id/speechprogress”

      android:layout_width=“match_parent”

      android:layout_height=“50dp”

      android:layout_margin=“8dp”

      android:layout_gravity=“center”/>

    • SpeechProgressView: It is a custom view which use to show the animation when the user gives speech input. When the user starts speaking, the animation starts. This custom view contains five bars and these five bars animate according to user input.

    Full-screen speech input implementation

    When the user clicks on mic button or uses ‘Hi SUSI’ hotword, a screen comes where the user can give speech input. As already mentioned I used SpeechRecognizerIntent class, RecognitionListner interface and RecognizerIntent class to implement speech-to-text functionality in SUSI Android. RecognizerIntent class starts an intent and asks for speech input

    val intent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)

    intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,

        RecognizerIntent.LANGUAGE_MODEL_FREE_FORM)

    intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, “com.domain.app”)

    intent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true)

    and send it through the speech recognizer. It does it through ACTION_RECOGNIZE_SPEECH. SpeechRecognizer class provides access to the speech recognition service. This service allows access to the speech recognizer and recognition related event occurs RecognitionListner receive notification from SpeechRecognizer class.

    recognizer = SpeechRecognizer

          .createSpeechRecognizer(activity.applicationContext)

    val listener = object : RecognitionListener {

     //implement all override methods

    }

    When the user starts speaking, the height of bars changes according to change in sound level. When sound level changes, onRmsChanged method get called where we are calling onRmsChanged method of SpeechProgressView class which is responsible for animating bars according to change in sound level.

    override fun onRmsChanged(rmsdB: Float) {

      if (speechprogress != null)

          speechprogress.onRmsChanged(rmsdB)

    }

    When user finished speaking onEndOfSpeech method get called where we call onEndOfSpeech method of SpeechProgressView class which is responsible for rotating animation. Rotation is used to show that SUSI Android has finished listening and now it is processing your input.

    override fun onEndOfSpeech() {

      if (speechprogress != null)

          speechprogress.onEndOfSpeech()

    }

    In case of any error, onError method get called and in case of successful speech input, onResults method get called. In both cases, we reset bars to their initial position and show chat activity user. The user can again give speech input either by clicking on mic or using ‘Hi SUSI’ hotword.

    override fun onResults(results: Bundle) {

      if (speechprogress != null)

          speechprogress.onResultOrOnError()

      activity.supportFragmentManager.popBackStackImmediate()

    }

    Reference

    Continue ReadingFull-Screen Speech Input Implementation in SUSI Android App