Author: amitiwary999

This author has written 17 articles
Read more about the article How RSS Action Type is Implemented in SUSI Android

How RSS Action Type is Implemented in SUSI Android

Important skills of SUSI.AI are to display web search queries, a map of any location and provide a list of relevant information of a topic. RSS action type is similar to websearch action type but when the web search is to be performed on the client side, it is denoted by websearch action type and when the web search is performed by the server itself, it is denoted by rss action type. In this blog, I will show you how rss action type is implemented in SUSI Android.

In case of RSS action type server searches the internet and using RSS feeds, returns an array of objects containing :

  • Title
  • Description
  • Link
{
   “title”: “dog-doh: Definitions Index”,
   “description”: “dog-doh: Definitions Index. dog dog and pony show dog biscuit dog collar dog days …”,
  “link”: “http://websters.yourdictionary.com/index/dog-doh/”,
 }

title: Title related to user query

description: Description of user query.

link: If user want to know more information then user can use link to find more information.

How rss action type is parsed in SUSI Android

SUSI  reply in json format. It should be parsed properly to show it in android app. We used retrofit library developed by square to parse json data. Retrofit library parse data according to model class. We code model class according to expected json reply. For example, each susi response contains answer jsonarray. There are two jsonarray data and action inside answer jsonarray. We made a different model class for each jsonarray.

First model class is SusiResponse. We used this model class to parse ‘answers’ jsonarray.

@SerializedName(“answers”)
 private List<Answer> answers = new ArrayList<>();

Here we used List<>  because ‘answers’ jsonarray contain a list of jsonarray and jsonobject. Answer is second model class. We used it to parse two important jsonarray ‘data’ and ‘action’. The ‘action’ attribute has information about action type.

public class Answer {
    @SerializedName(“data”)
    private RealmList<Datum> data = new RealmList<>();
}

Here also we used the list because data jsonarray also contains a list of jsonobject but instead of simple list we used RealmList<> because after parsing we save data using realm. ‘data’ jsonarray contain multiple jsonobject and each jsonobject contain three important information ‘title’, ‘description’ and ‘link’.

Datum class is the main model class which is used to save and retrieve ‘title’, ‘description’ and ‘link’. setTitle, setLink and setDescription method of Datum class are used to save ‘title’, ‘description’ and ‘link’ and getTitle, getDescription and getLink method are use to retrieve ‘title’, ‘description’ and ‘link’.

How rss action type data is retrieved and saved

As already mentioned we used retrofit to retrieve data and realm to save data. susiResponse is response we received from SUSI server. We used susiResponse to retrieve a list of Datum class type data.

List<Datum> datumList = susiResponse.getAnswers().get(0).getData();

We then loop through datumList and from each element we extract ‘title’, ‘description’ and ‘link’ using getTitle(), getDescription() and getLink() method respectively. Datum class is model class for both retrofit and realm. realmDatum is instance of Datum class and datumRealmList is an instance of RealmList of Datum class type. After extracting data we save data using setTitle(), setDescription() and setLink().

for (Datum datum : datumList) {
          Datum realmDatum = bgRealm.createObject(Datum.class);
          realmDatum.setDescription(datum.getDescription());
          realmDatum.setLink(datum.getLink());
          realmDatum.setTitle(datum.getTitle());
         datumRealmList.add(realmDatum);
       }      

Layout design to show rss action type reply

There are three textview with id ‘title’, ‘description’ and ‘link’ to show ‘title’, ‘description’ and ‘link’ retrieved from SUSI’s reply. We used recyclerview to show list of results.

Datum datum = datumList.get(position);

holder.titleTextView.setText(Html.fromHtml(datum.getTitle()));

holder.descriptionTextView.setText(Html.fromHtml(datum.getDescription()));

holder.descriptionTextView.setText(Html.fromHtml(datum.getDescription()));

Resources

Continue ReadingHow RSS Action Type is Implemented in SUSI Android
Read more about the article How Anonymous Mode is Implemented in SUSI Android

How Anonymous Mode is Implemented in SUSI Android

Login and signup are an important feature for some android apps like chat app because user wants to save messages and secure messages from others. In SUSI Android we save messages for logged in user on the server corresponding to their account. But  users can also  use the app without logging in. In this blog, I will show you how the anonymous mode is implemented in SUSI Android .

When the user logs in using the username and password we provide a token to user for a limited amount of time, but in case of anonymous mode we never provide a token to the user and also we set ANONYMOUS_LOGGED_IN flag true which shows that the user is using the app anonymously.

PrefManager.clearToken()

PrefManager.putBoolean(Constant.ANONYMOUS_LOGGED_IN, true)

We use ANONYMOUS_LOGGED_IN flag to check user is using the app anonymously or not. When a user opens the app we first check user is already logged in or not. If the user is not logged in then we check ANONYMOUS_LOGGED_IN flag is true or false. The true means user is using the app in anonymous mode.

if(PrefManager.getBoolean(Constant.ANONYMOUS_LOGGED_IN, false)) {

 intent = new Intent(LoginActivity.this, MainActivity.class);

startActivity(intent);

}

Else we show login page to the user. The user can use app either by login using username and password or anonymously by clicking skip button. On clicking skip button ANONYMOUS_LOGGED_IN flag set to true.

public void skip() {

Intent intent = new Intent(LoginActivity.this,MainActivity.class);

PrefManager.clearToken();

PrefManager.putBoolean(Constant.ANONYMOUS_LOGGED_IN, true);

startActivity(intent);

}

If the user is using the app in anonymous mode but he/she want to login then he/she can login. There is an option for login in menu.

When the user selects login option from the menu, then it redirects the user to the login screen and ANONYMOUS_LOGGED_IN flag is set to false. ANONYMOUS_LOGGED_IN flag is set to false to ensure that instead of login if the user closes the app and again open it, then he/she can’t use the app until logged in or click skip button.

case R.id.action_login:

PrefManager.putBoolean(Constant.ANONYMOUS_LOGGED_IN, false);

Reference

Continue ReadingHow Anonymous Mode is Implemented in SUSI Android
Read more about the article How Settings of SUSI Android App Are Saved on Server

How Settings of SUSI Android App Are Saved on Server

The SUSI Android allows users to specify whether they want to use the action button of soft keyboard as carriage return or else send action. The user can use SUSI.AI on different client like Android , iOS, Web. To give user uniform experience, we need to save user settings on the server so that if the user makes any change in setting in any chat client then that it changes in other chat clients too. So every chat client must store user specific data on the server to make sure that all chat clients access this data and maintain the same state for that particular user and must accordingly push and pull user data from and to the server to update the state of the app.

We used special key to store setting information on server. For eg.

Setting Key Value Use
Enter As Send enter_send true/false true means pressing enter key send message and false means pressing enter key adds new line.
Mic Input mic_input true/false true means default input method is speech but supports keyboard input too. false means the only input method is keyboard input.
Speech Always speech_always true/false true means we get speech output irrespective of input type.
Speech Output speech_output true/false true means we get speech output in case of speech input and false means we don’t get speech output.
Theme theme dark/light dark means theme is dark and light means theme is light

How setting is stored to server

Whenever user settings are changed, the client updates the changed settings on the server so that the state is maintained across all chat clients. When user change setting, we send three parameters to the server ‘key’, ‘value’ and ‘token’. For eg. let ‘Enter As Send’ is set to false. When user changes it from false to true, we immediately update it on the server. Here key will be ‘enter_send’ and value will be ‘true’.

The endpoint used to add or update User Settings is :

BASE_URL+’/aaa/changeUserSettings.json?key=SETTING_NAME&value=SETTING_VALUE&access_token=ACCESS_TOKEN’

‘SETTING_NAME’ is the key of the corresponding setting, ‘SETTING_VALUE’ is it’s updated value and ‘ACCESS_TOKEN’ is used to find correct user account. We used the retrofit library for network call.

settingResponseCall = ClientBuilder().susiApi .changeSettingResponse(key, value,  PrefManager.getToken())

If the user successfully updated the setting on the server then server reply with message ‘You successfully changed settings of your account!’

How setting is retrieved from server

We retrieve setting from the server when user login. The endpoint used to fetch User Settings is

BASE_URL+’/aaa/listUserSettings.json?access_token=ACCESS_TOKEN’

It requires “ACCESS_TOKEN” to retrieve setting data for a particular user. When user login, we use getUserSetting method to retrieve setting data from the server. PrefManager.getToken() is used to get “ACCESS_TOKEN”.

userSettingResponseCall = ClientBuilder().susiApi .getUserSetting(PrefManager.getToken())

We use userSettingResponseCall to get a response of ‘UserSetting’ type using which we can retrieve different setting from the server. ‘UserSetting’ contain ‘Session’ and ‘Settings’ and ‘Settings’ contain the value of all settings. We save the value of all settings on the server in string format, so after retrieving settings we convert them into the required format. For eg. ‘Enter As Send’ value is of boolean format, so after retrieving we convert it to boolean format.

var settings: Settings ?= response.body().settings

utilModel.setEnterSend((settings?.enterSend.toString()).toBoolean())

Reference

Continue ReadingHow Settings of SUSI Android App Are Saved on Server
Read more about the article How to Show Preview of Any Link in SUSI Android App

How to Show Preview of Any Link in SUSI Android App

Sometimes in a chat app like Whatsapp, messenger we find that scrolling over a link, shows a preview of the link. Android-Link-Preview is an open source library which can be used to show the preview of a link. This library is developed by Leonardo Cardoso. This library makes preview from an url using all the information such as title, relevant texts and images. TextCrawler is the main class of this library. As the name suggests this class is used to crawl given link and grab relevant information. In this blog, I will show you how I used this library in SUSI Android to show the preview of the link.

How to include this library in your project

To use android-link-preview in our project we must add the dependency of the library in build.gradle(project) file

repositories {
   maven { url ‘https://github.com/leonardocardoso/mvn-repo/raw/master/maven-deploy’   }
}

and build.gradle(module) file.

dependencies {
    compile ‘org.jsoup:jsoup:1.8.3‘ // required
    compile ‘com.leocardz:link-preview:1.2.1@aar‘
}

Now import these packages in class where you want to use link preview.

import com.leocardz.link.preview.library.LinkPreviewCallback;

import com.leocardz.link.preview.library.SourceContent;

import com.leocardz.link.preview.library.TextCrawler;

As the name suggests LinkpreviewCallback is a callback which has two important methods onPre() and onPos(). onPre() method called when library starts crawling the web to find relevant information and onPos() method called when crawling finished. This library uses SourceContent class to save and retrieve data.

LinkPreviewCallback linkPreviewCallback = new LinkPreviewCallback() {

@Override

public void onPre() { }

@Override

public void onPos(final SourceContent sourceContent, boolean b) {}

}

TextCrawler class is the main class which use makePreview() method to start web crawling.

makePreview() method needs two parameters

  • linkPreviewCallback : Instance of LinkPreviewCallback class.
  • Link : Link of web page to crawl.
TextCrawler textCrawler = new TextCrawler();

textCrawler.makePreview(linkPreviewCallback, link);

So when we call makePreview method, this library starts crawling the web. But before that, it checks if given link is URL of any image or not. If given link is URL of any image, then there is no need of crawling the web because we can’t find any other information from that link, but if it is not url of any image then it starts crawling the web. All websites are written in HTML. It uses jsoup library to parse HTML documents and extract useful data from that web page. After that it store different information separately i.e it store ‘title’, ‘description’ and ‘image url’ separately using setTitle(), setDescription() and setImage() methods of SourceContent class respectively.

sourceContent.setTitle(metaTags.get(“title”));

When it finishes all these tasks, it call onPos() method of LinkPreview class to inform that it has done its task and now we can use stored data using getTitle(), getDescription() and getImage() methods inside onPos() method.

sourceContent.getTitle();

Example :

We are using Android-link-preview in our SUSI Android app

Reference

Continue ReadingHow to Show Preview of Any Link in SUSI Android App
Read more about the article Using Picasso to Show Images in SUSI Android

Using Picasso to Show Images in SUSI Android

Important skills of SUSI.AI are to display web search queries, maps of any location and provide a list of relevant information of a topic. This blog post will cover why Glide is replaced by Picasso to show images related to these action types and how it is implemented in SUSI Android . Picasso is a powerful image downloading and caching open source library developed by Square.

Why Glide is replaced by Picasso to show images in SUSI Android?

Previously we used Glide library to show preview in SUSI Android but we replace it because it was creating an error continuously.

java.lang.IllegalArgumentException: You cannot start a load for a destroyed activity at com.bumptech.glide.manager.RequestManagerRetriever  at Com.bumptech.glide.manager.RequestManagerRetriever.get(RequestManagerRetriever.java:102) at com.bumptech.glide.manager.RequestManagerRetriever.get(RequestManagerRetriever.java:87)at com.bumptech.glide.Glide.with(Glide.java:629)
atnorg.fossasia.susi.ai.adapters.recycleradapters.WebSearchAdapter.onBindViewHolder(WebSearchAdapter.java:74)

Reason for this error is when activity destroyed and again recreated the context used by glide is old one and  that activity already destroyed .

Glide.with(context).load(imageList.get(0))

One solution of this error is to use context.getApplicationContext()  but it is a bad idea. Another solution is to replace glide by picasso and later one is good because picasso is also a very good image downloading and caching library.

To use Picasso in your project you have to add dependency in build.gradle(Module) file.

dependencies {
   …
   compile “com.squareup.picasso:picasso:2.4.0”
   …
}

How Picasso is used in different actiontype

Map

“actions”: [
      {
        “type”: “map”,
        “latitude”: “1.2896698812440377”,
        “longitude”: “103.85006683126556”,
        “zoom”: “13”
      }
    ]

Link we used to retrieve image url is

http://staticmap.openstreetmap.de/statucmap.php?center=latitude,longitude& zoom=zoom&size=lengthXbreadth

Picasso will load image from this url and show image in the imageview. Here mapImage is the imageview in which map image is shown.

Picasso.with(currContext).load(mapHelper.getMapURL())
                        .into(mapImage, new  com.squareup.picasso.Callback() {    
                            @Override
                            public void onSuccess() {
                                pointer.setVisibility(View.VISIBLE);
                            }
                            @Override
                            public void onError() {
                                Log.d(“Error”, “map image can’t loaded”);
                            }
                        });

WebSearch

When we query like “Search for fog” we get ‘query’ in reply from server

“query”: “fog”

Now we use this query to retrieve image url which we used in Picasso to show images.Picasso load this image into previewImageView imageview. Image url is retrieved using  DuckDuckGo api. We are using url

https://api.duckduckgo.com/?format=json&pretty=1&q=query&ia=meanings

It gives a json response which contains image url

Picasso.with(context).load(iconUrl)
      .into(holder.previewImageView, new  com.squareup.picasso.Callback() {
                  @Override
                    public void onSuccess() {
                          Log.d(“Sucess”,“image loaded successfully”);
                   }
                   @Override
                    public void onError() {
                       holder.previewImageView.setVisibility(View.GONE);
                     }
                });     

Here also com.squareup.picasso.Callback is use to find that image is loaded successfully or not.

RSS

When we query any like “dhoni” we get ‘link’ in reply from server

“title”: “Dhoni”,

“description”: “”,
 “link”: “http://de.wikipedia.org/wiki/Dhoni”

We use this link in android-link-preview library to retrieve relevant image url and then Picasso use this url to load image into imageview previewImageView.

Picasso.with(currContext).load(imageList.get(0))
       .fit().centerCrop()
       .into(previewImageView);

Reference

Continue ReadingUsing Picasso to Show Images in SUSI Android
Read more about the article Implementing Text-to-Speech (TTS) in SUSI Android

Implementing Text-to-Speech (TTS) in SUSI Android

Mobile assistants are designed to perform tasks that the user “commands” through by chat UI or speech. The Android OS already provides Text to speech (TTS) and Speech to text (STT) features. This feature is available from Android version 1.6 onward. In this blog post I will show how tts is implemented in SUSI Android and how I fix the issue ‘delay in speech response’.

TextToSpeech class controls the tts engine. To use TextToSpeech class import it in the activity where you want to use text to speech feature.

import android.speech.tts.TextToSpeech;

After you import TextToSpeech class now we need to initialize TextToSpeech

TextToSpeech tts = new TextToSpeech(this,this);

Here first parameter is the Context and the other one is the listener. The listener is  use  to  inform our app that the engine is ready to use. In order to be notified we have to  implement  TextToSpeech.OnInitListener.

TextToSpeech.OnInitListener listener = new  TextToSpeech.OnInitListener {
@Override
public void onInit(int status) {
if (status == TextToSpeech.SUCCESS)
tts.setLanguage(Locale.UK/* set the default language*/);
}
}

Hence the engine can be initialized asIf status is success then, it means that TTS is initialized successfully and now we can use it. Otherwise, we can’t use it. setLanguage method is used to set language in which we want reply.

TextToSpeech tts = new TextToSpeech(getApplicationContext,listener)

When you use TTS one thing you have to remember that TTS run  on main thread so sometimes it may cause delays in text to speech conversion or it may block UI for a while. It is better to wrap it like below code.

new Handler().post(new Runnable() {
      @Override
       public void run() {
         tts = new TextToSpeech(getApplicationContext(), listener);
        }
    });

Now our engine is ready to speak, we need simply pass the string we want to read.

tts.speak(“text to read”,TextToSpeech.QUEUE_FLUSH, null, null);

But before tts.speak, it is important to check for the audio focus change request. It is important because only one audio source can have focus at a time. You can check it using below code.

private AudioManager.OnAudioFocusChangeListener afChangeListener =
            new AudioManager.OnAudioFocusChangeListener() {
                 public void onAudioFocusChange(int focusChange) {
                                                        //check for focus
                                                   }
                                           };

OnAudioFocusChangeListener is called when audio focus of the system is changed and according to value of focusChange either we stop TTS or keep using it.

AudioManager audiofocus = (AudioManager)                                    getSystemService(Context.AUDIO_SERVICE);

audiofocus is instance of AudioManager class. We need it to call requestAudioFocus method of AudioManager class. requestAudioFocus method returns the status of request for audio focus change. This method requires three parameter  instance of AudioManager.OnAudioFocusChangeListener, stream type and duration hint. If request is granted only then we can we can use tts.speak .

int result = audiofocus.requestAudioFocus(afChangeListener,AudioManager.STREAM_MUSIC, AudioManager.AUDIOFOCUS_GAIN);

if (result == AudioManager.AUDIOFOCUS_REQUEST_GRANTED) {

tts.speak(“text to read”,TextToSpeech.QUEUE_FLUSH, null, null);

}

We were continuously facing issue ‘delay in speech response’ because voiceReply method implementation was wrong. We were initializing TextToSpeech on each call of voiceReply method and since onInit method runs on main thread causing delay in voice response. So I removed it and instead of initializing tts each time I used the tts instance already initialized when activity create.

 String spoken = reply;

textToSpeech.speak(spoken, TextToSpeech.QUEUE_FLUSH, null);

You can also control how the engine read text. Like we can modify pitch and speech rate.

tts.setPitch((float)pitch);

tts.setSpeechRate((float)speed);

Resource

Continue ReadingImplementing Text-to-Speech (TTS) in SUSI Android

How to keep crash records of SUSI.AI Android with Crashlytics

At this stage of the development of SUSI.AI Android there are many changes and at times this results in inconsistencies and crashes of the app. One important questions we face is how to keep record of crashes so that we can improve our app. Using Crashlytics is a way keep record of crashes. The easiest way to add crashlytics in an app is to integrate the fabric plugin in Android Studio.

  • First create an account at fabric.
  • When you create account it will send you confirmation mail.
  • After clicking confirmation mail it will redirect you to fabric page.
  • It show you different platform option. Select android as platform.

  • For window/linux user select setting from file menu. For Mac user select Preferences from menu.
  • Select Plugins, click the Browse repositories button and search for “Fabric for Android”
  • Click install plugin button to download and install plugin.
  • You can see Fabric option in right side. Click on it and enter your credentials to sign in.
  • Select susi_android project and click next.
  • Fabric will list all the organizations you registered, so select the organization you want to associate the app with and click next. In my case organization is susi. 
  • Fabric will then list all of its kits.select Crashlytics and click  Next .
  •  Click the install button.It will add crashlytics in project.
  • Fabric wants to make changes in MainApplication  and AndroidManifest.xml files , so click the Apply button for the changes to happen.

  • Build and run your application to make sure that everything is configured properly. If your app was successfully configured, you will get an email sent instantly to the email address you used to sign up with fabric.
  • Now you can track crashes of your app on the dashboard of your  fabric account.
  • It will give you all details like 1.) How many users are affected and how many times app crashes with date. 2.) Details of  devices in which app crashes . 3.) Cause of errors

For more information use these links:

https://fabric.io/home

https://fabric.io/kits/android/crashlytics

Continue ReadingHow to keep crash records of SUSI.AI Android with Crashlytics