SUSI.AI

Read more about the article Showing “Get started” button in SUSI Viber bot

Showing “Get started” button in SUSI Viber bot

When we start a chat with SUSI.AI on Viber i.e. SUSI Viberbot, there should be an option on how to get started with the bot. The response to it are some options like “Visit repository”, “How to contribute” which direct the user to check how SUSI.AI bot is made and prompts him/her to contribute to it. Along with that an option of “start chatting” can be shown to add up some sample queries for the user to try. To accomplish the task at hand, we will accomplish these sub tasks: To show the “Get started” button. To show the reply to “Get started” query. To respond to the queries, nested in the response of “Get started” Showing “Get started”: The Viber developers platform notifies us when a user starts a conversation with our bot. To be exact, a conversation_started event is sent to our webhook and can be handled accordingly. The Viberbot shows a welcome message to the user along with a Get started button to help him/her start. To send just the welcome message: if (req.body.event === 'conversation_started') { // Welcome Message var options = { method: 'POST', url: 'https://chatapi.viber.com/pa/send_message', headers: headerBody, body: { // some required body properties here text: 'Welcome to SUSI.AI!, ' + req.body.user.name + '.', // code for showing the get started button here. } json: true }; request(options, function(error, res, body) { // handle error }); } The next step is to show the “Get started” button. To show that we use a keyboard tool, provided by Viber developers platform. So after the “text” key we have the “keyboard” key and a value for it: keyboard: { "Type": "keyboard", "DefaultHeight": true, "Buttons": [{ "ActionType": "reply", "ActionBody": "Get started", }] } The action type as shown in the code, can be “reply” or “open-url”. The “reply” action type, triggers an automatic query sent back with “Get started” (i.e. the value of "ActionBody" key), when that button gets clicked. Hence, this code helps us tackle our first sub task: Reply to “Get started”: We target to make each SUSI.AI bot generic. The SUSI FBbot and SUSI Tweetbot shows some options like “Visit repository”, “Start chatting” and “How to contribute?” for the “Get started” query. We render the same answer structure in Viberbot. The “rich_media” type helps us send buttons in our reply message. As we ought to use three buttons in our message, the button rows are three in the body object: if(message === "Get started"){ var options = { method: 'POST', url: 'https://chatapi.viber.com/pa/send_message', headers: headerBody, body: { // some body object properties here type: 'rich_media', rich_media: { Type: "rich_media", ButtonsGroupColumns: 6, ButtonsGroupRows: 3, BgColor: "#FFFFFF", Buttons: buttons } }, json: true }; request(options, function(error, res, body) { if (error) throw new Error(error); console.log(body); }); As said before, 2 type of Action types are available - “open-url” and “reply”. “Visit repository” button has an “open-url” action type and “How to contribute?” or “start chatting” has a “reply” action type. Example of “Visit repository” button: var buttons…

Continue ReadingShowing “Get started” button in SUSI Viber bot
Read more about the article Creating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin

Creating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin

An Android application often includes settings that allow the user to modify features of the app. For example, SUSI Android app allows users to specify whether they want to use in built mic to give speech input or not. Different settings in SUSI Android app and their purpose are given below Setting                                        Purpose Enter As Send It allows users to specify whether they want to use enter key to send message or to add new line. Mic Input It allows users to specify whether they want to use in built mic to give speech input or not. Speech Always It allows users to specify whether they want voice output in case of speech input or not. Speech Output It allows users to specify whether they want speech output irrespective of input type or not. Language It allows users to set different query language. Reset Password It allows users to change password. Select Server It allows users to specify whether they want to use custom server or not. Android provides a powerful framework, Preference framework, that allows us to define the way we want preferences. In this blog post, I will show you how Settings UI is created using Preference framework and Kotlin in SUSI Android. Advantages of using Preference are: It has own UI so we don‘t have to develop our own UI for it It stores the string into the SharedPreferences so we don’t need to manage the values in SharedPreference. First, we will add the dependency in build.gradle(project) file as shown below. compile 'com.takisoft.fix:preference-v7:25.4.0.3' To create the custom style for our Settings Activity screen we can set android:theme="@style/PreferencesThemeLight" as the base theme and can apply various other modifications and colour over this. By default, it has the usual Day and Night theme with NoActionBar extension. Layout Design I used PreferenceScreen as the main container to create UI of Settings and filled it with the other components. Different components used are following: SwitchPreferenceCompat: This gives us the Switch Preference which we can use to toggle between two different modes in the setting. <com.takisoft.fix.support.v7.preference.SwitchPreferenceCompat android:defaultValue="true" PreferenceCategory: It is used for grouping the preference. For example, Chat Settings, Mic Settings, Speech Settings etc are different groups in settings. ListPreference: This preference display list of values and help in selecting one. For example in setLanguage option ListPreference is used to show a list of query language. List of query language is provided via xml file array.xml (res/values). Attribute android:entries point to arrays languagentries and android:entryValue holds the corresponding value defined for each of the languages. <ListPreference android:title="@string/Language" android:key="Lang_Select" android:entries="@array/languagentries" android:entryValues="@array/languagentry" </ListPreference> Implementation in SUSI Android All the logic related to Preferences and their action is written in ChatSettingsFragment class. ChatSettingsFragment extends PreferenceFragmentCompat class. class ChatSettingsFragment : PreferenceFragmentCompat() Fragment populate the preferences when created. addPreferencesFromResource method is used to inflate view from xml. addPreferencesFromResource(R.xml.pref_settings) Reference Main site link of PreferenceScreen by Google: https://developer.android.com/reference/android/preference/PreferenceScreen.html Main site link of PreferenceFragmentCompat: https://developer.android.com/reference/android/support/v7/preference/PreferenceFragmentCompat.html Article with example on android settings using PreferenceFragmentCompat: https://storiesandroid.wordpress.com/2015/10/06/android-settings-using-preference-fragments/ Article by Jakob Ulbrich on…

Continue ReadingCreating Settings Screen in SUSI Android Using PreferenceActivity and Kotlin
Read more about the article Implement Internationalization in SUSI Android With Weblate

Implement Internationalization in SUSI Android With Weblate

When you build an Android app, you must consider about users for whom you are building an app. It may be possible that you users are from the different region. To support the most users your app should show text in locale language so that user can use your app easily. Our app SUSI Android is also targeting users from different regions. Internationalization is a way that ensures our app can be adapted to various languages without requiring any change to source code. This also allows projects to collaborate with non-coders more easily and plugin translation tools like Weblate. Benefits of using Internationalization are: It reduces the time for localization i.e it will localize your app automatically. It helps us to keep and maintain only single source code for different regions. To achieve Internationalization in Android app you must follow below steps: Move all the required contents of your app’s user interface into the resource file. Create new directories inside res to add support for Internationalization. Each directory’s name should follow rule <resource type>-(language code). For example values-es contains string resource for es language code i.e Spanish. Now add different locale content in the respective folder. We need to create separate directories for different locale because to show locale specific content, Android check specific folder i.e res/<resource type>-(language code) like res/values-de and show content from that folder. That’s why we need to move all the required content into resource file so that each required content can be shown in the specific locale. How Internationalization is implemented in SUSI Android In SUSI Android there is not any locale specific image but only string. So I created only locale specific value resource folder to add locale specific strings. To create locale specific values folder I follow the above-mentioned rule i.e <resource type>-(language code). After that, I added specific language string in the respective folder. Instead of hard-coded strings, we used strings from string.xml file so that it will change automatically according to the region. android:text="@string/reset" and showToast(getString(R.string.wrong_password)) In absence of resource directory for any specific locale, Android use default resource directory. Integrate Weblate in SUSI Android Weblate is a web based translation tool. The best part of Weblate is its tight version control integration which makes it easy for translators to contribute because translator does not need to fork your repo and send pull request for each change but Weblate handle this part i.e translator translate strings of the project in Weblate site and Weblate will send pull request for those changes. Weblate can host your free software projects for free but it depends on them. Here is the link of SUSI Android project hosted on Weblate. If your project is good then they can host your project for free. But for that, you have to apply from this link and select ask for hosting. Now fill up form as shown in below picture. Once your project is hosted on Weblate, they will email you about it. After that, you have to integrate…

Continue ReadingImplement Internationalization in SUSI Android With Weblate
Read more about the article How to use Realm in SUSI Android to Save Data

How to use Realm in SUSI Android to Save Data

Sometimes we need to store information on the device locally so that we can use information offline and also query data faster. Initially, SQLite was only option to store information on the device. But working with SQLite is difficult sometimes and also it makes code difficult to understand. Also, SQL queries take a long time. But now we have realm a better alternative of SQLite. The Realm is a lightweight mobile database and better substitute of SQLite. The Realm has own C++ core and store data in a universal, table-based format by a C++ core. This allows Realm to allow data access from multiple languages as well as a range of queries. In this blog post, I will show you why we used Realm and how we save data in SUSI Android using Realm. “How about performance? Well, we’re glad you asked :) For all the API goodness & development productivity we give you, we’re still up to 100x faster than some SQLite ORMs and on average ~10x faster than raw SQLite and common ORMs for typical operations.” (compare: https://blog.realm.io/realm-for-android/) Advantages of Realm over SQLite are following: It is faster than SQLite as explained on the Realm blog. One of the reasons realm is faster than SQLite is, the traditional SQLite + ORM abstraction is leaky because ORM simply converts  Objects and their methods into SQL statements. Realm, on the other hand, is an object database, meaning your objects directly reflect your database. It is easier to use as it uses objects for storing data. When we use SQLite we need boilerplate code to convert values to and from the database, setting up mappings between classes and tables, fields and columns, foreign keys, etc. Whereas in Realm data is directly exposed as objects and can be queried without any conversion. Prerequisites To include this library in your project you need Android studio version 1.5.1 or higher. JDK version 7.0 or higher. Android API level 9 or higher. How to use realm in Android To use Realm in your project we must add the dependency of the library in build.gradle(project) file   dependencies {        classpath "io.realm:realm-gradle-plugin:3.3.1"    } and build.gradle(module) file. apply plugin: 'realm-android' dependencies { compile 'io.realm:android-adapters:1.3.0’ } Now you have to instantiate Realm in your application class. Setting a default configuration in your Application class, will ensure that it is available in the rest of your code. RealmConfiguration realmConfiguration = new RealmConfiguration.Builder(this)                                                               .deleteRealmIfMigrationNeeded().build(); Realm.setDefaultConfiguration(realmConfiguration); Now we need to create a model class. A model class is use to save data in Realm and retrieve saved data and it must extend RealmObject class. For eg. public class Person extends RealmObject {    private String name;    public String getName() {        return name;    }    public void setName(String name) {        this.name = name;    } } Field in the model class uses to define columns. For eg. ‘name’ is a column name. Method like setName() use to save data  and getName() use to retrieve saved data. Now create an instance of the Realm in the activity where you…

Continue ReadingHow to use Realm in SUSI Android to Save Data
Read more about the article List all the Users Registered on SUSI.AI

List all the Users Registered on SUSI.AI

In this blog, I'll be telling on how SUSI admins can access list of all the registered users from SUSI-server. Following this, they may modify/edit user role of any registered user. What is User Role? A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely : BOT ANONYMOUS USER   REVIEWER ACCOUNTCREATOR ADMIN BUREAUCRAT * Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely. All the users who are not logged in but interacting with SUSI are anonymous users. These are only subject to chat with SUSI, login, signup or may use forgot password service. Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them. Privileged users are those who have special rights with them. These are more like moderators with much special rights than any other user. At the top level of the hierarchy are the admins. These users have more rights than anyone. They can change role of any other user, override decision of any privileged user as well. Let us now look at the control flow of this. First things first, make a component of User List in the project. Let us name it ListUsers and since it has to be accessible by those users who possess ADMIN rights, you will find it enclosed in Admin package in components folder. Open up index.js file, import Listusers component  and add route to it in the following way : ...//other import statements import ListUser from "./components/Admin/ListUser/ListUser"; ...//class definition and other methods <Route path="/listUser" component={ListUser}/> …//other routes defined Find a suitable image for “List Users” option and add the option for List Users in static appbar component along with the image. We have used Material UI’s List image in our project. ...// other imports import List from 'material-ui/svg-icons/action/list'; …Class and method definition <MenuItem primaryText="List Users" onTouchTap={this.handleClose} containerElement={<Link to="/listUser" />} rightIcon={<List/>} /> ...//other options in top right corner menu Above code snippet will add an option to redirect admins to ‘/listUsers’ route. Let us now have a closer look at functionality of both client and server. By now you must have known what ComponentDidMount does. {If not, I’ll tell you. This is a method which is given first execution after the page is rendered. For more information, visit this link}. As mentioned earlier as well that this list will be available only for admins and may be even extended for privileged users but not for anonymous or any other user, an AJAX call is made to server in ComponentDidMount of ‘listuser’ route which returns the base user role of current user. If user is an Admin, another method,…

Continue ReadingList all the Users Registered on SUSI.AI
Read more about the article SUSI.AI User Roles and How to Modify Them

SUSI.AI User Roles and How to Modify Them

In this blog, I discuss what is ‘user-role’ in SUSI.AI, what are the various roles and how SUSI admins can modify/update a user’s roles. What is User Role? A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely : BOT ANONYMOUS USER   REVIEWER ACCOUNTCREATOR ADMIN BUREAUCRAT * Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely. If SUSI is active as a bot on some bot integrated platform (like line or kik), the user role assigned to it will be that of BOT. This user role just has technical access to the server. All the users who are not logged in but interacting with SUSI are ANONYMOUS users. These are only subject to chat, login and signup. They may use forgot password service and reset password services as well. Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them as USER(s). Users with role assigned as “REVIEWERS” are expected to manage the Skill CMS. There might be some dispute or conflict in a skill. REVIEWERS then take the access of skill data and finalise the conflict there itself for smooth functionality. ADMIN users are those who have special rights with them. These are more like moderators with much special rights than any other user. At the top level of the hierarchy are the BUREAUCRATS. These users have more rights than anyone. They can change role of any other user, override decision of any ADMIN user as well. Both admins and bureaucrats have the access to all the settings file on the server. They not only can look at the list, but also download and upload them. Now these users also have right to upgrade or downgrade any other user as well. All these user roles are defined in UserRole.java file. In each request received by the server, the user role of user making the request is compared with the minimal user role in getMinimalUserRole() method. This method is defined in AbstractAPIHandler which validates if a user is allowed to access a particular servlet or not. private void process(HttpServletRequest request, HttpServletResponse response, Query query) throws ServletException, IOException { // object initialisation and comparsions // user authorization: we use the identification of the user to get the assigned authorization Authorization authorization = DAO.getAuthorization(identity); if (authorization.getUserRole().ordinal() < minimalUserRole.ordinal()) { response.sendError(401, "Base user role not sufficient. Your base user role is '" + authorization.getUserRole().name() + "', your user role is '" + authorization.getUserRole().getName() + "'"); return; } // evaluations based on other request parameters. } Now that we know about what User Roles actually are, let us look at how…

Continue ReadingSUSI.AI User Roles and How to Modify Them
Read more about the article Auto Updating SUSI Android APK and App Preview on appetize.io

Auto Updating SUSI Android APK and App Preview on appetize.io

This blog will cover the way in which the SUSI Android APK is build automatically after each commit and pushed to “apk” branch in the github repo. Other thing which will be covered is that how the app preview on appetize.io can be updated after each commit. This is basically for the testers who wish to test the SUSI Android App. There are four ways to test the SUSI Android App. One is to simply download the alpha version of the app from the Google PlayStore. Here is the link to the app. Join the alpha testing and report bugs on the github issue tracker of the repo. Other way is to build the app from Android Studio but you may need to set the complete project. If you are looking to contribute in the project, this is the advised way to test the app. The other two ways are explained below. Auto Building of APK and pushing to “apk” branch We have written a script which does following steps whenever a PR is merged: Checks if the commit is of a PR or a commit to repo If not of PR, configures a user whose github account will be used to push the APKs. Clones the repo, generates the debug and release APK. Deletes everything in the apk branch. Commits and Pushes new changes to apk branch. This script is written for people or testers who do not have android studio installed in their computer and want to test the app. So, they can directly download the apk from the apk branch and install it in their phone. The APK is always updated after each commit. So, whenever a tester downloads the APK from apk branch, he will always get the latest app. if [[ $CIRCLE_BRANCH != pull* ]] then git config --global user.name "USERNAME" git config --global user.email "EMAIL" git clone --quiet --branch=apk https://USERNAME:$GITHUB_API_KEY@github.com/fossasia/susi_android apk > /dev/null ls cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-debug.apk apk/susi-debug.apk cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-release-unsigned.apk apk/susi-release.apk cd apk git checkout --orphan workaround git add -A git commit -am "[Circle CI] Update Susi Apk" git branch -D apk git branch -m apk git push origin apk --force --quiet > /dev/null fi Auto Updating of App Preview on appetize.io The APKs generated in the above step can now be used to set up the preview of the app on the appetize.io. Appetize.io is an online simulator to run mobile apps ( IOS and Android). Appetize.io provides a nice virtual mobile frame to run native apps with various options like screen size, mobile, OS version, etc. Appetize.io provides some API to update/publish the app. In SUSI, we once uploaded the app on appetize.io and now we are using the API provided by them to update the APK everytime a commit is pushed in the repository. API information (Derived from official docs of appetize.io): You may upload a new version of an existing app, or update app settings. Send an HTTP POST request to https://APITOKEN@api.appetize.io/v1/apps/PUBLICKEY Replace APITOKEN with your API token and…

Continue ReadingAuto Updating SUSI Android APK and App Preview on appetize.io
Read more about the article Showing location response in SUSI.AI bots

Showing location response in SUSI.AI bots

SUSI.AI has a capability to tell the basic information about a location, it is asked for. Along with the basic information about that place, it shows a map (i.e. open street map) pointing to that location. The task at hand is to inculcate this “location” feature to the SUSI.AI messenger bots. The SUSI Tweetbot and SUSI Fbbot are used as examples in this blog. Let’s first check on what response do we get, when a person asks a query like “where is london” to the SUSI API. Along with the basic information about that location, the result as shown below has the type of reply (i.e. map), latitude, longitude and a link to the open street map. "actions": [ { "type": "answer", "language": "en", "expression": "Ludhiana is a city and a municipal corporation in Ludhiana district in the Indian state of Punjab, and is the largest city north of Delhi." }, { "type": "anchor", "link": "https://www.openstreetmap.org/#map=13/30.912040570244187/75.85379021980509", "text": "Here is a map", "language": "en" }, { "type": "map", "latitude": "30.912040570244187", "longitude": "75.85379021980509", "zoom": "13", "language": "en" } ] The response for a location type query has these 3 main parts: Clickable static map image. A basic information of the place asked about. The link i.e. when the static map image is clicked it should redirect to the corresponding map location. Let’s try to make up with the 1st part of the response i.e. Static map image. The map quest API is used to result in a static map image of the location. We need an API key to access the map quest API, which can be requested from their developer site. Along with that we need the latitude and longitude of the location at hand. The SUSI API’s response helps us to get the latitude value: // if body represents the response object var lat = body.answers[0].actions[2].latitude; And the longitude value: var lon = body.answers[0].actions[2].longitude; Using the three values that are API key, latitude and longitude, the static image is rendered by this link: var static_image_url = "https://open.mapquestapi.com/staticmap/v4/getmap?key=API_KEY&size=600,400&zoom=13&center="+lat+","+lon; The second part is, basic information about the place asked, can be fetched from: // if body is the JSON response object from SUSI API var mapMessage = body.answers[0].actions[0].expression; The link to the map location can be easily fetched from the SUSI API’s response: var link = body.answers[0].actions[1].link; As all the three parts are ready, let’s look on how to render them on the SUSI.AI bot’s screen. Facebook: Sending a generic template message object: message = { "type":"template", "payload":{ "template_type":"generic", "elements":[{ "title": mapMessage, "image_url": static_image_url, "Item_url": link }] } }; sendTextMessage(sender, message, 1); Twitter: The Twitter API does not need a static image of the map to be rendered. It does that work for us. We just need to send an event to the Twitter API with the message data object constituting of the map message, the latitude and longitude values: "message_data": { "text": mapMessage, "attachment": { "type": "location", "location": { "type": "shared_coordinate", "shared_coordinate": { "coordinates": { "type": "Point", "coordinates": [lon, lat] }…

Continue ReadingShowing location response in SUSI.AI bots
Read more about the article Creating GUI for configuring SUSI Linux Settings

Creating GUI for configuring SUSI Linux Settings

SUSI Linux app provides access to SUSI on Linux distributions on desktop as well as hardware devices like Raspberry Pi. The settings for SUSI Linux are controlled with the use of a config.json file. You may edit the file manually, but to provide safe configurations, we have a config generator script. You may run the script to configure settings like TTS Engine, STT Engine, authentication, choice about the hotword engine etc. Generally, it is easier to configure application settings through a GUI. Thus, we added a GUI for it using PyGTK and Glade. Glade is a GUI designer for GNOME based Linux systems. I wrote a blog about how to create user interfaces in Glade and access it from Python code in SUSI Linux. Now, for creating UI for Configuration screen, we need to choose an ideal layout. Glade provides various choices like BoxLayout, GridLayout, FlowBox, ListBox , Notebook etc. Since, we need to display only basic settings options, we select the BoxLayout for this purpose. BoxLayout as the name suggests, forms a box like arrangement for widgets. You can arrange the widgets in either Landscape or Horizontal Layout. We select Application Window as a top-level container and add a BoxLayout container in it. Now, in each box of the BoxLayout, we need to add the widgets like ComboBox and Switch for user’s choice and a Label. This can be done by using a horizontal BoxLayout with corresponding widgets. After arranging the UI in above described manner, we have a GUI like below. If you see the current window in the preview now, you will find that the ComboBox do not have any items. We need to define items in the ComboBox using a GTKListStore. You may refer to this video tutorial to see how this can be done. Now, when we see the preview, our GUI is fully functional. We have options for Speech Recognition Service, Text to Speech Service in ComboBox. Other simple settings are available as switches. Now, we need to add functionality to our UI. We want our code to be modular and structured, therefore, we declare a ConfigurationWindow class. Though the ideal way to handle such cases is inheriting from the Gtk.Window class, but reading the documentation of PyGTK+ 3, I could not find a way to do this for windows created through Glade. Thus, we will use composition for storing the window object. We add window and other widgets present in the UI as properties of ConfigurationWindow class like this. class ConfigurationWindow: def __init__(self) -> None: super().__init__() builder = Gtk.Builder() builder.add_from_file(os.path.join("glade_files/configure.glade")) self.window = builder.get_object("configuration_window") self.stt_combobox = builder.get_object("stt_combobox") self.tts_combobox = builder.get_object("tts_combobox") self.auth_switch = builder.get_object("auth_switch") self.snowboy_switch = builder.get_object("snowboy_switch") self.wake_button_switch = builder.get_object("wake_button_switch") Now, we need to connect the Signals from our configuration window to the Handler. We declare the Handler as a nested class in the ConfigurationWindow class because its scope of usage is inside the ConfigurationWindow object. Then you may connect signals to an object of the Handler class. builder.connect_signals(ConfigurationWindow.Handler(self)) Since we may need to modify…

Continue ReadingCreating GUI for configuring SUSI Linux Settings