How to Store Mobile Settings in the Server from SUSI Web Chat Settings Page

While we are adding new features and capabilities to SUSI Web Chat application, we wanted to provide settings changing capability to SUSI users. SUSI team decided to maintain a settings page to give that capability to users.

This is how it’s interface looks like now.

In this blog post I’m going to add another setting category to our setting page. This one is for  saving mobile phone number and dial code in the server.

UI Development:

First we need to  add new category to settings page and it should be invisible when user is not logged in. Anonymous users should not get mobile phone category in settings page.

     let menuItems = cookies.get('loggedIn') ?
            <div>
                <div className="settings-list">
                    <Menu
                        onItemTouchTap={this.loadSettings}
                        selectedMenuItemStyle={blueThemeColor}
                        style={{ width: '100%' }}
                        value={this.state.selectedSetting}
                    >
                       <MenuItem value='Mobile' className="setting-item" leftIcon={<MobileIcon />}>Mobile<ChevronRight className="right-chevron" /></MenuItem>
                        <hr className="break-line" />
                    </Menu>
                </div>
            </div>

 

Next we have to show settings UI when user clicks on the category name.

 if (this.state.selectedSetting === 'Mobile' && cookies.get('loggedIn')) {}
                currentSetting = (
  <Translate text="Country/region : " />
                            <DropDownMenu maxHeight={300}
               value={this.state.countryCode?this.state.countryCode:'US'}

 

Show US if the state does not deines the country code

                                onChange={this.handleCountryChange}>
                                {countries}
                            </DropDownMenu>
<Translate text="Phone number : " />
                            <TextField name="selectedCountry"
                            disabled={true}
                            value={countryData.countries[this.state.countryCode?this.state.countryCode:'US'].countryCallingCodes[0] }
                         	/>
                            <TextField name="serverUrl"
                                onChange={this.handleTelephoneNoChange}
                                value={this.state.phoneNo }
 />
)}

 

Then we need to get list of country names and country dial codes to show in the above drop down. We used country-data node module for that.

To install country-data module use this  command.

npm install --save country-data

 

We have used it in the settings page as below.

import countryData from 'country-data';
    	countryData.countries.all.sort(function(a, b) {
            if(a.name < b.name){ return -1};
            if(a.name > b.name){ return 1};
            return 0;
        });
        let countries = countryData.countries.all.map((country, i) => {
         	return (<MenuItem value={countryData.countries.all[i].alpha2} key={i} primaryText={ countryData.countries.all[i].name+' '+ countryData.countries.all[i].countryCallingCodes[0] } />);
        });

 

First we sort the country data list from it’s name. After that we made a list of “”s from this list of data.
Then we have to check whether the user changed or added the phone number and region (dial code).
It handles by this function mentioned above. ( onChange={this.handleCountryChange}> and
onChange={this.handleTelephoneNoChange} )

    handleCountryChange = (event, index, value) => {
        this.setState({'countryCode': value });
    }

 

Then we have to get the phone number using below function.

    handleTelephoneNoChange = (event, value) => {
        this.setState({'phoneNo': value});
    }

 

Next we have to update the function that triggers when user clicks the save button.

    handleSubmit = () => {
        let newCountryCode = !this.state.countryCode?
        this.intialSettings.countryCode:this.state.countryCode;
        let newCountryDialCode = !this.state.countryDialCode?
        this.intialSettings.countryDialCode:this.state.countryDialCode;
        let newPhoneNo = this.state.phoneNo;
        let vals = {
            countryCode: newCountryCode,
            countryDialCode: newCountryDialCode,
            phoneNo: newPhoneNo
}
let settings = Object.assign({}, vals);
cookies.set('settings', settings);
 this.implementSettings(vals);
 }

 

This code snippet stores Country Code, Country Dial code and phone no in the server.
Now we have to update the Store. Here we are going to change UserPreferencesStore “UserPreferencesStore” .
First we have to setup default values for things we are going to store.

let _defaults = {
	  CountryCode: 'US',
   	  CountryDialCode: '+1',
   	  PhoneNo: ''
}

 

Finally we have to update the dispatchToken to change and get these new data

UserPreferencesStore.dispatchToken = ChatAppDispatcher.register(action => {
   switch (action.type) {
       case ActionTypes.SETTINGS_CHANGED: {
           let settings = action.settings;
           if(settings.hasOwnProperty('theme')){
                   _defaults.Theme = settings.theme;
           }
           if(settings.hasOwnProperty('countryDialCode')){
               _defaults.countryDialCode = settings.countryDialCode;
           }
           if(settings.hasOwnProperty('phoneNo')){
               _defaults.phoneNo = settings.phoneNo;
           }
           if(settings.hasOwnProperty('countryCode')){
               _defaults.countryCode = settings.countryCode;
           }
           UserPreferencesStore.emitChange();
           break;
}
}

 

Finally application is ready to store and update Mobile phone number and region code in the server.

Resources:

SUSI.AI Chrome Bot and Web Speech: Integrating Speech Synthesis and Recognition

Susi Chrome Bot is a Chrome extension which is used to communicate with Susi AI. The advantage of having chrome extensions is that they are very accessible for the user to perform certain tasks which sometimes needs the user to move to another tab/site.

In this blog post, we will be going through the process of integrating the web speech API to SUSI Chromebot.

Web Speech API

Web Speech API enables web apps to be able to use voice data. The Web Speech API has two components:

Speech Recognition:  Speech recognition gives web apps the ability to recognize voice data from an audio source. Speech recognition provides the speech-to-text service.

Speech Synthesis: Speech synthesis provides the text-to-speech services for the web apps.

Integrating speech synthesis and speech recognition in SUSI Chromebot

Chrome provides the webkitSpeechRecognition() interface which we will use for our speech recognition tasks.

var recognition = new webkitSpeechRecognition();

 

Now, we have a speech recognition instance recognition. Let us define necessary checks for error detection and resetting the recognizer.

var recognizing;

function reset() {
recognizing = false;
}

recognition.onerror = function(e){
console.log(e.error);
};

recognition.onend = function(){
reset();
};

 

We now define the toggleStartStop() function that will check if recognition is already being performed in which case it will stop recognition and reset the recognizer, otherwise, it will start recognition.

function toggleStartStop() {
    if (recognizing) {
      recognition.stop();
      reset();
    } else {
      recognition.start();
      recognizing = true;
    }
}

 

We can then attach an event listener to a mic button which calls the toggleStartStop() function to start or stop our speech recognition.

mic.addEventListener("click", function () {
    toggleStartStop();
});

 

Finally, when the speech recognizer has some results it calls the onresult event handler. We’ll use this event handler to catch the results returned.

recognition.onresult = function (event) {
    for (var i = event.resultIndex; i < event.results.length; ++i) {
      if (event.results[i].isFinal) {
        textarea.value = event.results[i][0].transcript;
        submitForm();
      }
    }
};

 

The above code snipped tests for the results produced by the speech recognizer and if it’s the final result then it sets textarea value with the result of speech recognition and then we submit that to the backend.

One problem that we might face is the extension not being able to access the microphone. This can be resolved by asking for microphone access from an external tab/window/iframe. For SUSI Chromebot this is being done using an external tab. Pressing on the settings icon makes a new tab which then asks for microphone access from the user. This needs to be done only once, so that does not cause a lot of trouble.

setting.addEventListener("click", function () {
chrome.tabs.create({
url: chrome.runtime.getURL("options.html")
});
});navigator.webkitGetUserMedia({
audio: true
}, function(stream) {
stream.stop();
}, function () {
console.log('no access');
});

 

In contrast to speech recognition, speech synthesis is very easy to implement.

function speakOutput(msg){
    var voiceMsg = new SpeechSynthesisUtterance(msg);
    window.speechSynthesis.speak(voiceMsg);
}

 

This function takes a message as input, declares a new SpeechSynthesisUtterance instance and then calls the speak method to convert the text message to voice.

There are many properties and attributes that come with this speech recognition and synthesis interface. This blog post only introduces the very basics.

Resources

 

Store User’s Personal Information with SUSI

In this blog, I discuss how SUSI.AI stores personal information of it’s users. This personal information is mostly about usernames/links to different websites like LinkedIn, GitHub, Facebook, Google/Gmail etc. To store such details, we have a dedicated API. Endpoint is :

https://api.susi.ai/aaa/storePersonalInfo.json

In this API/Servlet, storing the details and getting the details, both the aspects are covered. At the time of making the API call, user has an option either to ask the server for a list of available store names along with their values or request the server to store the value for a particular store name. If a store name already exists and a client makes a call with new/updated value, The servlet will update the value for that particular store name.

The reason you are looking at minimal user role as USER is quite obvious, i.e. these details correspond to a particular user. Hence neither we want someone writing such information anonymously nor we want this information to be visible to anonymous user until allowed by the user.

In the next steps, we start evaluating the API call made by the client. We look at the combination of the parameters present in the request. If the request is to fetch list of available stores, server first checks if Accounting object even has a JSONObject for “stores” or not. If not found, it sends an error message “No personal information is added yet.” and error code 420. Prior to all these steps, server first generates an accounting object for the user. If found, details are encoded as JSONObject’s parameter. Look at the code below to understand things fairly.

Accounting accounting = DAO.getAccounting(authorization.getIdentity());
        if(post.get("fetchDetails", false)) {
            if(accounting.getJSON().has("stores")){
                JSONObject jsonObject = accounting.getJSON().getJSONObject("stores");
                json.put("stores", jsonObject);
                json.put("accepted", true);
                json.put("message", "details fetched successfully.");
                return new ServiceResponse(json);
            } else {
                throw new APIException(420, "No personal information is added yet.");
            }
        }

If the request was not to fetch the list of available stores, It means client wants server to save a new field or update a previous value for that of a store name. A combination of If-else evaluates whether the call even contains required parameters.

if (post.get(“storeName”, null) == null) {
throw new APIException(422, “Bad store name encountered!”);
}

String storeName = post.get(“storeName”, null);
if (post.get(“value”, null) == null) {
throw new APIException(422, “Bad store name value encountered!”);
}

If request contains all the required data, then store name & value are extracted as key-value pair from the request.

In the next steps, since the server is expected to store list of the store names for a particular user, First the identity is gathered from the already present authorization object in “serviceImpl” method. If the server finds a “null” identity, It throws an error with error code 400 and error message “Specified User Setting not found, ensure you are logged in”.

Else, server first checks if a JSONObject with key “stores” exists or not. If not, It will create an object and will put the key value pair in the new JSONObject. Otherwise it would anyways do so.

Since these details are for a particular account (i.e. for a particular user), these are placed in the Accounting.json file. For better knowledge, Look at the code snippet below.

if (accounting.getJSON().has("stores")) {
                accounting.getJSON().getJSONObject("stores").put(storeName, value);
            } else {
                JSONObject jsonObject = new JSONObject(true);
                jsonObject.put(storeName, value);
                accounting.getJSON().put("stores", jsonObject);
            }

            json.put("accepted", true);
            json.put("message", "You successfully updated your account information!");
            return new ServiceResponse(json);

Additional Resources :

Enhancing SUSI Desktop to Display a Loading Animation and Auto-Hide Menu Bar by Default

SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop. The benefits of using chat.susi.ai as a submodule is that it inherits all the features that the webapp offers and thus serves them in a nicely build native application.

Display a loading animation during DOM load.

Electron apps should give a native feel, rather than feeling like they are just rendering some DOM, it would be great if we display a loading animation while the web content is actually loading, as depicted in the gif below is how I implemented that.
Electron provides a nice, easy to use API for handling BrowserWindow, WebContent events. I read through the official docs and came up with a simple solution for this, as depicted in the below snippet.

onload = function () {
	const webview = document.querySelector('webview');
	const loading = document.querySelector('#loading');

	function onStopLoad() {
		loading.classList.add('hide');
	}

	function onStartLoad() {
		loading.classList.remove('hide');
	}

	webview.addEventListener('did-stop-loading', onStopLoad);
	webview.addEventListener('did-start-loading', onStartLoad);
};

Hiding menu bar as default

Menu bars are useful, but are annoying since they take up space in main window, so I hid them by default and users can toggle their display on pressing the Alt key at any point of time, I used the autoHideMenuBar property of BrowserWindow class while creating an object to achieve this.

const win = new BrowserWindow({
	
	show: false,
	autoHideMenuBar: true
});

Resources

1. More information about BrowserWindow class in the official documentation at electron.atom.io.
2. Follow a quick tutorial to kickstart creating apps with electron at https://www.youtube.com/watch?v=jKzBJAowmGg.
3. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.

Enhancing Settings Menu in SUSI Webchat using Material-UI Menu React Component

Material-UI is a great library for react developers since you can directly use the material components in your app. The SUSI.AI Webchat uses Material-UI (https://github.com/callemall/material-ui). In this blog we’ll see how a Menu component is implemented in settings page of susi’s web chat app.

Menu & MenuItem Component

Menu component allows you execute an action on selecting from a list. In the settings menu we want to change the state variable to which setting details is binded (ie. on change of state variable the setting details corresponding to selected menu changes).

A simple menu component is defined like this –

<Menu>
        <MenuItem primaryText="Item 1" />
        <MenuItem primaryText="Item 2" />
        <MenuItem primaryText="Item 3" />
        <MenuItem primaryText="Item 4" />
</Menu>

Menu component and MenuItem accepts some properties too. Like if you add disable={true} then the MenuItem it will disable onClick action of MenuItem.

Adding Icon

MenuItem can contain icons through defining leftIcon and rightIcon properties.

Example snippet:

<MenuItem primaryText="Download" leftIcon={<Download />} />

Output:

You can also add style to the leftIcon or rightIcon in style property of MenuItem.

Active State of MenuItem

You must be interested in assigning a different style to the active MenuItem in the Menu. This can be achieved through selectedMenuItemStyle property. It allows overriding the inline-style property of selected MenuItem.

To implement this we need to use the concept of ‘controlled component’. Each MenuItem has to be assigned a value. Also, assign the Menu with a state variable.

<Menu 
 selectedMenuItemStyle={{color: '#FFFFFF'} }   
 value={this.state.selectedItem} > 
    <MenuItem primaryText="Item 1" value='1'/> 
    <MenuItem primaryText="Item 2" value='2' /> 
</Menu>

This way the state variable will control Menu’s value and the selectedMenuItemStyle property will override the inline-style of the corresponding MenuItem.

Implement onClick function for MenuItem and change the state value.

This way you can add style to active MenuItem.

You can see the demo of how it was implemented in chat settings at https://chat.susi.ai/settings

Also, you can check out the github repo https://github.com/fossasia/chat.susi.ai

Note – Make sure that you define the state change function outside render else it will get a warning like this.

Warning: setState(…): Cannot update during an existing state transition (such as within `render` or another component’s constructor). Render methods should be a pure function of props and state; constructor side-effects are an anti-pattern, but can be moved to `componentWillMount`.

This will result in abnormal behaviour in runtime. So keep that mind in while creating the function to change the state.

Resources

 

List all the Users Registered on SUSI.AI

In this blog, I’ll be telling on how SUSI admins can access list of all the registered users from SUSI-server. Following this, they may modify/edit user role of any registered user.

What is User Role?

A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely :

  • BOT
  • ANONYMOUS
  • USER  
  • REVIEWER
  • ACCOUNTCREATOR
  • ADMIN
  • BUREAUCRAT

* Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely.

All the users who are not logged in but interacting with SUSI are anonymous users. These are only subject to chat with SUSI, login, signup or may use forgot password service. Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them. Privileged users are those who have special rights with them. These are more like moderators with much special rights than any other user. At the top level of the hierarchy are the admins. These users have more rights than anyone. They can change role of any other user, override decision of any privileged user as well.

Let us now look at the control flow of this.

First things first, make a component of User List in the project. Let us name it ListUsers and since it has to be accessible by those users who possess ADMIN rights, you will find it enclosed in Admin package in components folder. Open up

index.js file, import Listusers component  and add route to it in the following way :

...//other import statements
import ListUser from "./components/Admin/ListUser/ListUser";
...//class definition and other methods
<Route path="/listUser" component={ListUser}/>
//other routes defined

Find a suitable image for “List Users” option and add the option for List Users in static appbar component along with the image. We have used Material UI’s List image in our project.

...// other imports

import List from 'material-ui/svg-icons/action/list';

Class and method definition

<MenuItem primaryText="List Users"
          onTouchTap={this.handleClose}
          containerElement={<Link to="/listUser" />}
                rightIcon={<List/>}
      />

...//other options in top right corner menu

Above code snippet will add an option to redirect admins to ‘/listUsers’ route. Let us now have a closer look at functionality of both client and server. By now you must have known what ComponentDidMount does. {If not, I’ll tell you. This is a method which is given first execution after the page is rendered. For more information, visit this link}. As mentioned earlier as well that this list will be available only for admins and may be even extended for privileged users but not for anonymous or any other user, an AJAX call is made to server in ComponentDidMount of ‘listuser’ route which returns the base user role of current user. If user is an Admin, another method, fetchUsers() is called.

let url;
        url = "http://api.susi.ai/aaa/account-permissions.json";
        $.ajax({
            url: url,
            dataType: 'jsonp',
            jsonpCallback: 'py',
            jsonp: 'callback',
            crossDomain: true,
            success: function (response) {
                console.log(response.userRole)
                if (response.userRole !== "admin") {
                    console.log("Not an admin")
                } else {
                    this.fetchUsers();
                    console.log("Admin")
                }
            }.bind(this),
});

In fetchUsers method, an AJAX call is made to server which returns username in JSONArray. The response looks something likes this :

{
	"users" : {
		"email:""[email protected]",
...
},
"Username":["[email protected]", "[email protected]"...]
}

Now, only rendering this data in a systematic form is left. To give it a proper look, we have used material-ui’s table. Import Table, TableBody, TableHeader,

   TableHeaderColumn, TableRow, TableRowColumn from material-ui/table.

In fetchUsers method, response is catched in data Oblect. Now the keys are extracted from the JSON response and mapped with an array. Iterating through array received as username array, we get list of all the registered users. Now, popuulate the data in the table you generated.

return (
                        <TableRow key={i}>
                            <TableRowColumn>{++i}>
                            <TableRowColumn>{name}</TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                            <TableRowColumn> </TableRowColumn>
                        </TableRow>
                    )

Above piece of code may help you while populating the table. These details are returned from susi server which gets a list of all the registered in the following manner. First, it checks if base url of this user is something apart from admin. If not, it returns error which may look like this :

Failed to load resource: the server responded with a status of 401 (Base user role not sufficient. Your base user role is 'ANONYMOUS', your user role is 'anonymous')

Otherwise, it will generate a client identity, use to to get an authorization object which will loop through authorization.json file and return all the users encoded as JSONArray.

Additional Resources

  1. Official Material UI Documentation on Tables from marterial-ui
  2. Answer by Marco Bonelli on Stackoverflow on How to map JSON response in JavaScript?
  3. Answer by janpieter_z on Stackoverflow – on Render JSON data in ReactJS table

SUSI.AI User Roles and How to Modify Them

In this blog, I discuss what is ‘user-role’ in SUSI.AI, what are the various roles and how SUSI admins can modify/update a user’s roles.

What is User Role?

A UserRole defines the servlet access right. Not all users are allowed to access all the data and services. For  example, To list all the users, minimal user role expected is ADMIN. This classification of users are inspired by the wikipedia User Access Levels, see https://en.wikipedia.org/wiki/Wikipedia:User_access_levels.While querying SUSI, Users are classified into 7 different categories, namely :

  • BOT
  • ANONYMOUS
  • USER  
  • REVIEWER
  • ACCOUNTCREATOR
  • ADMIN
  • BUREAUCRAT

* Please see that these are as of the date of publish of this blog. These are subject to change, which is very unlikely.

If SUSI is active as a bot on some bot integrated platform (like line or kik), the user role assigned to it will be that of BOT. This user role just has technical access to the server.

All the users who are not logged in but interacting with SUSI are ANONYMOUS users. These are only subject to chat, login and signup. They may use forgot password service and reset password services as well.

Once a user login to the server, a token is generated and sent back to client to maintain the identity, hence acknowledging them as USER(s).

Users with role assigned as “REVIEWERS” are expected to manage the Skill CMS. There might be some dispute or conflict in a skill. REVIEWERS then take the access of skill data and finalise the conflict there itself for smooth functionality.

ADMIN users are those who have special rights with them. These are more like moderators with much special rights than any other user.

At the top level of the hierarchy are the BUREAUCRATS. These users have more rights than anyone. They can change role of any other user, override decision of any ADMIN user as well. Both admins and bureaucrats have the access to all the settings file on the server. They not only can look at the list, but also download and upload them. Now these users also have right to upgrade or downgrade any other user as well.

All these user roles are defined in UserRole.java file.

In each request received by the server, the user role of user making the request is compared with the minimal user role in getMinimalUserRole() method. This method is defined in AbstractAPIHandler which validates if a user is allowed to access a particular servlet or not.

private void process(HttpServletRequest request, HttpServletResponse response, Query query) throws ServletException, IOException {
	// object initialisation and comparsions
// user authorization: we use the identification of the user to get the assigned authorization
        Authorization authorization = DAO.getAuthorization(identity);

        if (authorization.getUserRole().ordinal() < minimalUserRole.ordinal()) {
        	response.sendError(401, "Base user role not sufficient. Your base user role is '" + authorization.getUserRole().name() + "', your user role is '" + authorization.getUserRole().getName() + "'");
			return;
        }
// evaluations based on other request parameters.
}

Now that we know about what User Roles actually are, let us look at how the servlet which allows the users {with at least ADMIN login} to change user role of some other user works.

In the request, 2 parameters are expected. These are :

  • user : email id of the user whose role has to be changed.
  • role : new role which will be assigned to this user.

Using a switch case, we identify the user role which is requested. If role is found to be null or any other value apart from “bot”, “anonymous”, “user”, “reviewer”, “accountcreator”, “admin” or “bureaucrat”, an error with error code 400 and error message “Bad User role” is thrown.

In the next steps, server generates client identity in order to get the corresponding Authorization object. If the user is not found in the database, again an error is thrown with error code 400 and error message “role not found

ClientCredential credential = new ClientCredential(ClientCredential.Type.passwd_login, userTobeUpgraded);
            ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, credential.getName());
            if (!DAO.hasAuthorization(identity)) {
                throw new APIException(400, "Username not found");
            }

By now, server is clear with the user identity and new role to be assigned. Since the user role is defined in authorization.json file, we overwrite the existing user role and finally server sends back the new user role of the use

Authorization auth = DAO.getAuthorization(identity);
            try {
                auth.setUserRole(userRole);
            } catch (IllegalArgumentException e) {
                throw new APIException(400, "role not found");
            }

            // Print Response
            result.put("newDetails", auth.getJSON());
            result.put("accepted", true);
            result.put("message", "User role changed successfully!!");
            return new ServiceResponse(result);

 

Automatic Signing and Publishing of Android Apps from Travis

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login using travis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tar fastlane.json key.jks

 

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

 

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

#!/bin/sh
set -e

export DEPLOY_BRANCH=${DEPLOY_BRANCH:-master}

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_REPO_SLUG" != "iamareebjamal/android-test-fastlane" -o "$TRAVIS_BRANCH" != "$DEPLOY_BRANCH" ]; then
    echo "We decrypt key only for pushes to the master branch and not PRs. So, skip."
    exit 0
fi

openssl aes-256-cbc -K $encrypted_4dd7_key -iv $encrypted_4dd7_iv -in ./scripts/secrets.tar.enc -out ./scripts/secrets.tar -d
tar xvf ./scripts/secrets.tar -C scripts/

 

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

cp app-release-unsigned.apk app-release-unaligned.apk
jarsigner -verbose -tsa http://timestamp.comodoca.com/rfc3161 -sigalg SHA1withRSA -digestalg SHA1 -keystore ../scripts/key.jks -storepass $STORE_PASS -keypass $KEY_PASS app-release-unaligned.apk $ALIAS

 

Then run this command to zipalign the app

${ANDROID_HOME}/build-tools/25.0.2/zipalign -v -p 4 app-release-unaligned.apk app-release.apk

 

Remember that the build tools version should be the same as the one specified in .travis.yml

This will create an apk named app-release.apk

Publishing App

This is the easiest step. First install fastlane using this command

gem install fastlane

 

Then run this command to publish the app to alpha channel on Play Store

fastlane supply --apk app-release.apk --track alpha --json_key ../scripts/fastlane.json --package_name com.iamareebjamal.fastlane

 

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

Auto Updating SUSI Android APK and App Preview on appetize.io

This blog will cover the way in which the SUSI Android APK is build automatically after each commit and pushed to “apk” branch in the github repo. Other thing which will be covered is that how the app preview on appetize.io can be updated after each commit. This is basically for the testers who wish to test the SUSI Android App. There are four ways to test the SUSI Android App. One is to simply download the alpha version of the app from the Google PlayStore. Here is the link to the app. Join the alpha testing and report bugs on the github issue tracker of the repo. Other way is to build the app from Android Studio but you may need to set the complete project. If you are looking to contribute in the project, this is the advised way to test the app. The other two ways are explained below.

Auto Building of APK and pushing to “apk” branch

We have written a script which does following steps whenever a PR is merged:

  1. Checks if the commit is of a PR or a commit to repo
  2. If not of PR, configures a user whose github account will be used to push the APKs.
  3. Clones the repo, generates the debug and release APK.
  4. Deletes everything in the apk branch.
  5. Commits and Pushes new changes to apk branch.

This script is written for people or testers who do not have android studio installed in their computer and want to test the app. So, they can directly download the apk from the apk branch and install it in their phone. The APK is always updated after each commit. So, whenever a tester downloads the APK from apk branch, he will always get the latest app.

if [[ $CIRCLE_BRANCH != pull* ]]
then
    git config --global user.name "USERNAME"
    git config --global user.email "EMAIL"

    git clone --quiet --branch=apk https://USERNAME:[email protected]/fossasia/susi_android apk > /dev/null
    ls
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-debug.apk apk/susi-debug.apk
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-release-unsigned.apk apk/susi-release.apk
    cd apk

    git checkout --orphan workaround
    git add -A

    git commit -am "[Circle CI] Update Susi Apk"

    git branch -D apk
    git branch -m apk

    git push origin apk --force --quiet > /dev/null
fi

Auto Updating of App Preview on appetize.io

The APKs generated in the above step can now be used to set up the preview of the app on the appetize.io. Appetize.io is an online simulator to run mobile apps ( IOS and Android). Appetize.io provides a nice virtual mobile frame to run native apps with various options like screen size, mobile, OS version, etc. Appetize.io provides some API to update/publish the app. In SUSI, we once uploaded the app on appetize.io and now we are using the API provided by them to update the APK everytime a commit is pushed in the repository.

API information (Derived from official docs of appetize.io):

You may upload a new version of an existing app, or update app settings.

Send an HTTP POST request to

https://[email protected]/v1/apps/PUBLICKEY

Replace APITOKEN with your API token and PUBLICKEY with the public key of the app you’re updating. Your API token must be permissioned to the same account as was used to upload the app. The POST body must be a JSON object. To delete a previously set field, use a value of null.

Optional Fields

  1. url: (string) a publicly accessible link to your .zip, .tar.gz, or .apk file, used to upload a new version of your app.
  2. note: (string) a note for your own purposes, will appear on your management dashboard.

For the url parameter, we have used https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk and note can be anything. We have used Update SUSI Preview.

curl https://[email protected]/v1/apps/mbpprq4xj92c119j7nxdhttjm0 -H 'Content-Type: application/json' -d '{"url":"https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk", "note": "Update SUSI Preview"}'

Summary

This blog covered about how to implement an automatic structure to generate APKs for testing and using that APK to build a preview on websites like appetize.io and then using the APIs provided by them to update the APK after each PR merge in the repo. Check out the resources below to learn more about the topic. So, if you are thinking of contributing to SUSI Android App, this may help you a little in testing the app. But if not, then you can also use the similar technique for your android app as well and ease the life of testers.

Resources

  1. Docs of appetize.io to learn more about the API https://appetize.io/docs
  2. Tutorial on using curl to make API requests https://curl.haxx.se/docs/httpscripting.html
  3. Tutorial on writing basic shell scripts https://ryanstutorials.net/bash-scripting-tutorial/

Creating GUI for configuring SUSI Linux Settings

SUSI Linux app provides access to SUSI on Linux distributions on desktop as well as hardware devices like Raspberry Pi. The settings for SUSI Linux are controlled with the use of a config.json file. You may edit the file manually, but to provide safe configurations, we have a config generator script. You may run the script to configure settings like TTS Engine, STT Engine, authentication, choice about the hotword engine etc. Generally, it is easier to configure application settings through a GUI. Thus, we added a GUI for it using PyGTK and Glade.

Glade is a GUI designer for GNOME based Linux systems. I wrote a blog about how to create user interfaces in Glade and access it from Python code in SUSI Linux. Now, for creating UI for Configuration screen, we need to choose an ideal layout. Glade provides various choices like BoxLayout, GridLayout, FlowBox, ListBox , Notebook etc. Since, we need to display only basic settings options, we select the BoxLayout for this purpose.

BoxLayout as the name suggests, forms a box like arrangement for widgets. You can arrange the widgets in either Landscape or Horizontal Layout. We select Application Window as a top-level container and add a BoxLayout container in it. Now, in each box of the BoxLayout, we need to add the widgets like ComboBox and Switch for user’s choice and a Label. This can be done by using a horizontal BoxLayout with corresponding widgets. After arranging the UI in above described manner, we have a GUI like below.

If you see the current window in the preview now, you will find that the ComboBox do not have any items. We need to define items in the ComboBox using a GTKListStore. You may refer to this video tutorial to see how this can be done.

Now, when we see the preview, our GUI is fully functional. We have options for Speech Recognition Service, Text to Speech Service in ComboBox. Other simple settings are available as switches.

Now, we need to add functionality to our UI. We want our code to be modular and structured, therefore, we declare a ConfigurationWindow class. Though the ideal way to handle such cases is inheriting from the Gtk.Window class, but reading the documentation of PyGTK+ 3, I could not find a way to do this for windows created through Glade. Thus, we will use composition for storing the window object. We add window and other widgets present in the UI as properties of ConfigurationWindow class like this.

class ConfigurationWindow:
   def __init__(self) -> None:
       super().__init__()
       builder = Gtk.Builder()
       builder.add_from_file(os.path.join("glade_files/configure.glade"))

       self.window = builder.get_object("configuration_window")
       self.stt_combobox = builder.get_object("stt_combobox")
       self.tts_combobox = builder.get_object("tts_combobox")
       self.auth_switch = builder.get_object("auth_switch")
       self.snowboy_switch = builder.get_object("snowboy_switch")
       self.wake_button_switch = builder.get_object("wake_button_switch")

Now, we need to connect the Signals from our configuration window to the Handler. We declare the Handler as a nested class in the ConfigurationWindow class because its scope of usage is inside the ConfigurationWindow object. Then you may connect signals to an object of the Handler class.

builder.connect_signals(ConfigurationWindow.Handler(self))

Since we may need to modify the state of the widgets, we hold a reference of the parent ConfigurationWindow object in the Handler and pass the self as a parameter to the Handler. You may read more about using the handlers in my previous blog.

In the Handler, we connect to the config.json file and change the parameters of the the file based on the user inputs on the GUI. We handle it for the Text to Speech selection comboBox in the following manner. We also declare two addition Dialogs for handling the input of credentials by the users for the Watson and Bing services.

def on_stt_combobox_changed(self, combo: Gtk.ComboBox):
   selection = combo.get_active()

   if selection == 0:
       config['default_stt'] = 'google'

   elif selection == 1:
       credential_dialog = WatsonCredentialsDialog(self.config_window.window)
       response = credential_dialog.run()

       if response == Gtk.ResponseType.OK:
           username = credential_dialog.username_field.get_text()
           password = credential_dialog.password_field.get_text()
           config['default_stt'] = 'watson'
           config['watson_stt_config']['username'] = username
           config['watson_stt_config']['password'] = password
       else:
           self.config_window.init_stt_combobox()

       credential_dialog.destroy()

   elif selection == 2:
       credential_dialog = BingCredentialDialog(self.config_window.window)
       response = credential_dialog.run()

       if response == Gtk.ResponseType.OK:
           api_key = credential_dialog.api_key_field.get_text()
           config['default_stt'] = 'bing'
           config['bing_speech_api_key']['username'] = api_key
       else:
           self.config_window.init_stt_combobox()

       credential_dialog.destroy()

Now, we declare two more methods to show and exit the Window.

def show_window(self):
   self.window.show_all()
   Gtk.main()

def exit_window(self):
   self.window.destroy()
   Gtk.main_quit()

Now, we may use the ConfigurationWindow class object anywhere from our code. This modularized approach is better when you need to manage multiple windows as you can just declare the Window of a particular type and show it whenever need in your code.

Resources

  • Glade usage Youtube tutorial: https://www.youtube.com/watch?v=vOGK3TveDDk
  • Creating GUI using PyGTK for SUSI Linux: https://blog.fossasia.org/making-gui-for-susi-linux-with-pygtk/
  • PyGObject Documentation: http://pygobject.readthedocs.io/en/latest/getting_started.html