Implementing Text to Speech on SUSI Web Chat

SUSI Web Chat now gives voice replies while chatting with it similar to SUSI Android and SUSI iOS clients. To test the Voice Output on Chrome,

  1. Visit chat.susi.ai
  2. Click on the Mic input button.
  3. Say something using the Mic when the Speak Now View appears

The simplest way to add text-to-speech to your website is by using the official Speech API currently available for Chrome Browser. The following steps help to achieve it in ReactJS.

  1. Initialize state defaults in a component named, VoicePlayer.
const defaults = {
      text: '',
      volume: 1,
      rate: 1,
      pitch: 1,
      lang: 'en-US'
    } 

There are two states which need to be maintained throughout the component and to be passed as such in our props which will maintain the state. Thus our state is simply

this.state = {
      started: false,
      playing: false
    }
  1. Our next step is to make use of functions of the Web Speech API to carry out the relevant processes.
  1. speak() – window.speechSynthesis.speak(this.speech) – Calls the speak method of the Speech API
  2. cancel() – window.speechSynthesis.cancel() – Calls the cancel method of the Speech API
  1. We then use our component helper functions to assign eventListeners and listen to any changes occurring in the background. For this purpose we make use of the functions componentWillReceiveProps(), shouldComponentUpdate(), componentDidMount(), componentWillUnmount(), and render().
  1. componentWillReceiveProps() – receives the object parameter {pause} to listen to any paused action
  2. shouldComponentUpdate() simply returns false if no updates are to be made in the speech outputs.
  3. componentDidMount() is the master function which listens to the action start, adds the eventListener start and end, and calls the speak() function if the prop play is true.
  4. componentWillUnmount() destroys the speech object and ends the speech.

Here’s a code snippet for Function componentDidMount()

componentDidMount () {
  
    const events = [
      { name: 'start', action: this.props.onStart }
    ]
    // Adding event listeners
    events.forEach(e => {
      this.speech.addEventListener(e.name, e.action)
    })

    this.speech.addEventListener('end', () => {
      this.setState({ started: false })
      this.props.onEnd()
    })

    if (this.props.play) {
      this.speak()
    }
  }
  1. We then add proper props validation in the following way to our VoicePlayer component.
VoicePlayer.propTypes = {
  play: PropTypes.bool,
  text: PropTypes.string,
  onStart: PropTypes.func,
  onEnd: PropTypes.func
};
  1. The next step is to pass the props from a listener view to the VoicePlayer component. Hence the listener here is the component MessageListItem.js from where the voice player is initialized.
  1. First step is to initialise the state.
this.state = {
      play: false,
    }
  onStart = () => {
    this.setState({ play: true });
  }
  onEnd = () => {
    this.setState({ play: false });
  }
  1. Next, we set play to true when we want to pass the props and the text which is to be said and append it to the message lists which have voice set as `true`
 { this.props.message.voice &&
      (<VoicePlayer
             play
             text={voiceOutput}
             onStart={this.onStart}
             onEnd={this.onEnd}
 />)}

Finally, our message lists with voice true will be heard on the speaker as they have been spoken on the microphone. To get access to the full code, go to the repository https://github.com/fossasia/chat.susi.ai or on our chat channel at gitter.im/fossasia/susi_webchat

Resources

  1. Speak-easy-synthesis repository http://mdn.github.io/web-speech-api/speak-easy-synthesis
  2. Web-speech-api repository https://github.com/mdn/web-speech-api/
Continue ReadingImplementing Text to Speech on SUSI Web Chat

Exporting Functions in PSLab Firmware

Computer programs consist of several hundreds line of code. Smaller programs contain around few hundreds while larger programs like PSLab firmware contains lines of code expanding over 2000 lines. The major drawback with a code with lots of lines is the difficulty to debug and hardship for a new developer to understand the code. As a solution modularization techniques can be used to have the long code broken down to small set of codes. In C programming this can be achieved using different .c files and relevant .h(header) files.

The task is tricky as the main code contains a lot of interrelated functions and variables. Lots of errors were encountered when the large code in pslab-firmware being modularized into application specific files. In a scenario like this, global variables or static variables were a much of a help.

Global Variables in C

A global variable in C is a variable whose scope spans wide across the entire program. Updation to the variable will be reflected everywhere it is used.

extern TYPE VARIABLE = VALUE;

Static Variables in C

This type of variables preserve their content regardless of the scope they are in.

static TYPE VARIABLE = VALUE;

Both the variables preserve their values but the memory usage is different depending on the implementation. This can be explained using a simple example.

Suppose a variable is required in different C files and it is defined in one of the header files as a local variable. The header file is then added to several other c files. When the program is compiled the compiler will create several copies of the same variable which will throw a compilation error as variable is already declared. In PSLab firmware, a variable or a method from one library has a higher probability of it being used in another one or many libraries.

This issue can be addressed in two different ways. They are by using static variables or global variables. Depending on the selection, the implementation is different.

The first implementation is using static variables. This type of variables at the time of compilation, create different copies of himself in different c files. Due to the fact that C language treats every C file as a separate program, if we include a header file with static variables in two c files, it will create two copies of the same variable. This will avoid the error messages with variables redeclared. Even though this fixes the issue in firmware, the memory allocation will be of a wastage. This is the first implementation used in PSLab firmware. The memory usage was very high due to duplicate variables taking much memory than they should take. This lead to the second implementation.

first_header.h

#ifndef FIRST_HEADER_H 
#define FIRST_HEADER_H 

static int var_1;

#endif 

/* FIRST_HEADER_H */

second_header.h

#ifndef SECOND_HEADER_H
#define SECOND_HEADER_H

static int var_1;

#endif    

/* SECOND_HEADER_H */

first_header.c

#include <stdio.h>
#include "first_header.h"
#include "second_header.h"

int main() {
    var_1 = 10;
    printf("%d", var_1);
}

The next implementation uses global variables. This type of variables need to be declared only in one header file and can be reused by declaring the header file in other c files. The global variables must be declared in a header file with the keyword extern and defined in the relevant c file once. Then it will be available throughout the application and no errors of variable redeclaration will occur while compiling. This became the final implementation for the PSLab-firmware to fix the compilation issues modularizing application specific C and header files.

Resources:

Continue ReadingExporting Functions in PSLab Firmware

Drawing Lines on Images in Phimpme

In the editing section of the Phimpme app, we want a feature which lets the user write something on the image in their own handwriting. The user can also select different colour palette available inside the app. We aligned this feature in our editor section as Phimpme Image app allows a user to use our custom camera with the large number of editing options (Enhancement, Transform, Applying Stickers, Applying filters and writing on images). In this post, I am explaining how I implemented the draw feature in Phimpme Android app.

Let’s get started

Step -1: Create bitmap of Image and canvas to draw

The first step is to create a bitmap of Image on which you want to draw lines. Now we need a canvas to draw and canvas requires bitmap to work. So in this step, we will create a bitmap and new canvas to draw the line.

BitmapFactory.Options bmOptions = new BitmapFactory.Options();

Bitmap bitmap =  BitmapFactory.decodeFile(ImagePath, bmOptions);

Canvas canvas = new Canvas(bitmap);

Once the canvas is initialized, we use paint class to draw on the canvas.

Step-2: Create Paint class to draw on canvas

In this step, we will create a new Paint class with some properties like colour, Stroke Width and Coordinate where we want to draw a line and it can be done by using the following code.

Paint paint = new Paint();

paint.setColor(Color.rgb(255, 153, 51));

paint.setStrokeWidth(10);

int startx = 50;

int starty = 90;

int endx = 150;

int endy = 360;

canvas.drawLine(startx, starty, , endy, paint);

The above code will result in a straight line on the canvas like the below screenshot.

Step-3: Add support to draw on touch dynamically

In step 2 we have added the feature to draw a straight line, but what if the user wants to draw the lines on canvas with on touch event. So to achieve this we have to apply onTouchListener and applying coordinates dynamically and it can be done by using the following code.

@Override

public boolean onTouchEvent(MotionEvent event) {

 boolean ret = super.onTouchEvent(event);

 float x = event.getX();

 float y = event.getY();

 switch (event.getAction()) {

     case MotionEvent.ACTION_DOWN:

         ret = true;

         last_x = x;

         last_y = y;

         break;

     case MotionEvent.ACTION_MOVE:

         ret = true;

         canvas.drawLine(last_x, last_y, x, y);

         last_x = x;

         last_y = y;

         this.postInvalidate();

         break;

     case MotionEvent.ACTION_CANCEL:

     case MotionEvent.ACTION_UP:

         ret = false;

         break;

 }

 return ret;

}

Now the above code will let you draw the lines dynamically. But, how? The answer is In on touch event, I am fetching coordinates of touch and applying it dynamically to a canvas. Now our feature is ready to draw the lines on canvas with finger touch.

Drawing line in Phimpme Editor

Step – 4: Adding eraser functionality

Till now we have added the draw on canvas functionality, but what if the user wants to erase particular area. So now we have to implement the eraser functionality. It is very simple now what we have to do again we have to draw, but we will set paint color to the transparent color. It will draw the transparent color on the existing line which results in the previous picture and it looks like we are erasing. It can be done by adding one line.

paint.setColor(eraser ? Color.TRANSPARENT : color);

Now we have done by drawing the lines on an image and erasing functionality.

Resources:

Continue ReadingDrawing Lines on Images in Phimpme

Upload image on Imgur account with user authentication in Phimpme

In Phimpme app we wanted to allow a user to upload an image into their personal Imgur account. To achieve this I have to authenticate the user Imgur account in our Phimpme android app and upload the image to their account. In this post, I will be explaining how to upload an image on Imgur from Phimpme Android app from user account using authentication token.

Let’s get started

Step – 1: Register your application on Imgur to receive your Client ID and Client Secret.

Register your application on Imgur from here and obtain the Client-Id and Client-Secret. Client-Id will be used to authenticate user account.

Step-2: Authenticate the user

Well, This is the tedious task as they haven’t provided any direct API for Java/Android or any other language to authenticate. So, I achieved it via WebView of Android, in  Phimpme app we asked user to login into their WebView and the we intercepted network calls and parsed URL to get the access token once the user logged in.

a.) Load below URL in WebView with your application’s client id.

https://api.imgur.com/oauth2/authorize?client_id=ClientId&response_type=token&state=APPLICATION_STATE

WebView imgurWebView = (WebView) findViewById(R.id.LoginWebView);
imgurWebView.setBackgroundColor(Color.TRANSPARENT);
imgurWebView.loadUrl(IMGUR_LOGIN_URL);
imgurWebView.getSettings().setJavaScriptEnabled(true);

imgurWebView.setWebViewClient(new WebViewClient() {
  @Override
  public boolean shouldOverrideUrlLoading(WebView view, String url) {

      if (url.contains(REDIRECT_URL)) {
          splitUrl(url, view);
      } else {
          view.loadUrl(url);
      }

      return true;
  }


b.)  Now, It will ask the user to enter the Imgur username and password after that it will redirect to the URL, which we added as a callback URL while adding our app in Imgur dashboard. In Phimpme I added https://org.fossaisa.phimpme as callback URL.

c.)  After logging in WebView it will redirect to the callback URL with the following parameters: access_token, refresh_token, authorization_code:

https://api.imgur.com/oauth2/authorize?client_id=YOUR_CLIENT_ID&response_type=REQUESTED_RESPONSE_TYPE&state=APPLICATION_STATE

Step -3

To obtain the access token from the right URL we have to intercept each and every URL is getting loaded in webview and detect the right URL and parse it. For which I have overridden the shouldOverrideUrlLoading of webViewclient and pass on it to splitUrl method.

private void splitUrl(String url, WebView view) {
  String[] outerSplit = url.split("\\#")[1].split("\\&");
  String username = null;
  String accessToken = null;

  int index = 0;

  for (String s : outerSplit) {
      String[] innerSplit = s.split("\\=");

      switch (index) {
          // Access Token
          case 0:
              accessToken = innerSplit[1];
              break;

          // Refresh Token
          case 3:
              refreshToken = innerSplit[1];
              break;

          // Username
          case 4:
              username = innerSplit[1];
              break;
          default:

      }

      index++;
  }

Now we have user access token of the user to access API to Imgur to do operations(read, write) for that particular user.

Note: Authentication token is valid for 28 days only, after 28 days you will have to get the new access token.

Step- 4: Upload image using authentication token

Now we are ready to upload images on Imgur on the behalf of user’s account and to upload we need to send requests to Imgur API to this URL with authentication header and body image. The body will contain the imageString, title and description.

When we post image on Imgur anonymously, we need to add header as Authorization Client-Id {client-id} but to upload image on imgur from an account, we need to add header Authorization Bearer with value equals to access-token and image data in base64 string as request body, as we did in our previous post on upload image on Imgur.

Imgur Account Image uploads

That’s it.

The problem I faced:

I have noticed that one token is valid for 28 days only, so after 28 you will need to ask the user to log in again in Imgur app.

To avoid this what I did is before 28 days I do send a request to get a new access token which is again valid for next 28 days.

Resources:

Continue ReadingUpload image on Imgur account with user authentication in Phimpme

Building Preference Screen in SUSI Android

SUSI provides various preferences to the user in the settings to customize the app. This allows the user to configure the application according to his own choice. There are different preferences available such as to select the theme or the language for text to speech. Preference Setting Activity is an important part of an Android application. Here we will see how we can implement it in an Android app taking SUSI Android (https://github.com/fossasia/susi_android) as the example.

Firstly, we will proceed by adding the Gradle Dependency for the Setting Preferences

compile 'com.takisoft.fix:preference-v7:25.4.0.3'

Then to create the custom style for our setting preference screen we can set

@style/PreferenceFixTheme

as the base theme and can apply various other modifications and color over this. By default it has the usual Day and Night theme with NoActionBar extension.

Now to make different preferences we can use different classes as shown below:

SwitchPreferenceCompat: This gives us the Switch Preference which we can use to toggle between two different modes in the setting.

EditTextPreference: This preference allows the user to give its own choice of number or string in the settings which can be used for different actions.

For more details on this you can refer the this link.

Implementation in SUSI Android

In SUSI Android we have created an activity named activity_settings which holds the Preference Fragment for the setting.

<?xml version="1.0" encoding="utf-8"?>


<fragment

  xmlns:android="http://schemas.android.com/apk/res/android"

  xmlns:tools="http://schemas.android.com/tools"

  android:id="@+id/chat_pref"

  android:name="org.fossasia.susi.ai.activities.SettingsActivity$ChatSettingsFragment"

  android:layout_width="match_parent"

  android:layout_height="match_parent"

  tools:context="org.fossasia.susi.ai.activities.SettingsActivity"/>

The Preference Settings Fragment contains different Preference categories that are implemented to allow the user to have different customization option while using the app. The pref_settings.xml is as follows

<?xml version="1.0" encoding="utf-8"?>

<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android"

  xmlns:app="http://schemas.android.com/apk/res-auto"

  android:title="@string/settings_title">


  <PreferenceCategory

      android:title="@string/server_settings_title">

      <PreferenceScreen

          android:title="@string/server_pref"

          android:key="Server_Select"

          android:summary="@string/server_select_summary">

      </PreferenceScreen>

  </PreferenceCategory>


  <PreferenceCategory

      android:title="@string/settings_title">

      <com.takisoft.fix.support.v7.preference.SwitchPreferenceCompat

          android:id="@+id/enter_key_pref"

          android:defaultValue="true"

          android:key="@string/settings_enterPreference_key"

          android:summary="@string/settings_enterPreference_summary"

          android:title="@string/settings_enterPreference_label" />

  </PreferenceCategory>

All the logic related to Preferences and their action is written in SettingsActivity Java class. It listens for any change in the preference options and take actions accordingly in the following way.

public class SettingsActivity extends AppCompatActivity {


  private static final String TAG = "SettingsActivity";

  private static SharedPreferences prefs;


  @Override

  protected void onCreate(Bundle savedInstanceState) {

      super.onCreate(savedInstanceState);

      prefs = getSharedPreferences(Constant.THEME, MODE_PRIVATE);

      Log.d(TAG, "onCreate: " + (prefs.getString(Constant.THEME, DARK)));

      if(prefs.getString(Constant.THEME, "Light").equals("Dark")) {

          setTheme(R.style.PreferencesThemeDark);

      }

      else {

          setTheme(R.style.PreferencesThemeLight);

      }

      setContentView(R.layout.activity_settings);


  }

The class contains a ChatSettingFragment which extends the PreferenceFragmentCompat to give access to override functions such as onPreferenceClick. The code below shows the implementation of it.

public boolean onPreferenceClick(Preference preference) {

              Intent intent = new Intent();

              intent.setComponent( new ComponentName("com.android.settings","com.android.settings.Settings$TextToSpeechSettingsActivity" ));

              startActivity(intent);

              return true;

          }

      });


      rate=(Preference)getPreferenceManager().findPreference(Constant.RATE);

      rate.setOnPreferenceClickListener(new Preference.OnPreferenceClickListener() {

          @Override

          public boolean onPreferenceClick(Preference preference) {

              startActivity(new Intent(Intent.ACTION_VIEW, Uri.parse("http://play.google.com/store/apps/details?id=" + getContext().getPackageName())));

              return true;

          }

      });
}

For diving more into the code we can refer to the github repo of Susi Android (https://github.com/fossasia/susi_android).

Resources

Continue ReadingBuilding Preference Screen in SUSI Android

Upload Image to Imgur Anonymously Using Volley in Phimpme Android app

As Phimpme Android is an image app in which lets you share your image on multiple platform without installing that apps like Twitter, Facebook, Pinterest and Imgur. Imgur is the best place to share and enjoy the most awesome images on the Internet. Imgur provides APIs to  upload image from your account as well as anonymously. In this blog I am going to explain how I added the imgur upload feature in Phimpme Android app.

I have implemented upload to  Imgur anonymously. There is step by step guide to implementing it.

Step 1: Register your application on Imgur.

To register your application click here, For more detail read their documentation here Imgur API site.

Once you have registered your application, you will be provided with two keys, Client ID and Client Secret. Save them, we will need them later to upload an image and for = authentication purposes.

Step-2: Add a networking library in your Gradle, I have used volley.

Olley is a good networking library, works well for both API requests and File upload. To add volley in the project, add the following dependency

'com.android.volley:volley:1.0.0'

Step-3 Convert your image into the desired format to upload.

Imgur three kinds of images: A binary file, base64 data, or  URL of an image. (Up to 10MB).

In phimpme Android, we are uploading base64 image string.

So, we have to first convert our image to the bitmap and then convert the bitmap to base64. I recommend using an Image library to decode bitmap otherwise, there are chances of OutOfMemory exception thrown for large image files.

To convert bitmap to base64 string

public static String get64BaseImage (Bitmap bmp) {
       ByteArrayOutputStream baos = new ByteArrayOutputStream();
       bmp.compress(Bitmap.CompressFormat.JPEG, 100, baos);
       byte[] imageBytes = baos.toByteArray();
       return Base64.encodeToString(imageBytes, Base64.DEFAULT);
   }

Best practice to add such methods into theutility class. Also, you can apply the image compression in above if you want to reduce the image size, here number 100 represents, preserve the  100% quality of the image.

Step-4:  Now we need 2 things, add image string in body and add AUTHENTICATION key in headers of the request.

We have added two methods to do above mentioned tasks.

  1. To upload the image data to Imgur.
  2. To add a header in our network request.

The header is required for Imgur to authenticate the client who uploads the file and your authentication key is your  CLIENT-ID, which we have generated in step 1.

@Override
               protected Map<String, String> getParams() throws AuthFailureError {
                   Map<String, String> parameters = new HashMap<String, String>();
                   parameters.put("image", imageString);
                   if(caption!=null && !caption.isEmpty())
                       parameters.put("title",caption);
                   return parameters;
               }


Now add the headers to authenticate the client.The above code contains the body part with the key as “image” and value as the imageString data, which was the result of get64BaseImage () method.

   @Override
               public Map<String, String> getHeaders() throws AuthFailureError {
                   Map<String, String> headers = new HashMap<String, String>();
                   headers.put("Authorization",”Client-Id {client-id});
                   return headers;
               }

Override the getHeader method of Volley library and return a map which has a key named “Authorization” and value is client id of Imgur.

Now we are ready to upload an image on Imgur through Phimpme Android app.

The problem I faced:

Whenever I was trying to upload large size images, I was getting volleytimeout exception, by default connection timeout was not sufficient to upload large files, so I resolved this error by adding below line in the request policy.

           request.setRetryPolicy(new DefaultRetryPolicy( 50000, 5, DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));

Now it works seamlessly with large files even on slow internet network and we are receiving the URL of the image in the response.

Resources :

Continue ReadingUpload Image to Imgur Anonymously Using Volley in Phimpme Android app

Selecting Best persistent storage for Phimpme Android and how to use it

As we are progressing in our Phimpme Android app. I added account manager part which deals with connecting all other accounts to phimpme. Showing a list of connected accounts.

We need a persistent storage to store all the details such as username, full name, profile image url, access token (to access API). I researched on various Object Relation mapping (ORMs) such as:

  1. DBFlow: https://github.com/Raizlabs/DBFlow
  2. GreenDAO: https://github.com/greenrobot/greenDAO
  3. SugarORM: http://satyan.github.io/sugar/
  4. Requery: https://github.com/requery/requery

and other NoSQL databases such as Realm Database : https://github.com/realm/realm-java.

After reading a lot from some blogs on the benchmarking of these ORMs and database, I came to know that Realm database is quite better in terms of Speed of writing data and ease of use.

Steps to integrate Realm Database:

  • Installation of Realm database in android

Following these steps https://realm.io/docs/java/latest/#installation quickly setup realm in android. Add

classpath "io.realm:realm-gradle-plugin:3.3.2"

in Project level build.gradle file and Add

apply plugin: 'realm-android' 

in app level build.gradle, That’s it for using Realm

  • Generating required Realm models

Firstly, make sure what you need to store in your database. In case of phimpme, I first go through the account section and noted down what needs to be there.  Profile image URL, username, full name, account indicator image name. Below image illustrate this better.

This is the Realm Model class I made in Kotlin to store name, username and access token for accessing API.

open class AccountDatabase(
       @PrimaryKey var name: String = "",
       var username: String = "",
       var token: String = ""
) : RealmObject()

  • Writing data in database

In Account manager, I create a add account option from where a dialog appear with a list of accounts. Currently, Twitter is working, when onSuccess function invoke in AccountPickerFragment I start a twitter session and store values in database. Writing data in database:

// Begin realm transaction
realm.beginTransaction();

// Creating Realm object for AccountDatabase Class
account = realm.createObject(AccountDatabase.class,
       accountsList[0]);

account.setUsername(session.getUserName());
account.setToken(String.valueOf(session.getAuthToken()));
realm.commitTransaction();

Begin and commit block in necessary. There is one more way of doing this is using execute function in Realm

  • Use Separate Database Helper class for Database operations

It’s good to use a separate class for all the Database operations needed in the project. I created a DatabaseHelper Class and added a function to query the result needed. Query the database

public RealmResults<AccountDatabase> fetchAccountDetails(){
   return realm.where(AccountDatabase.class).findAll();
}

It give all of the results, stored in the database like below

  • Problems I faced with annotation processor while using Kotlin and Realm together

The Kotlin annotation processor not running due to the plugins wrong order. This issue https://github.com/realm/realm-java/pull/2568 helped me in solving that. I addded apply plugin: ‘kotlin-kapt’. In app gradle file and shift apply plugin: ‘realm-android’ In below the order.

Resources:

 

Continue ReadingSelecting Best persistent storage for Phimpme Android and how to use it

Implementing Wallpapers in React JS SUSI Web Chat Application

The different SUSI AI clients need to match in their feature set. One feature that was missing in the React JS SUSI Web Chat application was the ability for users to change the application wallpaper or background. This is how we implemented it on SUSI Web Chat.
Firstly we added a text field after the circle picker that change the color of the Application body. Because there should be a place to add the wallpaper image URL. Added that text field like this.

   const components = componentsList.map((component) => {
       return

key={component.id} className=‘circleChoose’>

Change color of {component.name}:

 

         color={component} width={'100%'}
         onChangeComplete={ this.handleChangeComplete.bind(this,
         component.component) }
         onChange={this.handleColorChange.bind(this,component.id)}>
       

CirclePicker shows the circular color picker to choose the colors for each component of the application.

        
         name="backgroundImg"
         style={{display:component.component==='body'?'block':'none'}}
         onChange={
           (e,value)=>
           this.handleChangeBackgroundImage(value) }
         value={this.state.bodyBackgroundImage}
          floatingLabelText="Body Background Image URL" />
       

})

In ’TextField’ I have checked below condition whether to display or not to display the text field after the ‘body’ color picker.

style={{display:component.component==='body'?'block':'none'}}

To apply changes as soon as the user enters the image url, we refer the value of the ‘TextField’ and pass it into the ‘handleChangeBackgroundImage()’ function as ‘value’ on change like this.

         onChange={
           (e,value)=>
           this.handleChangeBackgroundImage(value) }

In ‘‘handleChangeBackgroundImage()’ function we change the state of the application and background of the application like this.

  handleChangeBackgroundImage(backImage){
   document.body.style.setProperty('background-image', 'url('+ backImage+')');
   document.body.style.setProperty('background-repeat', 'no-repeat');
   document.body.style.setProperty('background-size', 'cover');
   this.setState({bodyBackgroundImage:backImage});
 }

In here ‘document.body.style.setProperty’ we change the style of the application’s ‘’ tag. This is how wallpapers are changing on SUSI Web Chat Application.

Resources:

React Refs: https://facebook.github.io/react/docs/refs-and-the-dom.html

Refer Value from text field: https://stackoverflow.com/questions/43960183/material-ui-component-reference-does-not-work

Continue ReadingImplementing Wallpapers in React JS SUSI Web Chat Application

Datewise splitting of the Bookmarks in the Homescreen of Open Event Android

In the Open Event Android app we had already incorporated bookmarks in the homescreen along with an improved UI. Now there was scope for further improvement in terms of user experience. The bookmarks were already sorted date wise but we needed to place them under separate date headers. In this blog I will be talking about how this was done in the app.

Initial Design
Current Design

 

 

 

 

 

 

 

 

Initially the user had no way of knowing which session belonged to which day. This could be fixed with a simple addition of a header indicating the day each bookmark belonged to. One way to do this was to add a day header and then get the bookmarks for each day and so on. But this proved to be difficult owing to the fact the number of days could be dynamic owing to the fact that this is a generic app. Another issue was that adding change listeners for the realm results to the bookmarks list for each day produced view duplication and other unexpected results whenever the bookmark list changed. So another approach was chosen that was to get all the bookmarks first and then add the date header and traverse through the bookmarks and only add sessions which belong to the date for which the date header was added earlier.

Bookmark Item Support in GlobalSearchAdapter

The main reason why we are reusing the GlobalSearchAdapter is that we have already defined a DIVIDER type in this adapter which can be reused as the date header.

We needed to initialize a constant for the Bookmark type.

private final int BOOKMARK = 5; //Can be any number

Then we add the Bookmark type in the getItemViewType() function which would return a constant that we defined earlier to indicate that in the filteredResultList we have an object of type Bookmark.

@Override
 public int getItemViewType(int position) {
    if (filteredResultList.get(position) instanceof Track) {
        return TRACK;
    }

    //Other Cases here
    } else if(filteredResultList.get(position) instanceof Session){
        return BOOKMARK;
    } else {
        return 1;
    }
 }

Now we create the viewholder if the list item is of the type Session which in this case will be a bookmark.

@Override
 public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
    RecyclerView.ViewHolder resultHolder = null;
    LayoutInflater inflater = LayoutInflater.from(parent.getContext());
    //Other cases for Track,Location etc
 case BOOKMARK:
    View bookmark = inflater.inflate(R.layout.item_schedule, parent, false);
    resultHolder = new DayScheduleViewHolder(bookmark,context);
    break;
   //Some code
 
 }

Now we do the same in onBindViewHolder(). We bind the contents of the object to the ViewHolder here by calling the bindSession() function. We also pass in an argument which is our database repository i.e realmRepo here.

@Override
 public void onBindViewHolder(RecyclerView.ViewHolder holder, int position) {

    switch (holder.getItemViewType()) {
     //Other cases handled here

case BOOKMARK:
    DayScheduleViewHolder bookmarkTypeViewHolder = (DayScheduleViewHolder) holder;
    Session bookmarkItem = (Session) getItem(position);
    bookmarkTypeViewHolder.setSession(bookmarkItem);
    bookmarkTypeViewHolder.bindSession(realmRepo);
    break;
 }

Updating the AboutFragment

private GlobalSearchAdapter bookMarksListAdapter;
 private List<Object> mSessions = new ArrayList<>();

Earlier the DayScheduleAdapter was being used to display the list of bookmarks. Now we are reusing the GlobalSearchAdapter. Now we have also converted mSessions into a list of objects from a list of sessions.

Now we initialize the adapter so that we can start adding our date headers.

bookMarksListAdapter = new GlobalSearchAdapter(mSessions, getContext());
 bookmarksRecyclerView.setAdapter(bookMarksListAdapter);

In this function loadEventDates() we are storing the all the dates for the event. For example the list for the FOSSASIA17 sample stores the dates in the dateList as [2017-03-17,2017-03-18,2017-03-19]. We fetch the event dates by calling the getEventDateSync() function which has been defined in our Realm Database.

private void loadEventDates() {
 
    dateList.clear();
    RealmResults<EventDates> eventDates = realmRepo.getEventDatesSync();
    for (EventDates eventDate : eventDates) {
        dateList.add(eventDate.getDate());
    }
 }

Now we move on to the core logic of the feature which is to get the date headers to work correctly.

  • Fetch the list of bookmarks from the local Realm database asynchronously.
  • Remove any existing changeListeners to the bookmarkResult.
  • Add a changeListener to our list of results to notify us of the completion of the query or changes in the bookmark list.
  • After this is done, inside the changeListener we first clear the mSessions
  • We now traverse through our date list and compare it with the session startDate which we can obtain by calling the getStartDate(). If the date match occurs for the first time we add a date header after converting the date string into another format using the DateUtils class. So the function formatDay() of DateUtils converts 2017-03-17 to 17 Mar. This format is easily more readable.
  • Repeat for all dates.
private void loadData() {
    loadEventDates();
 
    bookmarksResult = realmRepo.getBookMarkedSessions();
    bookmarksResult.removeAllChangeListeners();
    bookmarksResult.addChangeListener((bookmarked, orderedCollectionInnerChangeSet) -> {
 
        mSessions.clear();
        for (String eventDate : dateList) {
            boolean headerCheck = false;
            for(Session bookmarkedSession : bookmarked){
                if(bookmarkedSession.getStartDate().equals(eventDate)){
                    if(!headerCheck){
                        String headerDate = "Invalid";
                      try {
                       headerDate = DateUtils.formatDay(eventDate);
                      }
                      catch (ParseException e){
                            e.printStackTrace();
                      }
                        mSessions.add(headerDate);
                        headerCheck = true;
                    }
                    mSessions.add(bookmarkedSession);
                }
            }
            bookMarksListAdapter.notifyDataSetChanged();
            handleVisibility();
        }
    });
 }

So, this is how the date-wise organization for the bookmarks in the homescreen was done.

Resources

Continue ReadingDatewise splitting of the Bookmarks in the Homescreen of Open Event Android

Showing Offline and Online Status in SUSI Web Chat

A lot of times while chatting on SUSI Web Chat, one does not receive responses, this could either be due to no Internet connection or a down server.

For a better user experience, the user needs to be notified whether he is connected to the SUSI Chat. If one ever loses out on the Internet connection, SUSI Web Chat will notify you what’s the status through a snack bar.

Here are the steps to achieve the above:-

  1. The first step is to initialize the state of the Snackbar and the message.
this.state = {
snackopen: false,
snackMessage: 'It seems you are offline!'
}
  1. Next, we add eventListeners in the component which will bind to our browser’s window object. While this event can be added to any component in the App but we need it in the MessageSection particularly to show the snack bar.

The window object listens to any online offline activity of the browser.

  1. In the file MessageSection.react.js, inside the function componentDidMount()
  1. We initialize window listeners in our constructor section and bind them to the function handleOnline and handleOffline to set states to the opening of the SnackBar.
// handles the Offlines event
window.addEventListener('offline', this.handleOffline.bind(this));  

// handles the Online event
window.addEventListener('online', this.handleOnline.bind(this));
  1. We then create the handleOnline and handleOffline functions which sets our state to make the Snackbar open and close respectively.
// handles the Offlines event
window.addEventListener('offline', this.handleOffline.bind(this));  

// handles the Online event
window.addEventListener('online', this.handleOnline.bind(this));
  1. Next, we create a Snackbar component using the material-ui snackbar component.

The Snackbar is visible as soon as the the snackopen state is set to true with the message which is passed whether offline or online.

<Snackbar
       autoHideDuration={4000}
       open={this.state.snackopen}
       message={this.state.snackMessage}
 />

To get access to the full code, go to the repository https://github.com/fossasia/chat.susi.ai

Resources

Continue ReadingShowing Offline and Online Status in SUSI Web Chat