Implementing Text to Speech on SUSI Web Chat

SUSI Web Chat now gives voice replies while chatting with it similar to SUSI Android and SUSI iOS clients. To test the Voice Output on Chrome,

  1. Visit chat.susi.ai
  2. Click on the Mic input button.
  3. Say something using the Mic when the Speak Now View appears

The simplest way to add text-to-speech to your website is by using the official Speech API currently available for Chrome Browser. The following steps help to achieve it in ReactJS.

  1. Initialize state defaults in a component named, VoicePlayer.
const defaults = {
      text: '',
      volume: 1,
      rate: 1,
      pitch: 1,
      lang: 'en-US'
    } 

There are two states which need to be maintained throughout the component and to be passed as such in our props which will maintain the state. Thus our state is simply

this.state = {
      started: false,
      playing: false
    }
  1. Our next step is to make use of functions of the Web Speech API to carry out the relevant processes.
  1. speak() – window.speechSynthesis.speak(this.speech) – Calls the speak method of the Speech API
  2. cancel() – window.speechSynthesis.cancel() – Calls the cancel method of the Speech API
  1. We then use our component helper functions to assign eventListeners and listen to any changes occurring in the background. For this purpose we make use of the functions componentWillReceiveProps(), shouldComponentUpdate(), componentDidMount(), componentWillUnmount(), and render().
  1. componentWillReceiveProps() – receives the object parameter {pause} to listen to any paused action
  2. shouldComponentUpdate() simply returns false if no updates are to be made in the speech outputs.
  3. componentDidMount() is the master function which listens to the action start, adds the eventListener start and end, and calls the speak() function if the prop play is true.
  4. componentWillUnmount() destroys the speech object and ends the speech.

Here’s a code snippet for Function componentDidMount()

componentDidMount () {
  
    const events = [
      { name: 'start', action: this.props.onStart }
    ]
    // Adding event listeners
    events.forEach(e => {
      this.speech.addEventListener(e.name, e.action)
    })

    this.speech.addEventListener('end', () => {
      this.setState({ started: false })
      this.props.onEnd()
    })

    if (this.props.play) {
      this.speak()
    }
  }
  1. We then add proper props validation in the following way to our VoicePlayer component.
VoicePlayer.propTypes = {
  play: PropTypes.bool,
  text: PropTypes.string,
  onStart: PropTypes.func,
  onEnd: PropTypes.func
};
  1. The next step is to pass the props from a listener view to the VoicePlayer component. Hence the listener here is the component MessageListItem.js from where the voice player is initialized.
  1. First step is to initialise the state.
this.state = {
      play: false,
    }
  onStart = () => {
    this.setState({ play: true });
  }
  onEnd = () => {
    this.setState({ play: false });
  }
  1. Next, we set play to true when we want to pass the props and the text which is to be said and append it to the message lists which have voice set as `true`
 { this.props.message.voice &&
      (<VoicePlayer
             play
             text={voiceOutput}
             onStart={this.onStart}
             onEnd={this.onEnd}
 />)}

Finally, our message lists with voice true will be heard on the speaker as they have been spoken on the microphone. To get access to the full code, go to the repository https://github.com/fossasia/chat.susi.ai or on our chat channel at gitter.im/fossasia/susi_webchat

Resources

  1. Speak-easy-synthesis repository http://mdn.github.io/web-speech-api/speak-easy-synthesis
  2. Web-speech-api repository https://github.com/mdn/web-speech-api/
Continue ReadingImplementing Text to Speech on SUSI Web Chat

Showing Offline and Online Status in SUSI Web Chat

A lot of times while chatting on SUSI Web Chat, one does not receive responses, this could either be due to no Internet connection or a down server.

For a better user experience, the user needs to be notified whether he is connected to the SUSI Chat. If one ever loses out on the Internet connection, SUSI Web Chat will notify you what’s the status through a snack bar.

Here are the steps to achieve the above:-

  1. The first step is to initialize the state of the Snackbar and the message.
this.state = {
snackopen: false,
snackMessage: 'It seems you are offline!'
}
  1. Next, we add eventListeners in the component which will bind to our browser’s window object. While this event can be added to any component in the App but we need it in the MessageSection particularly to show the snack bar.

The window object listens to any online offline activity of the browser.

  1. In the file MessageSection.react.js, inside the function componentDidMount()
  1. We initialize window listeners in our constructor section and bind them to the function handleOnline and handleOffline to set states to the opening of the SnackBar.
// handles the Offlines event
window.addEventListener('offline', this.handleOffline.bind(this));  

// handles the Online event
window.addEventListener('online', this.handleOnline.bind(this));
  1. We then create the handleOnline and handleOffline functions which sets our state to make the Snackbar open and close respectively.
// handles the Offlines event
window.addEventListener('offline', this.handleOffline.bind(this));  

// handles the Online event
window.addEventListener('online', this.handleOnline.bind(this));
  1. Next, we create a Snackbar component using the material-ui snackbar component.

The Snackbar is visible as soon as the the snackopen state is set to true with the message which is passed whether offline or online.

<Snackbar
       autoHideDuration={4000}
       open={this.state.snackopen}
       message={this.state.snackMessage}
 />

To get access to the full code, go to the repository https://github.com/fossasia/chat.susi.ai

Resources

Continue ReadingShowing Offline and Online Status in SUSI Web Chat

Implementing Login Functionality in SUSI Web Chat

SUSI Web Chat is fully equipped with all the accounting features which are being provided by the SUSI.AI API. This blog discloses all the API features one needs to know to embed the Login functionality in SUSI Web Chat.

  1. To embed the Login feature, first we create a form using material-ui.com components with the followng fields
    1. Email
    2. Password
    3. Note: We can also chose a Custom Server while logging in, here I have used the Standard Server ie. http://api.susi.ai to make the user Login

The form can be made with the help of the following fields

  • TextField for Email, props to be passed
    • Name – email
    • Value – this.state.email which gets the value of the current email
    • floatingLabelText is Email,
    • errorText is the message which we want to show when the email does not match the regex or its empty.

Code Snippet –

<TextField name="email" value={this.state.email} onChange={this.handleChange} errorText={this.emailErrorMessage}    floatingLabelText="Email" />
  • PasswordField for Password
    • Name – password
    • Value – this.state.password which gets the value of the current email
    • floatingLabelText is Password,
    • errorText is the message which we want to show when the password is not filled.

Code Snippet-

<PasswordField name='password' value={this.state.password} onChange={this.handleChange} errorText={this.passwordErrorMessage}   floatingLabelText='Password' />
  • The next elements are RadioButton groups taken from material-ui.com. This ensures the user signs into a standard server or even to a custom server. This is not compulsory as of now.
  • And lastly we need a submit button, which is disabled until all the fields are filled.

Code Snippet –

<RaisedButton label="Login" type="submit" labelColor="#fff" disable={!this.state.validForm} />

For the full form, check out this file at Login.react.js

  1. A Sample UI could be as shown in the image
  2. Next after creating the Login Screen, we make the onSubmit prop which is to be hooked up with another function called handleSubmit. An example code snippet from Login.react.js
 handleSubmit = (e) => {
        e.preventDefault();
        // Get the trimmed values from the fields
        var email = this.state.email.trim();
        var password = this.state.password.trim();
        // Set the default server to login
        let BASE_URL = defaults.Server;
            // handle all the details of the chosen server
        let serverUrl = this.state.serverUrl;
        if(serverUrl.slice(-1) === '/'){
            serverUrl = serverUrl.slice(0,-1);
        }
        if(serverUrl !== ''){
            BASE_URL = serverUrl;
        }
// if email and password is filled return true
        if (!email || !password) { return this.state.isFilled; }
// Check the regex of email
        let validEmail = /^[A-Z0-9._%+-][email protected][A-Z0-9.-]+\.[A-Z]{2,4}$/i.test(email); 
// Pass the parameters to the loginEndPoint
        let loginEndPoint =
            BASE_URL+'/aaa/login.json?type=access-token&login=' +
            this.state.email + '&password=' + this.state.password;
        // If email and password is filled and valid call AJAX
        if (email && validEmail) {
            // AJAX Calls
        }
    }
    1. Then we make the Ajax Calls and store the created token from hitting the URL at http://api.susi.ai/aaa/login.json?type=access-token&login=EMAIL&password=PASSWORD. We store the cookie in browser and generate a session for the user using a package ‘universal-cookies’.
$.ajax({
    url: loginEndPoint,
    dataType: 'jsonp',
    jsonpCallback: 'p',
    jsonp: 'callback',
    crossDomain: true,
    success: function (response) {
        cookies.set('serverUrl', BASE_URL, { path: '/' });
        let accessToken = response.access_token;
        let state = this.state;// Adding the current State
        let time = response.valid_seconds; // Get the valid time of the cookie
        state.isFilled = true; // Set isFilled to true
        state.accessToken = accessToken; // Get the token
        state.success = true; // Set Success to true
        state.msg = response.message; // Get the server message
        state.time = time; // Get the time in the state
        this.setState(state); // Set the  state with the values
/* Pass the token to the binding function handleOnSubmit passing the arguments - token and the valid time */
        this.handleOnSubmit(accessToken, time);
    }.bind(this),
    error: function (errorThrown) {
        let msg = 'Login Failed. Try Again';
        let state = this.state;
        state.msg = msg;
        this.setState(state);
    }.bind(this)
});

 

    1. We then fire up the handleOnSubmit(accessToken, time) which saves the token for the given expiration time from the server.

Here’s the sample code

handleOnSubmit = (loggedIn, time) => {
        let state = this.state;
        if (state.success) {
            cookies.set('loggedIn', loggedIn, { path: '/', maxAge: time }); // set the cookie in the browser to maintain the loggedIn state
            this.props.history.push('/', { showLogin: false });
            window.location.reload();// reload after the loggedIn cookie creation
        }
        else {
            this.setState({
                error: true,
                accessToken: '',
                success: false
            });
        }
    }
  1. We then check the access token and redirect him based on his login state. This is handled in MessageSection.react.js
import Cookies from 'universal-cookie';
const cookies = new Cookies();
if (cookies.get('loggedIn')) {
    //Show all functionalities of loggedIn state
}
else {
//Redirect user to login page
}

 

To have a look at the full project, visit https://github.com/fossasia/chat.susi.ai and feel free to contribute. To test the project visit http://chat.susi.ai

Resources

Continue ReadingImplementing Login Functionality in SUSI Web Chat

Adding Snackbar to undo recent change in theme of SUSI.AI Web Chat

SUSI.AI Web Chat has two primary themes: Light and Dark. The user can switch between these two from settings, but what if the user does not prefer the theme change. He/She has to repeat the entire process of Going to Settings -> Selecting another theme -> saving it. To enable the user to instantly change the theme, we decided to add snackbar on theme change.

What is Snackbar ?

Snackbar contains info regarding the operation just performed and it displays them in a visual format. Snackbar can also have a button like “Undo” to revert the recent operation just made.

It appears at the bottom of screen by default. A simple snackbar looks like this:

Now we needed this to pop-up, whenever the theme is changed. When a user changes theme we run a function named “implementSettings” which checks what the theme is changed to.

The method is:

implementSettings = (values) => {
    this.setState({showSettings: false});
    if(values.theme!==this.state.currTheme){
      this.setState({SnackbarOpen: true});
    }
    this.changeTheme(values.theme);
    this.changeEnterAsSend(values.enterAsSend);
    setTimeout(() => {
       this.setState({
           SnackbarOpen: false
       });
   }, 2500);
  }

The argument values is an object that contains all the change that user has made in settings. Here values.theme contains the value of theme user selected from settings. We first check if the theme is same as the current one if so, we do nothing. If the theme is different from current, then we update the theme by calling this.changeTheme(values.theme) and also initiate snackbar by setting SnackbarOpen to open.

The snackbar component looks like:

<Snackbar
     open={this.state.SnackbarOpen}
     message={'Theme Changed'}
     action="undo"
     autoHideDuration={4000}
     onActionTouchTap={this.handleActionTouchTap}
     onRequestClose={this.handleRequestClose}
/>

This appears on the screen as follows :

Now if a user wants to change the theme instantly he/she can press the undo button. For this undo button, we have a method handleActionTouchTap. In this method, we change the theme back to previous one. The method is implemented in the following manner:

handleActionTouchTap = () => {
    this.setState({
      SnackbarOpen: false,
    });
    switch(this.state.currTheme){
      case 'light': {
          this.changeTheme('dark');
          break;
      }
      case 'dark': {
          this.changeTheme('light');
          break;
      }
      default: {
          // do nothing
      }
    }
  };

The above method is pretty self-explanatory which checks the current theme(“light” or “dark”) and then revert it. We also change the state of SnackbarOpen to “false” so that on clicking “UNDO” button, the theme changes back to previous one and the snackbar closes.Now user is having an option of instantly changing back to previous theme.

Resources:

Testing Link:

http://chat.susi.ai/

Continue ReadingAdding Snackbar to undo recent change in theme of SUSI.AI Web Chat

How SUSI AI Tabulates Answers For You

SUSI is an artificial intelligence chat bot that responds to all kinds of user queries. It isn’t any regular chat bot replying in just plain text. It supports various response types which we refer to as ‘actions’. One such action is the “table” type. When the response to a user query contains a list of answers which can be grouped, it is better visualised as a table rather than plain text.

Lets visit SUSI WebChat and try it out. In our example we ask SUSI for the 2009 race statistics of British Formula 1 racing driver Lewis Hamilton.

Query: race stats of hamilton in f1 season 2009

Response: <table> (API response)

 

 

How does SUSI do that? Let us look at the skill teaching SUSI to give table responses.

# Returns race stats as a table

race summary of  * in f1 season *|race stats of  * in f1 season *
!console:
{
  "url":"http://ergast.com/api/f1/$2$/drivers/$1$/status.json",
  "path":"$.MRData.StatusTable.Status",
  "actions":[{
     "type":"table",
     "columns":{"status":"Race Status","count":"Number Of Races"}
   }]
}
eol

Here, we are telling SUSI that the data type is a table through type attribute in actions and also defining column names and under which column each value must be put using their respective keys. Using this information SUSI generates a response accordingly with the table schema and data points.

How do we know when to render a table?

We know it through the type attribute in the actions from the API response.

"actions": [{
  "type": "table",
  "columns": {
    "status": "Race Status",
    "count": "Number Of Races"
  },
  "count": -1
  }]
}],

We can see that the type is table so we now know that we have to render a table.

But what is the table schema? What do we fill it with?

There is a columns key under actions and from the value of the columns key we get a object whose key value pairs give us column names and what data to put under each column.

Here, we have two columns – Race Status and Number Of Races

And the data to put under each column is found in answers[0].data with same keys as those for each column name i.e ‘status’ and ‘count’.

Sample data object from the API response:

{
  "statusId": "2",
  "count": "1",
  "status": "Disqualified"
}

The value under ‘status’ key is ‘Disqualified’ and the column name for ‘status’ key is ‘Race Status’, so Disqualified is entered under Race Status column in the table. Similarly 1  is entered under Number Of Races column. We thus have a row of our table. We populate the table for each object in the data array using the same procedure.

let coloumns = data.answers[0].actions[index].columns;
let count = data.answers[0].actions[index].count;
let table = drawTable(coloumns,data.answers[0].data,count);

We also have a ’count’ attribute in the API response . This is used to denote how many rows to populate in the table. If count = -1 , then it means infinite or to display all the results.

function drawTable(coloumns,tableData,count){

let parseKeys;
let showColName = true;

if(coloumns.constructor === Array){
  parseKeys = coloumns;
  showColName = false;
}
else{
  parseKeys = Object.keys(coloumns);
}

let tableheader = parseKeys.map((key,i) =>{
return(<TableHeaderColumn key={i}>{coloumns[key]}</TableHeaderColumn>);
});

let rowCount = tableData.length;

if(count > -1){
  rowCount = Math.min(count,tableData.length);
}

let rows = [];

for (var j=0; j < rowCount; j++) {

  let eachrow = tableData[j];

  let rowcols = parseKeys.map((key,i) =>{
    return(
        <TableRowColumn key={i}>
          <Linkify properties={{target:'_blank'}}>
            {eachrow[key]}
          </Linkify>
        </TableRowColumn>
      );
  });

  rows.push(
      <TableRow key={j}>{rowcols}</TableRow>
  );

}

const table =
  <MuiThemeProvider>
    <Table selectable={false}>
      <TableHeader displaySelectAll={false} adjustForCheckbox={false}>
        { showColName && <TableRow>{tableheader}</TableRow>}
      </TableHeader>
      <TableBody displayRowCheckbox={false}>{rows}</TableBody>
    </Table>
  </MuiThemeProvider>

return table;

}

Here we first determine how many rows to populate using the count attribute and then parse the columns to get the column names and keys. We then loop through the data and populate each row.

This is how SUSI responds with tabulated data!

You can create your own table skill and SUSI will give the tabulated response you need. Check out this tutorial to know more about SUSI and the various other action types it supports.

Resources

Continue ReadingHow SUSI AI Tabulates Answers For You