Use of Flux Architecture in SUSI.AI

SUSI Web Chat is based on Flux, a React.js Architecture. It provides a simple way for implementing many features in SUSI Web Chat. This blog post explains how Flux Architecture works, and also how it is used in SUSI Web Chat to implement the feature of switching between Light and Dark themes.

What is Flux?

Flux is a micro-architecture for software application development. It is the application architecture that Facebook uses for building client-side web applications. It complements React’s composable view components by utilizing a unidirectional data flow.

Flux Overview

As can be seen from the above diagram, Flux has four main components:

Component Description
Actions Helper methods that pass data to the Dispatcher.
Dispatcher Receives these Actions and broadcast payloads to registered callbacks.
Stores The Stores that are registered to the Dispatcher will update the Views accordingly.
Views  Views are actually React Components that grab the state from the stores and then pass it down to the child components.

How these 4 components work together?

  • Application Initialisation:
    1. Stores let the registered Dispatcher know that they want to be notified whenever an action comes in.
    2. Then the controller views ask the stores for the latest state.
    3. When the stores give the state to the controller views, they pass that state along to their child views to render.
    4. The controller views also ask the stores to keep them notified when state changes so that they can re-render accordingly.
  • The Data Flow:

Once the setup is done, the application is ready to accept user input. So let us trigger an action by having the user make a change.

    1. The view tells the action creator to prepare an action.
    2. The action creator formats the action and sends it off to the Dispatcher.
    3. The Dispatcher dispatches the action off to all of the stores in sequence. Then the store decides whether it cares about this action or not, and changes the state accordingly.
    4. Once it’s done changing state, the store lets its subscribed view controllers know.
    5. Those view controllers will then ask the store to give them the updated state.
    6. After the store gives it the state, the view controller will tell its child views to re-render based on the new state.

So this is how Flux works. Now let us see an example of Flux being used in SUSI Web Chat.

Use of Flux Architecture to Switch Between Themes in SUSI Web Chat

Different Flux components used in SUSI Web Chat:

  1. Actions – Settings.actions.js
  2. Dispatcher – ChatAppDispatcher.js
  3. Store – UserPreferencesStore.js
  4. View – MessageSection.react.js

Let us now see in detail how we are able to switch themes:
In the MessageSection.react.js file, we have the following functions to handle theme change:

handleThemeChanger = () => {
    //code
    this.applyLightTheme();
}

applyLightTheme = () =>{
    //code
    Actions.settingsChanged(settingsChanged);
}

 
Hence, the view tells the action creator to prepare an action.
Now, the settingsChanged() function in Settings.actions.js file is called as follows:

export function settingsChanged(settings) {
    ChatAppDispatcher.dispatch({
    type: ActionTypes.SETTINGS_CHANGED,
    settings
});
    //code
}

 
Hence, the process of invoking the callbacks is initiated through the dispatch() method, which takes a data payload object as its sole argument.

Now, as the UserPreferencesStore is registered with the ChatAppDispatcher, it receives the actions dispatched by the Dispatcher. It checks for the type of action, and with the help of switch cases, executes the code for the corresponding action type accordingly. In this case, the states inside the Store are updated with the new state which the User wants to switch to.

UserPreferencesStore.dispatchToken = ChatAppDispatcher.register(action => {
    switch (action.type) {
        //code
        case ActionTypes.SETTINGS_CHANGED: {
            let settings = action.settings;
            //code
            UserPreferencesStore.emitChange();
            break;
        }
        //code
    }
});

 
This is how Flux Architecture is facilitating the switching of themes in SUSI Web Chat.

Resources

 

Continue Reading

How search works on Susi Skill CMS

The SUSI Skill CMS is a central dashboard where the developers can add skills to make SUSI more intelligent. One of the major thing that was missing there was a search bar. So I recently added one that can search a skill based on:

  • Skill name
  • Skill description
  • Author name
  • Examples of the skill

How to add the search bar?

  1. Install the Material Search Bar component from the terminal.

npm i --save material-ui-search-bar

 

  1. Import this component in the BrowseSkill.js file.

import SearchBar from 'material-ui-search-bar'

 

  1. Create a state variables for the search query and initialize it to an empty string.

this.state = {
	...
	searchQuery:''
	...
};

 

  1. Add the search bar (UI Part) below the filters on CMS. Add a listener function to it that is called when the value of the search query changes.

<SearchBar
     onChange={this.handleSearch}
     value={this.state.searchQuery}
   />

 

  1. The search handler : Create a handler (handleSearch) to that listens to onChange events on the SearchBar. This function sets the value of searhQuery state variable and loads the card again based on the search filter.

handleSearch
  = (value) => {
    this.setState({searchQuery: value}, function () {
    this.loadCards();
    });
};

 

  1. The loadCards() function : The loadCards() function send a request to the Susi Server which in turn send a json response. Then this function makes cards for every skill and adds them to the CMS. Modify the loadCards() function to filter the cards array based on the search query. The javascript string function match() is used to check if the skill name, description, author’s name or examples match the search query. The filter() function adds the skill card to the filtered data if the match() function returns true (i.e., the skill is relevant to the search query).

How the filter works?

  1. First check if there’s something to search for. If the searchQuery.length is equal to zero then that means that there is nothing to search for.

self.state.searchQuery.length >0

 

  1. Then filter the related results based on
  • Skill name : The filter() function adds the skill card to the filtered data if the match function returns true (i.e., the skill is relevant to the search query). The match() function retuns true if the skill name matches the search query.

if (i.skill_name) {
    	result =  i.skill_name.toLowerCase()
        	.match( self.state.searchQuery.toLowerCase() );
    	if (result) {
        	return result;
    	}
}

 

Similarly, filter the cards on the basis of skill description and skill author’s name.

  • Skill examples: Loop over all the skill examples and check if any example matches the search query.

if (i.examples && i.examples.length>0) {
    	i.examples.map((el,j)=>{
      	result =  el.toLowerCase()
        	.match( self.state.searchQuery.toLowerCase() );
      	if (result) {
          	return result;
      	}
    	})
}

Example

Here the search query is “country”. The word “country” appears the skill description of the filtered cards.

Resources

Continue Reading

Automatically deploy SUSI Web Chat on surge after Travis passes

We are using surge from the very beginning of this SUSI web chat and SUSI skill cms projects development. We used surge for provide preview links for Pull requests. Surge is really easy tool to use. We can deploy our static web pages really easily and quickly.  But If user had to change something in pull request user has to deploy again in surge and update the link. If we can connect this operation with travis ci we can minimise re-works. We can embed the deploying commands inside the travis.yml.

We can tell travis to make a preview link (surge deployment) if test cases are passed by embedding the surge deployment commands inside the travis.yml like below.

This is travis.yml file

sudo: required
dist: trusty
language: node_js
node_js:
 - 6
script:
 - npm test
after_success:
 - bash ./surge_deploy.sh
 - bash ./deploy.sh
cache:
 directories:
   - node_modules
branches:
 only:
   - master

Surge deployment commands are inside the “surge_deploy.sh” file.
In that we have to check the status of the pull request whether it is passing test cases or not. We can do it like below.

if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then
   echo "Not a PR. Skipping surge deployment"
   exit 0
fi

Then we have to install surge in the environment. Then after install all npm packages and run build.

npm i -g surge
npm install
npm run build

Since there is a issue with displaying moving to child routes we have to take a copy of index.html file and name it as a 404.html.

cp ./build/index.html ./build/404.html

Then make two environment variables for your surge email address and surge token

export [email protected]
# surge Token (run ‘surge token’ to get token)
export SURGE_TOKEN=d1c28a7a75967cc2b4c852cca0d12206

Now we have to make the surge deployment URL (Domain). It should be unique so we made a URL that contains pull request number.

export DEPLOY_DOMAIN=https://pr-${TRAVIS_PULL_REQUEST}-susi-web-chat.surge.sh
surge --project ./build/ --domain $DEPLOY_DOMAIN;

Since all our static contents which made after the build process are in “build” folder we have to tell surge to get static html files from that.
Now make a pull request. you would find the deployment link in travis ci report after travis passed.

Expand the output of the surge_deploy.sh

You will find the deployment link as we defined in the surge_deploy.sh file

References:

  • Integrating with travis ci – https://surge.sh/help/integrating-with-travis-ci
  • React Routes to Deploy 404 page on gh-pages and surge – https://blog.fossasia.org/react-routes-to-deploy-404-page-on-gh-pages-and-surge/
Continue Reading

Recognise new SUSI users and Welcome them

SUSI web chat application is up and running now. It gives better answers for most of the questions that users ask. But for new users application does not display a welcome message or introduction about the application. It is a distraction for new users. So a new requirement arrived that is to show a welcome message for new users or give them a introduction about the application.

To give a introduction or to show a welcome message we need to identify new users. For that I used cookies.
I added a new dialog to show welcome message and introductory video. Then placed below code in the DialogSection.js file which contains codes about every dialog-box of the application.

 <Dialog
          contentStyle={{ width: '610px' }}
          title="Welcome to SUSI Web Chat"
          open={this.props.tour}
        >
          
            width="560"
            height="315"
            src="https://www.youtube.com/embed/9T3iMoAUKYA"
            gesture="media"
            allow="encrypted-media"
            >
          
          <Close style={closingStyle} onTouchTap={this.props.onRequestCloseTour()} />
        </Dialog>

We already have installed ‘universal-cookie’ npm module in our application so we can use this module to identify cookies.
I used this way to check whether user is new or not.

           <DialogSection
             {...this.props}
             openLogin={this.state.showLogin}
              .
              .
              .
              onRequestCloseTour={()=>this.handleCloseTour}
              tour={!cookies.get('visited')}

           	/>

Now it shows dialog-box for each and every user we don’t need to display the welcome message to old users so we need to store a cookie in users computer.
I stored a cookie in users computer when user clicks on the close button of the welcome dialog-box.
Below function makes new cookie in users computer.

  handleCloseTour = ()=>{
   this.setState({
     showLogin: false,
     showSignUp: false,
     showThemeChanger: false,
     openForgotPassword: false,
     tour:false
    });
    cookies.set('visited', true, { path: '/' });
 }

 

Below line sets a cookie and { path : ’/’ } makes cookie accessible on all pages.

References:

Continue Reading

Implementing Version Control System for SUSI Skill CMS

SUSI Skill CMS now has a version control system where users can browse through all the previous revisions of a skill and roll back to a selected version. Users can modify existing skills and push the changes. So a skill could have been edited many times by the same or different users and so have many revisions. The version control functionalities help users to :

  • Browse through all the revisions of a selected skill
  • View the content of a selected revision
  • Compare any two selected revisions highlighting the changes
  • Option to edit and rollback to a selected revision.

Let us visit SUSI Skill CMS and try it out.

  1. Select a skill
  2. Click on versions button
  3. A table populated with previous revisions is displayed

  1. Clicking on a single revision opens the content of that version
  2. Selecting 2 versions and clicking on compare selected versions loads the content of the 2 selected revisions and shows the differences between the two.
  3. Clicking on Undo loads the selected revision and the latest version of that skill, highlighting the differences and also an editor loaded with the code of the selected revision to make changes and save to roll back.

How was this implemented?

Firstly, to get the previous revisions of a selected skill, we need the skills meta data including model, group, language and skill name which is used to make an ajax call to the server using the endpoint :

http://api.susi.ai/cms/getSkillHistory.json?model=MODEL&group=GROUP&language=LANGUAGE&skill=SKILL_NAME

We create a new component SkillVersion and pass the skill meta data in the pathname while accessing that component. The path where SkillVersion component is loaded is /:category/:skill/versions/:lang . We parse this data from the path and set our state with skill meta data. In componentDidMount we use this data to make the ajax call to the server to get all previous version data and update our state. A sample response from getSkillHistory endpoint looks like :

{
  "session": {
    "identity": {
      "type": "",
      "name": "",
      "anonymous":
    }
  },
  "commits": [
    {
      "commitRev": "",
      "author_mail": "AUTHOR_MAIL_ID",
      "author": "AUTOR_NAME",
      "commitID": "COMMIT_ID",
      "commit_message": "COMMIT_MESSAGE",
     "commitName": "COMMIT_NAME",
     "commitDate": "COMMIT_DATE"
    },
  ],
  "accepted": TRUE/FALSE
}

We now populate the table with the obtained revision history. We used Material UI Table for tabulating the data. The first 2 columns of the table have radio buttons to select any 2 revisions. The left side radio buttons are for selecting the older versions and the right side radio buttons to select the more recent versions. We keep track of the selected versions through onCheck function of the radio buttons and updating state accordingly.

if(side === 'right'){
  if(!(index >= currLeft)){
    rightChecks.fill(false);
    rightChecks[index] = true;
    currRight = index;
  }
}
else if(side === 'left'){
  if(!(index <= currRight)){
    leftChecks.fill(false);
    leftChecks[index] = true;
    currLeft = index;
  }
}
this.setState({
  currLeftChecked: currLeft,
  currRightChecked: currRight,
  leftChecks: leftChecks,
  rightChecks: rightChecks,
});

Once 2 versions are selected and we click on compare selected versions button, we get the currently selected versions stored from getCheckedCommits function and we are redirected to /:category/:skill/compare/:lang/:oldid/:recentid where we pass the selected 2 revisions commitIDs in the URL.

{(this.state.commitsChecked.length === 2) &&
<Link to={{
  pathname: '/'+this.state.skillMeta.groupValue+
            '/'+this.state.skillMeta.skillName+
            '/compare/'+this.state.skillMeta.languageValue+
            '/'+checkedCommits[0].commitID+
            '/'+checkedCommits[1].commitID,
}}>
  <RaisedButton
    label='Compare Selected Versions'
    backgroundColor='#4285f4'
    labelColor='#fff'
    style={compareBtnStyle}
  />
</Link>
}

SkillHistory Component is now loaded and the 2 selected revisions commitIDs are parsed from the URL pathname. Once we have the commitIDs we make ajax calls to the server to get the code for that particular commit. The skill meta data is also parsed from the URL path which is required to make the server call to getFileAtCommitID.

http://api.susi.ai/cms/getSkillHistory.json?model=MODEL&group=GROUP&language=LANGUAGE&skill=SKILL_NAME&commitID=COMMIT_ID

We make the ajax calls in componentDidMount and update the state with the received data. A sample response from getFileAtCommitID looks like :

{
  "accepted": TRUE/FALSE,
  "file": "CONTENT",
  "session": {
    "identity": {
       "type": "",
       "name": "",
       "anonymous":
    }
  }
}

We populate the code of each revision in an editor. We used react-ace as our editor component where we use the value prop to populate the content and display it in read-only mode.

<AceEditor
  mode='java'
  readOnly={true}
  theme={this.state.editorTheme}
  width='100%'
  fontSize={this.state.fontSizeCode}
  height= '400px'
  value={this.state.commitData[0].code}
  showPrintMargin={false}
  name='skill_code_editor'
  editorProps={{$blockScrolling: true}}
/>

We then show the differences between the 2 selected versions content. To compare and highlight the differences, we used react-diff package which takes in the content of both the commits as inputA and inputB props and we compare character by character using the type chars prop. Here input A is compared with input B. The component compares and returns the highlighted element which we display in a scrollable div preventing overflows.

{/* latest code should be inputB */}
<Diff
  inputA={this.state.commitData[0].code}
  inputB={this.state.commitData[1].code}
  type='chars'
/>

Clicking on Undo then redirects to /:category/:skill/edit/:lang/:latestid/:revertid where latest id is the commitID of the latest revision and revert id is the commitID of the oldest commit ID selected amongst the 2 commits selected initially. This redirects to SkillRollBack component where we again parse the skill meta data and the commit IDs from the URL pathname and call getFileAtCommitID to get the content for the latest and the reverting commit and again populate the content in editor using react-ace and also show the differences using react-diff and finally load the modify skill component where an editor is preloaded with the content of the reverting commit and a similar interface like modify skill is shown where user can edit the content of the reverting commit and push the changes.

let baseUrl = this.getSkillAtCommitIDUrl() ;
let self = this;
var url1 = baseUrl + self.state.latestCommit;
$.ajax({
  url: url1,
  jsonpCallback: 'pc',
  dataType: 'jsonp',
  jsonp: 'callback',
  crossDomain: true,
  success: function (data1) {
    var url2 = baseUrl + self.state.revertingCommit;
    $.ajax({
      url: url2,
      jsonpCallback: 'pd',
      dataType: 'jsonp',
      jsonp: 'callback',
      crossDomain: true,
      success: function (data2) {
        self.updateData([{
        code:data1.file,
        commitID:self.state.latestCommit,
      },{
        code:data2.file,
        commitID:self.state.revertingCommit,
      }])
      }
    });
  }
});

Here, we make nested ajax calls to maintain synchronization and update state after we receive data from both the calls else if we make ajax calls in a loop, then the second ajax call doesn’t wait for the first one to finish and is most likely to fail.

This is how the skill version system was implemented in SUSI Skill CMS. You can find the complete code at SUSI Skill CMS Repository. Feel free to contribute.

Resources:

Continue Reading

Fetching Images for RSS Responses in SUSI Web Chat

Initially, SUSI Web Chat rendered RSS action type responses like this:

The response from the server initially only contained

  • Title
  • Description
  • Link

We needed to improvise the web search & RSS results display and also add images for the results.

The web search & RSS results are now rendered as :

How was this implemented?

SUSI AI uses Yacy to fetchRSSs feeds. Firstly the server using the console process to return the RSS feeds from Yacy needs to be configured to return images too.

"yacy":{
  "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20title,%20link%20FROM%20yacy%20WHERE%20query=%27java%27;%22",
  "url":"http://yacy.searchlab.eu/solr/select?wt=yjson&q=",
  "test":"java",
  "parser":"json",
  "path":"$.channels[0].items",
  "license":""
}

In a console process, we provide the URL needed to fetch data from, the query parameter needed to be passed to the URL and the path to look for the answer in the API response.

  • url = <url>   – the URL to the remote JSON service which will be used to retrieve information. It must contain a $query$ string.
  • test = <parameter> – the parameter that will replace the $query$ string inside the given URL. It is required to test the service.

Here the URL used is :

http://yacy.searchlab.eu/solr/select?wt=yjson&q=QUERY

To include images in RSS action responses, we need to parse the images also from the Yacy response. For this, we need to add `image` in the selection rule while calling the console process

"process":[
  {
    "type":"console",
    "expression":"SELECT title,description,link FROM yacy WHERE query='$1$';"
  }
]

Now the response from the server for RSS action type will also include `image` along with title, description, and link. An example response for the query `Google` :

{
  "title": "Terms of Service | Google Analytics \u2013 Google",
  "description": "Read Google Analytics terms of service.",
  "link": "http://www.google.com/analytics/terms/",
  "image":   "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png",
}

However, the results at times, do not contain images because there are none stored in the index. This may happen if the result comes from p2p transmission within Yacy where no images are transmitted. So in cases where images are not returned by the server, we use the link preview service to preview the link and fetch the image.

The endpoint for previewing the link is :

BASE_URL+'/susi/linkPreview.json?url=URL'

On the client side, we first search the response for data objects with images in API actions. And the amongst the remaining data objects in answers[0].data, we preview the link to fetch image keeping a check on the count. This needs to be performed for processing the history cognitions too.To preview the remaining links in a loop, we cannot make ajax calls directly in a loop. To handle this, nested ajax calls are made using the function previewURLForImage() where we loop through the remaining links and on the success we decrement the count and call previewURLForImage() on the next link and on error we try previewURLForImage() on the next link without decrementing the count.

success: function (rssResponse) {
  if(rssResponse.accepted){
    respData.image = rssResponse.image;
    respData.descriptionShort = rssResponse.descriptionShort;
    receivedMessage.rssResults.push(respData);
  }
  if(receivedMessage.rssResults.length === count ||
    j === remainingDataIndices.length - 1){
    let message = ChatMessageUtils.getSUSIMessageData(receivedMessage, currentThreadID);
    ChatAppDispatcher.dispatch({
      type: ActionTypes.CREATE_SUSI_MESSAGE,
      message
    });
  }
  else{
    j+=1;
    previewURLForImage(receivedMessage,currentThreadID,
BASE_URL,data,count,remainingDataIndices,j);
  }
},

And we store the results as rssResults which are used in MessageListItems to fetch the data and render. The nested calling of previewURLForImage() ends when we have the required count of results or we have finished trying all links for previewing images. We then dispatch the message to the message store. We now improvise the UI. I used Material UI Cards to display the results and for the carousel like display, react-slick.

<Card className={cardClass} key={i} onClick={() => {
  window.open(tile.link,'_blank')
}}>
  {tile.image &&
    (
      <CardMedia>
        <img src={tile.image} alt="" className='card-img'/>
      </CardMedia>
    )
  }
  <CardTitle title={tile.title} titleStyle={titleStyle}/>
  <CardText>
    <div className='card-text'>{cardText}</div>
    <div className='card-url'>{urlDomain(tile.link)}</div>
  </CardText>
</Card>

We used the full width of the message section to display the results by not wrapping the result in message-list-item class. The entire card is hyperlinked to the link. Along with title and description, the URL info is also shown at the bottom right. To get the domain name from the link, urlDomain() function is used which makes use of the HTML anchor tag to get the domain info.

function urlDomain(data) {
  var a = document.createElement('a');
  a.href = data;
  return a.hostname;
}

To prevent stretching of images we use `object-fit: contain;` to make the images fit the image container and align it to the middle.

We finally have our RSS results with images and an improvised UI. The complete code can be found at SUSI WebChat Repo. Feel free to contribute

Resources
Continue Reading

Implementing Text To Speech Settings in SUSI WebChat

SUSI Web Chat has Text to Speech (TTS) Feature where it gives voice replies for user queries. The Text to Speech functionality was added using Speech Synthesis Feature of the Web Speech API. The Text to Speech Settings were added to customise the speech output by controlling features like :

  1. Language
  2. Rate
  3. Pitch

Let us visit SUSI Web Chat and try it out.

First, ensure that the settings have SpeechOutput or SpeechOutputAlways enabled. Then click on the Mic button and ask a query. SUSI responds to your query with a voice reply.

To control the Speech Output, visit Text To Speech Settings in the /settings route.

First, let us look at the language settings. The drop down list for Language is populated when the app is initialised. speechSynthesis.onvoiceschanged function is triggered when the app loads initially. There we call speechSynthesis.getVoices() to get the voice list of all the languages currently supported by that particular browser. We store this in MessageStore using ActionTypes.INIT_TTS_VOICES action type.

window.speechSynthesis.onvoiceschanged = function () {
  if (!MessageStore.getTTSInitStatus()) {
    var speechSynthesisVoices = speechSynthesis.getVoices();
    Actions.getTTSLangText(speechSynthesisVoices);
    Actions.initialiseTTSVoices(speechSynthesisVoices);
  }
};

We also get the translated text for every language present in the voice list for the text – `This is an example of speech synthesis` using google translate API. This is called initially for all the languages and is stored as translatedText attribute in the voice list for each element. This is later used when the user wants to listen to an example of speech output for a selected language, rate and pitch.

https://translate.googleapis.com/translate_a/single?client=gtx&sl=en-US&tl=TARGET_LANGUAGE_CODE&dt=t&q=TEXT_TO_BE_TRANSLATED

When the user visits the Text To Speech Settings, then the voice list stored in the MessageStore is retrieved and the drop down menu for Language is populated. The default language is fetched from UserPreferencesStore and the default language is accordingly highlighted in the dropdown. The list is parsed and populated as a drop down using populateVoiceList() function.

let voiceMenu = voices.map((voice,index) => {
  if(voice.translatedText === null){
    voice.translatedText = this.speechSynthesisExample;
  }
  langCodes.push(voice.lang);
  return(
    <MenuItem value={voice.lang}
              key={index}
              primaryText={voice.name+' ('+voice.lang+')'} />
  );
});

The language selected using this dropdown is only used as the language for the speech output when the server doesn’t specify the language in its response and the browser language is undefined. We then create sliders using Material UI for adjusting speech rate and pitch.

<h4 style={{'marginBottom':'0px'}}><Translate text="Speech Rate"/></h4>
<Slider
  min={0.5}
  max={2}
  value={this.state.rate}
  onChange={this.handleRate} />

The range for the sliders is :

  • Rate : 0.5 – 2
  • Pitch : 0 – 2

The default value for both rate and pitch is 1. We create a controlled slider saving the values in state and using onChange function to record change in values. The Reset buttons can be used to reset the rate and pitch values respectively to their default values. Once the language, rate and pitch values have been selected we can click on `Play a short demonstration of speech synthesis`  to listen to a voice reply with the chosen settings.

{ this.state.playExample &&
  (
    <VoicePlayer
       play={this.state.play}
       text={voiceOutput.voiceText}
       rate={this.state.rate}
       pitch={this.state.pitch}
       lang={this.state.ttsLanguage}
       onStart={this.onStart}
       onEnd={this.onEnd}
    />
  )
}

We use the VoicePlayer by passing the required props to get the speech output. onStart and onEnd functions are triggered at the beginning and ending of the speech synthesis and are used to control the state from the parent component. Chosen language, rate, pitch and translated text are passed as props to VoicePlayer which creates a new SpeechSynthesisUtterance() with the passed props and plays the speech output.

On saving these settings and then using the Mic button to get voice replies we see that the voice output is controlled according to the selected settings.

Finally, we have to store the selected settings on the server and ensure that these are pulled when the app is initialized. The format in which these settings are stored in the server is :

Speech Rate

- Used to control rate of speech output.
- SETTING_NAME :  `speechRate`
- SETTING_VALUE : `0.5 - 2`
- DEFAULT_VALUE : `1`
 
Speech Pitch

- Used to control pitch of speech output.
- SETTING_NAME :  `speechPitch`
- SETTING_VALUE : `0 - 2`
- DEFAULT_VALUE : `1`
 
TTS Language

- Used to set the language for Text-To-Speech used when the response from server doesnt specify language and the browser language is also undefined.
- SETTING_NAME :  `ttsLanguage`
- SETTING_VALUE : `Language Code (string)`
- DEFAULT_VALUE : `en-US`

This is how the Text To Speech Settings were implemented in SUSI Web Chat. The complete code can be found at SUSI Web Chat Repository.

PS: To test whether your browser supports Text To Speech, open your browser console and try the following :

  • var msg = new SpeechSynthesisUtterance(‘Hello World’);
  • window.speechSynthesis.speak(msg)

If you get a speech output then the Web API Speech Synthesis is supported by your browser and Text To Speech features of SUSI Web Chat will work. The Web Speech API has support for all latest Chrome browsers as mentioned in the Web Speech API Mozilla docs.However there are few bugs with some Chromium versions please check out more on how to fix them locally here in this link.

Resources:

 

 

Continue Reading

Sorting Users and Implementing Animations on SUSI Web Chat Team Page

While we were developing the chat application, we wanted to show details of Developers.  So we planned to build a team page for SUSI Web Chat Application. In this post I discuss few things we built there. Like sorting feature, animations of the page, usage of Material UI.

First we made an array of objects to store user details. In that array we grouped them in sub arrays so we can refer them in different sections separately. We stored following data in “TeamList.js” in an array.

var team = [{
 'mentors': [{
   'name': 'Mario Behling',
   'github': 'http://github.com/mariobehling',
   'avatar': 'http://fossasia.org/img/mariobehling.jpg',
   'twitter': 'https://twitter.com/mariobehling',
   'linkedin': 'https://www.linkedin.com/in/mariobehling/',
   'blog': '#'
 }]
},{ 'server': [{
    }]
}

There was a requirement to sort developers by their name so we had to build a way to sort out array data which are in main array. This is how we built that.
The function we are going to use for sorting.

   function compare(a, b) {
     if (a.name < b.name) { return -1; }
     if (a.name > b.name) { return 1; }
     return 0;
   }

This is how we have used it to sort developers.

import team from './TeamList';
team[1].server.sort(compare);

In this function we took values of object two by two and compared.
Now we have to show these sorted information on view.
Extract data that we want from array and we used material UI Cards to show these data on view.
This is how we extracted data from the array.

   team[1].server.sort(compare);
   let server = team[1].server.map((serv, i) => {
     return ( <Card className='team-card' key={i}>
         <CardMedia className="container" >
           <img src={serv.avatar} alt={serv.name} className="image" />
           <div className="overlay" >
             <div className="text"> <FourButtons member={serv} /> </div>
           </div>
         </CardMedia>
         <CardTitle title={serv.name} subtitle={serv.designation} />
       </Card>)   })

Now it shows data on the view.
“” contains an image of the member. We need to show social media links of the user on mouseover. We did that using simple CSS. I added a block comment around those particular styles. Check those styles here.

.overlay {
 position: absolute;
 bottom: 100%;
 left: 0;
 right: 0;
 background-color: #000;
 overflow: hidden;
 width: 100%;
 height:0;
 transition: .3s ease;
 opacity:0;
}
.container:hover .overlay {
 bottom: 0;
 height: 100%;
 opacity:0.7;
}

Above lines show that how we made the animation of the overlay animation.

Now we want to show social media buttons on the overlay. We made another separate component for buttons and return those like this.

render() {
       let member= this.props.member;
       return (<div>
         <CardActions>
           <IconButton href={member.github} target="_blank" >
  <CardActions>
		</div>)}

Finally we showed those data on Team page. We returned these JSX from render(){} method.

         <div className="team-header">
           <div className="support__heading">Developers</div>
         </div>
         <div className='team-container'>{server}</div>

I have mentioned few resources which I used to implement these features. If you are willing to contribute SUSI AI Web Chat application. Fork our repository on github.

Resources

Documentation of Array.sort https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort

How to use Image overlay CSS effects: https://www.w3schools.com/howto/howto_css_image_overlay.asp

Continue Reading

Implementation of React Routers in SUSI Web Chat

When we were developing the SUSI Web Chat application we wanted to implement set of static pages with the chat application. In the start we just wanted to navigate  through different static pages and move back to the web chat application. But it takes time to load a new page when user clicks on a link. Our goal was therefore to minimize the loading time by using lazy loading. For that we used react-route .It is standard library for react js.

From the docs:

“React Router keeps your UI synced with the URL. It has a simple API with powerful features like lazy code loading, dynamic route matching, and location transition handling built right in. Make the URL your first thought, not an after-thought.” (https://www.npmjs.com/package/react-router-plus)

We need react-route to be installed in our application first. We can install it using NPM by running this command on project folder.

npm install --save react-router-dom

Next we have to set up our routes. We have two types of headers in our application. One is chat application header, second one is static page header. In static page header we have a navigation to switch between static pages.
First we need to choose the router type because there are two types of routers in react. “” and “” we can use “” in our example because our server can handle dynamic requests. If we are requesting data from static page we should use “” .
We used that in “” and made another new component called “” and used it on “index.js” like this.

import { BrowserRouter as Router } from 'react-router-dom';
Import App from .App;
 ReactDOM.render(
  	<IntlProvider locale={defaultPrefLanguage}>
		<Router> <App /> </Router>
	</IntlProvider>,
  	document.getElementById('root')  );

In “App.js” we can set up routes like this.

        <Switch>
            <Route exact path='/' component={ChatApp}/>
            <Route exact path="/overview" component={Overview} />
            <Route exact path='/blog' component={Blog} />
            <Route exact path="/logout" component={Logout} />
            <Route exact path="/settings" component={Settings} />
            <Route exact path="*" component={NotFound} />
        </Switch>

We use elements to render component when they match with the corresponding path. We use “path” to add router location that we need to match width the component. We use “exact” property to render the component if location exactly matches with the “path”. If we do not use “exact” property it renders when we have child routes after the path like “/blog/1 “ .
We used “” element to group routes.
We can’t use anchor () tags to navigate pages without reloading. We have to use tags instead of that. We have to replace all the places we have used

<a href= ‘URL’>lable name </a>

with this,

<Link to=’URL’>Lable name</Link>   

After doing above changes application will perform faster and it will load all page contents soon after you click the navigation links.

If you would like to join with FOSSASIA and contribute to SUSI Web Chat Application please fork this repository on Github.

Resources

Continue Reading

Adding a Scroll To Bottom button in SUSI WebChat

SUSI Web Chat now has a scroll-to-bottom button which helps the users to scroll the app automatically to the bottom of the scroll area on button click. When the chat history is lengthy and the user has to scroll down manually it results in a bad UX. So the basic requirements of this scroll-to-bottom button are:

  1. The button must only be displayed when the user has scrolled up the message section
  2. On clicking the scroll-to-bottom button, the scroll area must be automatically scrolled to bottom.

Let’s visit SUSI Web Chat and try this out.

The button is not visible until there are enough messages to enable scrolling and the user has scrolled up. On clicking the button, the app automatically scrolls to the bottom pointing to the most recent message.

How was this implemented?

We first design our scroll-to-bottom button using Material UI  Floating Action Button and SVG Icons.

import FloatingActionButton from 'material-ui/FloatingActionButton';
import NavigateDown from 'material-ui/svg-icons/navigation/expand-more';

The button needs to be styled to be displayed at a fixed position on the bottom right corner of the message section. Positioning it on top of MessageSection above the MessageComposer, the button is also aligned with respect to the edges.

const scrollBottomStyle = {
  button : {
    float: 'right',
    marginRight: '5px',
    marginBottom: '10px',
    boxShadow:'none',
  },
  backgroundColor: '#fcfcfc',
  icon : {
    fill: UserPreferencesStore.getTheme()==='light' ? '#90a4ae' : '#7eaaaf'
  }
}

The button must only be displayed when the user has scrolled up. To implement this we need a state variable showScrollBottom which must be set to true or false accordingly based on the scroll offset.

{this.state.showScrollBottom &&
  <div className='scrollBottom'>
    <FloatingActionButton mini={true}
      style={scrollBottomStyle.button}
      backgroundColor={scrollBottomStyle.backgroundColor}
      iconStyle={scrollBottomStyle.icon}
      onTouchTap={this.forcedScrollToBottom}>
      <NavigateDown />
    </FloatingActionButton>
  </div>
}

Now we have to set our state variable showScrollBottom corresponding to the scroll offset. It must be set to true is the user has scrolled up and false if the scrollbar is already at the bottom. To implement this we need to listen to the scrolling events. We used react-custom-scrollbars for the scroll area wrapping the message section. We can listen to the scrolling events using the onScroll props. We also need to tag the scroll area using refs to access the scroll area instead of using findDOMNode as it is being deprecated.

import { Scrollbars } from 'react-custom-scrollbars';

<Scrollbars
  ref={(ref) => { this.scrollarea = ref; }}
  onScroll={this.onScroll}
>
  {messageListItems}
</Scrollbars>

Now, whenever a scroll action is performed, the onScroll() function is triggered. We now have to know if the scroll bar is at the bottom or not. We make use of the scroll area’s props to get the scroll offsets. The getValues() function returns an object containing different scroll offsets and scroll area dimensions. We are interested in values.top which tells about the scroll-top’s progress from 0 to 1 i.e when the scroll bar is at the top most point values.top is 0 and when its on the bottom most point, values.top is 1. So whenever values.top is 1, showScrollBottom is false else true.

onScroll = () => {
  let scrollarea = this.scrollarea;
  if(scrollarea){
    let scrollValues = scrollarea.getValues();
    if(scrollValues.top === 1){
      this.setState({
        showScrollBottom: false,
      });
    }
    else if(!this.state.showScrollBottom){
      this.setState({
        showScrollBottom: true,
      });
    }
  }
}

Finally, we need to scroll the chat app to the bottom on button click. Whenever showScrollBottom is updated, the state is changed, so componentDidUpdate is triggered which calls the _scrollToBottom() function. But we should change this to avoid scrolling to bottom on showScrollBottom update and the user is intending to scroll here. We use the function forcedScrollToBottom to be triggered on clicking the scroll-to-bottom button, which resets the scrollTop value to the height of the scroll area, thus pointing the scrollbar to the bottom.

forcedScrollToBottom = () => {
  let ul = this.scrollarea;
  if (ul) {
    ul.scrollTop(ul.getScrollHeight());
  }
}

We don’t have to worry about resetting showScrollBottom on forced scroll to bottom as the scrolling will trigger the onScroll function where the showScrollBottom state is handled accordingly.

This is how the scroll to bottom button has been implemented in SUSI Web Chat. The entire code can be found at SUSI Web Chat Repository.

Resources

 

Continue Reading
  • 1
  • 2
Close Menu