Personal Assistant

Read more about the article Connecting SUSI iOS App to SUSI Smart Speaker

Connecting SUSI iOS App to SUSI Smart Speaker

SUSI Smart Speaker is an Open Source speaker with many exciting features. The user needs an Android or iOS device to set up the speaker. You can refer this post for initial connection to SUSI Smart Speaker. In this post, we will see how a user can connect SUSI Smart Speaker to iOS devices (iPhone/iPad). Implementation - The first step is to detect whether an iOS device connects to SUSI.AI hotspot or not. For this, we match the currently connected wifi SSID with SUSI.AI hotspot SSID. If it matches, we show the connected device in Device Activity to proceed further with setups. Choosing Room - Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them. When the user clicks on Wi-Fi displayed cell, it starts the initial setups. We are using didSelectRowAt method of UITableViewDelegate to get which cell is selected. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field. override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) { if indexPath.row == 0, let speakerSSID = fetchSSIDInfo(), speakerSSID == ControllerConstants.DeviceActivity.susiSSID { // Open a popup to select Rooms presentRoomsPopup() } } When the user clicks the Next button, we send the speaker room location to the local server of the speaker by the following API endpoint with room name as a parameter: http://10.0.0.1:5000/speaker_config/ Refer this post for getting more detail about how choosing room work and how it is implemented in SUSI iOS. Sharing Wi-Fi Credentials - On successfully choosing the room, we present a popup that asks the user to enter the Wi-Fi credentials of previously connected Wi-Fi so that we can connect our Smart Speaker to the wifi which can provide internet connection to play music and set commands over the speaker. We present a popup with a text field for entering wifi password. When the user clicks the Next button, we share the wifi credentials to wifi by the following API endpoint: http://10.0.0.1:5000/wifi_credentials/ With the following params- Wifissid - Connected Wi-Fi SSID Wifipassd - Connected Wi-Fi password In this API endpoint, we are sharing wifi SSID and wifi password with Smart Speaker. If the credentials successfully accepted by speaker than we present a popup for user SUSI account password, otherwise we again present Enter Wifi Credentials popup. Client.sharedInstance.sendWifiCredentials(params) { (success, message) in DispatchQueue.main.async { self.alertController.dismiss(animated: true, completion: nil) if success { self.presentUserPasswordPopup() } else { self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in UIApplication.shared.endIgnoringInteractionEvents() self.presentWifiCredentialsPopup() }) } } }   Sharing SUSI Account Credentials - In the method above we have seen that when SUSI Smart Speaker accept the wifi credentials, we proceed further with SUSI account credentials. We open a popup to Enter user’s SUSI account password: When the user clicks the Next button, we use following API endpoint to share user’s SUSI account credentials to SUSI Smart Speaker: http://10.0.0.1:5000/auth/…

Continue ReadingConnecting SUSI iOS App to SUSI Smart Speaker
Read more about the article Skill Development using SUSI Skill CMS

Skill Development using SUSI Skill CMS

There are a lot of personal assistants around like Google Assistant, Apple’s Siri, Windows’ Cortana, Amazon’s Alexa, etc. What is then special about SUSI.AI which makes it stand apart from all the different assistants in the world? SUSI is different as it gives users the ability to create their own skills in a Wiki-like system. You don’t need to be a developer to be able to enhance SUSI. And, SUSI is an Open Source personal assistant which can do a lot of incredible stuff for you, made by you. So, let’s say you want to create your own Skill and add it to the existing SUSI Skills. So, these are the steps you need to follow regarding the same - The current SUSI Skill Development Environment is based on an Etherpad. An Etherpad is a web-based collaborative real-time editor. https://dream.susi.ai/ is one such Etherpad. Open https://dream.susi.ai/ and name your dream (in lowercase letters). Define your skill in the Etherpad. The general skill format is ::name <Skill_name> ::author <author_name> ::author_url <author_url> ::description <description> ::dynamic_content <Yes/No> ::developer_privacy_policy <link> ::image <image_url> ::term_of_use <link> #Intent User query1|query2|query3.... Answer answer1|answer2|answer3...   Patterns in query can be learned easily via this tutorial. Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad. Once you’ve tested your skill, write ‘stop dreaming’ to disable dreaming for SUSI. If the testing was successful and you want your skill to be added to SUSI Skills, send a Pull Request to susi_skill_data repository providing your dream name. How do you modify an existing skill? SUSI Skill CMS is a web interface where you can modify the skills you’ve made. All the skills of SUSI are directly in sync with the susi_skill_data. To edit any skill, you need to follow these steps - Login to SUSI Skill CMS website using your email and password (or Sign Up to the website if you haven’t already). Click on the skill which you want to edit and then click on the “edit” icon. You can edit all aspects of the skill in the next state. Below is a preview: Make the changes and then click on “SAVE” button to save the skill. What’s happening Behind The Scenes in the EDIT process? SkillEditor.js is the file which is responsible for keeping a check over various validations in the Skill Editing process. There are certain validations that need to be made in the process. Those are as follows - Check whether User has logged in or not if (!cookies.get('loggedIn')) { notification.open({ message: 'Not logged In', description: 'Please login and then try to create/edit a skill', icon: <Icon type='close-circle' style={{ color: '#f44336' }} />, }); this.setState({ loading: false }); return 0; }   Check whether Commit Message has been entered by User or not if (this.state.commitMessage === null) { notification.open({ message: 'Please add a commit message', icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,…

Continue ReadingSkill Development using SUSI Skill CMS
Read more about the article Implementing Version Control System for SUSI Skill CMS

Implementing Version Control System for SUSI Skill CMS

SUSI Skill CMS now has a version control system where users can browse through all the previous revisions of a skill and roll back to a selected version. Users can modify existing skills and push the changes. So a skill could have been edited many times by the same or different users and so have many revisions. The version control functionalities help users to : Browse through all the revisions of a selected skill View the content of a selected revision Compare any two selected revisions highlighting the changes Option to edit and rollback to a selected revision. Let us visit SUSI Skill CMS and try it out. Select a skill Click on versions button A table populated with previous revisions is displayed Clicking on a single revision opens the content of that version Selecting 2 versions and clicking on compare selected versions loads the content of the 2 selected revisions and shows the differences between the two. Clicking on Undo loads the selected revision and the latest version of that skill, highlighting the differences and also an editor loaded with the code of the selected revision to make changes and save to roll back. How was this implemented? Firstly, to get the previous revisions of a selected skill, we need the skills meta data including model, group, language and skill name which is used to make an ajax call to the server using the endpoint : http://api.susi.ai/cms/getSkillHistory.json?model=MODEL&group=GROUP&language=LANGUAGE&skill=SKILL_NAME We create a new component SkillVersion and pass the skill meta data in the pathname while accessing that component. The path where SkillVersion component is loaded is /:category/:skill/versions/:lang . We parse this data from the path and set our state with skill meta data. In componentDidMount we use this data to make the ajax call to the server to get all previous version data and update our state. A sample response from getSkillHistory endpoint looks like : { "session": { "identity": { "type": "", "name": "", "anonymous": } }, "commits": [ { "commitRev": "", "author_mail": "AUTHOR_MAIL_ID", "author": "AUTOR_NAME", "commitID": "COMMIT_ID", "commit_message": "COMMIT_MESSAGE", "commitName": "COMMIT_NAME", "commitDate": "COMMIT_DATE" }, ], "accepted": TRUE/FALSE } We now populate the table with the obtained revision history. We used Material UI Table for tabulating the data. The first 2 columns of the table have radio buttons to select any 2 revisions. The left side radio buttons are for selecting the older versions and the right side radio buttons to select the more recent versions. We keep track of the selected versions through onCheck function of the radio buttons and updating state accordingly. if(side === 'right'){ if(!(index >= currLeft)){ rightChecks.fill(false); rightChecks[index] = true; currRight = index; } } else if(side === 'left'){ if(!(index <= currRight)){ leftChecks.fill(false); leftChecks[index] = true; currLeft = index; } } this.setState({ currLeftChecked: currLeft, currRightChecked: currRight, leftChecks: leftChecks, rightChecks: rightChecks, }); Once 2 versions are selected and we click on compare selected versions button, we get the currently selected versions stored from getCheckedCommits function and we are redirected to /:category/:skill/compare/:lang/:oldid/:recentid where we pass the selected 2…

Continue ReadingImplementing Version Control System for SUSI Skill CMS
Read more about the article Fetching Images for RSS Responses in SUSI Web Chat

Fetching Images for RSS Responses in SUSI Web Chat

Initially, SUSI Web Chat rendered RSS action type responses like this: The response from the server initially only contained Title Description Link We needed to improvise the web search & RSS results display and also add images for the results. The web search & RSS results are now rendered as : How was this implemented? SUSI AI uses Yacy to fetchRSSs feeds. Firstly the server using the console process to return the RSS feeds from Yacy needs to be configured to return images too. "yacy":{ "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20title,%20link%20FROM%20yacy%20WHERE%20query=%27java%27;%22", "url":"http://yacy.searchlab.eu/solr/select?wt=yjson&q=", "test":"java", "parser":"json", "path":"$.channels[0].items", "license":"" } In a console process, we provide the URL needed to fetch data from, the query parameter needed to be passed to the URL and the path to look for the answer in the API response. url = <url>   - the URL to the remote JSON service which will be used to retrieve information. It must contain a $query$ string. test = <parameter> - the parameter that will replace the $query$ string inside the given URL. It is required to test the service. Here the URL used is : http://yacy.searchlab.eu/solr/select?wt=yjson&q=QUERY To include images in RSS action responses, we need to parse the images also from the Yacy response. For this, we need to add `image` in the selection rule while calling the console process "process":[ { "type":"console", "expression":"SELECT title,description,link FROM yacy WHERE query='$1$';" } ] Now the response from the server for RSS action type will also include `image` along with title, description, and link. An example response for the query `Google` : { "title": "Terms of Service | Google Analytics \u2013 Google", "description": "Read Google Analytics terms of service.", "link": "http://www.google.com/analytics/terms/", "image": "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png", } However, the results at times, do not contain images because there are none stored in the index. This may happen if the result comes from p2p transmission within Yacy where no images are transmitted. So in cases where images are not returned by the server, we use the link preview service to preview the link and fetch the image. The endpoint for previewing the link is : BASE_URL+'/susi/linkPreview.json?url=URL' On the client side, we first search the response for data objects with images in API actions. And the amongst the remaining data objects in answers[0].data, we preview the link to fetch image keeping a check on the count. This needs to be performed for processing the history cognitions too.To preview the remaining links in a loop, we cannot make ajax calls directly in a loop. To handle this, nested ajax calls are made using the function previewURLForImage() where we loop through the remaining links and on the success we decrement the count and call previewURLForImage() on the next link and on error we try previewURLForImage() on the next link without decrementing the count. success: function (rssResponse) { if(rssResponse.accepted){ respData.image = rssResponse.image; respData.descriptionShort = rssResponse.descriptionShort; receivedMessage.rssResults.push(respData); } if(receivedMessage.rssResults.length === count || j === remainingDataIndices.length - 1){ let message = ChatMessageUtils.getSUSIMessageData(receivedMessage, currentThreadID); ChatAppDispatcher.dispatch({ type: ActionTypes.CREATE_SUSI_MESSAGE, message }); } else{ j+=1; previewURLForImage(receivedMessage,currentThreadID, BASE_URL,data,count,remainingDataIndices,j); } }, And we store…

Continue ReadingFetching Images for RSS Responses in SUSI Web Chat
Read more about the article Implementing Text To Speech Settings in SUSI WebChat

Implementing Text To Speech Settings in SUSI WebChat

SUSI Web Chat has Text to Speech (TTS) Feature where it gives voice replies for user queries. The Text to Speech functionality was added using Speech Synthesis Feature of the Web Speech API. The Text to Speech Settings were added to customise the speech output by controlling features like : Language Rate Pitch Let us visit SUSI Web Chat and try it out. First, ensure that the settings have SpeechOutput or SpeechOutputAlways enabled. Then click on the Mic button and ask a query. SUSI responds to your query with a voice reply. To control the Speech Output, visit Text To Speech Settings in the /settings route. First, let us look at the language settings. The drop down list for Language is populated when the app is initialised. speechSynthesis.onvoiceschanged function is triggered when the app loads initially. There we call speechSynthesis.getVoices() to get the voice list of all the languages currently supported by that particular browser. We store this in MessageStore using ActionTypes.INIT_TTS_VOICES action type. window.speechSynthesis.onvoiceschanged = function () { if (!MessageStore.getTTSInitStatus()) { var speechSynthesisVoices = speechSynthesis.getVoices(); Actions.getTTSLangText(speechSynthesisVoices); Actions.initialiseTTSVoices(speechSynthesisVoices); } }; We also get the translated text for every language present in the voice list for the text - `This is an example of speech synthesis` using google translate API. This is called initially for all the languages and is stored as translatedText attribute in the voice list for each element. This is later used when the user wants to listen to an example of speech output for a selected language, rate and pitch. https://translate.googleapis.com/translate_a/single?client=gtx&sl=en-US&tl=TARGET_LANGUAGE_CODE&dt=t&q=TEXT_TO_BE_TRANSLATED When the user visits the Text To Speech Settings, then the voice list stored in the MessageStore is retrieved and the drop down menu for Language is populated. The default language is fetched from UserPreferencesStore and the default language is accordingly highlighted in the dropdown. The list is parsed and populated as a drop down using populateVoiceList() function. let voiceMenu = voices.map((voice,index) => { if(voice.translatedText === null){ voice.translatedText = this.speechSynthesisExample; } langCodes.push(voice.lang); return( <MenuItem value={voice.lang} key={index} primaryText={voice.name+' ('+voice.lang+')'} /> ); }); The language selected using this dropdown is only used as the language for the speech output when the server doesn’t specify the language in its response and the browser language is undefined. We then create sliders using Material UI for adjusting speech rate and pitch. <h4 style={{'marginBottom':'0px'}}><Translate text="Speech Rate"/></h4> <Slider min={0.5} max={2} value={this.state.rate} onChange={this.handleRate} /> The range for the sliders is : Rate : 0.5 - 2 Pitch : 0 - 2 The default value for both rate and pitch is 1. We create a controlled slider saving the values in state and using onChange function to record change in values. The Reset buttons can be used to reset the rate and pitch values respectively to their default values. Once the language, rate and pitch values have been selected we can click on `Play a short demonstration of speech synthesis`  to listen to a voice reply with the chosen settings. { this.state.playExample && ( <VoicePlayer play={this.state.play} text={voiceOutput.voiceText} rate={this.state.rate} pitch={this.state.pitch} lang={this.state.ttsLanguage} onStart={this.onStart} onEnd={this.onEnd} /> ) }…

Continue ReadingImplementing Text To Speech Settings in SUSI WebChat
Read more about the article Generating Map Action Responses in SUSI AI

Generating Map Action Responses in SUSI AI

SUSI AI responds to location related user queries with a Map action response. The different types of responses are referred to as actions which tell the client how to render the answer. One such action type is the Map action type. The map action contains latitude, longitude and zoom values telling the client to correspondingly render a map with the given location. Let us visit SUSI Web Chat and try it out. Query: Where is London Response: (API Response) The API Response actions contain text describing the specified location, an anchor with text ‘Here is a map` linked to openstreetmaps and a map with the location coordinates. Let us look at how this is implemented on server. For location related queries, the key where is used as an identifier. Once the query is matched with this key, a regular expression `where is (?:(?:a )*)(.*)` is used to parse the location name. "keys"   : ["where"], "phrases": [ {"type":"regex", "expression":"where is (?:(?:a )*)(.*)"}, ] The parsed location name is stored in $1$ and is used to make API calls to fetch information about the place and its location. Console process is used to fetch required data from an API. "process": [ { "type":"console", "expression":"SELECT location[0] AS lon, location[1] AS lat FROM locations WHERE query='$1$';"}, { "type":"console", "expression":"SELECT object AS locationInfo FROM location-info WHERE query='$1$';"} ], Here, we need to make two API calls : For getting information about the place For getting the location coordinates First let us look at how a Console Process works. In a console process we provide the URL needed to fetch data from, the query parameter needed to be passed to the URL and the path to look for the answer in the API response. url = <url>   - the url to the remote json service which will be used to retrieve information. It must contain a $query$ string. test = <parameter> - the parameter that will replace the $query$ string inside the given url. It is required to test the service. For getting the information about the place, we used Wikipedia API. We name this console process as location-info and added the required attributes to run it and fetch data from the API. "location-info": { "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20*%20FROM%20location-info%20WHERE%20query=%27london%27;%22", "url":"https://en.wikipedia.org/w/api.php?action=opensearch&limit=1&format=json&search=", "test":"london", "parser":"json", "path":"$.[2]", "license":"Copyright by Wikipedia, https://wikimediafoundation.org/wiki/Terms_of_Use/en" } The attributes used are : url : The Media WIKI API endpoint test : The Location name which will be appended to the url before making the API call. parser : Specifies the response type for parsing the answer path : Points to the location in the response where the required answer is present The API endpoint called is of the following format : https://en.wikipedia.org/w/api.php?action=opensearch&limit=1&format=json&search=LOCATION_NAME For the query where is london, the API call made returns [ "london", ["London"], ["London  is the capital and most populous city of England and the United Kingdom."], ["https://en.wikipedia.org/wiki/London"] ] The path $.[2] points to the third element of the array i.e "London  is the capital and most populous city of England and the United Kingdom.” which…

Continue ReadingGenerating Map Action Responses in SUSI AI
Read more about the article Adding Push Wake Button to SUSI on Raspberry PI

Adding Push Wake Button to SUSI on Raspberry PI

SUSI Linux for Raspberry Pi provides the ability to call SUSI with the help of a Hotword ‘Susi’. Calling via Hotword is a natural way of interaction but it is even handier to invoke SUSI listening mode with the help of a Push button. It enables to call SUSI in a noisy environment where detection of Hotword is not that accurate. To enable Push Wake button is Susi, we need access to Hardware Pins. Devices like Raspberry PI provides GPIO (General Purpose Input Output) Pins for interacting with Hardware Devices. In this tutorial, we are adding support for Push Wake Button in Raspberry PI, though similar procedure can be extended to add Wake Button to Orange Pi, Beaglebone Black, and other devices. For adding push wake button, we require: Tactile Push Button Male to Female Jumper wires Breadboard Raspberry Pi We now need to do wiring to connect button to Raspberry Pi. The button can be connected to Raspberry Pi following the connection diagram.  After this, we need to install the Raspberry Pi GPIO Python Library. Install it using: $ pip3 install RPi.GPIO Now, we may detect the press of the button in our code. We declare an abstract class for implementing Wake Button. In this way, we can later extend our code to include Wake Buttons for more platforms. import os from abc import ABC, abstractclassmethod from queue import Queue from threading import Thread from utils.susi_config import config class WakeButton(ABC, Thread): def __init__(self, detection_callback, callback_queue: Queue): super().__init__() self.detection_callback = detection_callback self.callback_queue = callback_queue self.is_active = False @abstractclassmethod def run(self): pass def on_detected(self): if self.is_active: self.callback_queue.put(self.detection_callback) os.system('play {0} &'.format(config['detection_bell_sound'])) self.is_active = False We defined WakeButton class as a Thread. This is done to ensure that listening to Wake Buttons is done in background thread and it does not disturb the main thread. The callback to be executed on main thread after button press is detected is added to callback queue. Main Thread listens on the callback queue and executes any pending functions from other threads. We also play an Audio File additionally on detection of a button press to confirm the activation of detection to the user. Now, we define Raspberry Pi Wake Button class. This class extends from abstract WakeButton declared above. from queue import Queue import RPi.GPIO as GPIO import time from .wake_button import WakeButton class RaspberryPiWakeButton(WakeButton): def __init__(self, detection_callback, callback_queue: Queue): super().__init__(detection_callback, callback_queue) GPIO.setmode(GPIO.BCM) GPIO.setup(18, GPIO.IN, pull_up_down=GPIO.PUD_UP) def run(self): while True: input_state = GPIO.input(18) if not input_state: self.on_detected() self.is_active = False time.sleep(0.2) This class defines the Wake Button for Raspberry Pi. We continuously poll for the input value of GPIO Pin number 18 on which button is connected. If value is negative, it indicated that button was pressed. Now, we need to add an option if configuration script to give users a choice to enable or disable wake button. We first need to check, if device is Raspberry Pi, since feature is available on Raspberry PI only. To do this, we try to import RPi.GPIO module. If module…

Continue ReadingAdding Push Wake Button to SUSI on Raspberry PI
Read more about the article Implementing the Feedback Functionality in SUSI Web Chat

Implementing the Feedback Functionality in SUSI Web Chat

SUSI AI now has a feedback feature where it collects user’s feedback for every response to learn and improve itself. The first step towards guided learning is building a dataset through a feedback mechanism which can be used to learn from and improvise the skill selection mechanism responsible for answering the user queries. The flow behind the feedback mechanism is : For every SUSI response show thumbs up and thumbs down buttons. For the older messages, the feedback thumbs are disabled and only display the feedback already given. The user cannot change the feedback already given. For the latest SUSI response the user can change his feedback by clicking on thumbs up if he likes the response, else on thumbs down, until he gives a new query. When the new query is given by the user, the feedback recorded for the previous response is sent to the server. Let’s visit SUSI Web Chat and try this out. We can find the feedback thumbs for the response messages. The user cannot change the feedback he has already given for previous messages. For the latest message the user can toggle feedback until he sends the next query. How is this implemented? We first design the UI for feedback thumbs using Material UI SVG Icons. We need a separate component for the feedback UI because we need to store the state of feedback as positive or negative because we are allowing the user to change his feedback for the latest response until a new query is sent. And whenever the user clicks on a thumb, we update the state of the component as positive or negative accordingly. import ThumbUp from 'material-ui/svg-icons/action/thumb-up'; import ThumbDown from 'material-ui/svg-icons/action/thumb-down'; feedbackButtons = ( <span className='feedback' style={feedbackStyle}> <ThumbUp onClick={this.rateSkill.bind(this,'positive')} style={feedbackIndicator} color={positiveFeedbackColor}/> <ThumbDown onClick={this.rateSkill.bind(this,'negative')} style={feedbackIndicator} color={negativeFeedbackColor}/> </span> ); The next step is to store the response in Message Store using saveFeedback Action. This will help us later to send the feedback to the server by querying it from the Message Store. The Action calls the Dispatcher with FEEDBACK_RECEIVED ActionType which is collected in the MessageStore and the feedback is updated in the Message Store. let feedback = this.state.skill; if(!(Object.keys(feedback).length === 0 && feedback.constructor === Object)){ feedback.rating = rating; this.props.message.feedback.rating = rating; Actions.saveFeedback(feedback); } case ActionTypes.FEEDBACK_RECEIVED: { _feedback = action.feedback; MessageStore.emitChange(); break; } The final step is to send the feedback to the server. The server endpoint to store feedback for a skill requires other parameters apart from feedback to identify the skill. The server response contains an attribute `skills` which gives the path of the skill used to answer that query. From that path we need to parse : Model : Highest level of abstraction for categorising skills Group : Different groups under a model Language : Language of the skill Skill : Name of the skill For example, for the query `what is the capital of germany` , the skills object is "skills": ["/susi_skill_data/models/general/smalltalk/en/English-Standalone-aiml2susi.txt"] So, for this skill, Model : general Group : smalltalk Language : en Skill :…

Continue ReadingImplementing the Feedback Functionality in SUSI Web Chat
Read more about the article Adding a Scroll To Bottom button in SUSI WebChat

Adding a Scroll To Bottom button in SUSI WebChat

SUSI Web Chat now has a scroll-to-bottom button which helps the users to scroll the app automatically to the bottom of the scroll area on button click. When the chat history is lengthy and the user has to scroll down manually it results in a bad UX. So the basic requirements of this scroll-to-bottom button are: The button must only be displayed when the user has scrolled up the message section On clicking the scroll-to-bottom button, the scroll area must be automatically scrolled to bottom. Let’s visit SUSI Web Chat and try this out. The button is not visible until there are enough messages to enable scrolling and the user has scrolled up. On clicking the button, the app automatically scrolls to the bottom pointing to the most recent message. How was this implemented? We first design our scroll-to-bottom button using Material UI  Floating Action Button and SVG Icons. import FloatingActionButton from 'material-ui/FloatingActionButton'; import NavigateDown from 'material-ui/svg-icons/navigation/expand-more'; The button needs to be styled to be displayed at a fixed position on the bottom right corner of the message section. Positioning it on top of MessageSection above the MessageComposer, the button is also aligned with respect to the edges. const scrollBottomStyle = { button : { float: 'right', marginRight: '5px', marginBottom: '10px', boxShadow:'none', }, backgroundColor: '#fcfcfc', icon : { fill: UserPreferencesStore.getTheme()==='light' ? '#90a4ae' : '#7eaaaf' } } The button must only be displayed when the user has scrolled up. To implement this we need a state variable showScrollBottom which must be set to true or false accordingly based on the scroll offset. {this.state.showScrollBottom && <div className='scrollBottom'> <FloatingActionButton mini={true} style={scrollBottomStyle.button} backgroundColor={scrollBottomStyle.backgroundColor} iconStyle={scrollBottomStyle.icon} onTouchTap={this.forcedScrollToBottom}> <NavigateDown /> </FloatingActionButton> </div> } Now we have to set our state variable showScrollBottom corresponding to the scroll offset. It must be set to true is the user has scrolled up and false if the scrollbar is already at the bottom. To implement this we need to listen to the scrolling events. We used react-custom-scrollbars for the scroll area wrapping the message section. We can listen to the scrolling events using the onScroll props. We also need to tag the scroll area using refs to access the scroll area instead of using findDOMNode as it is being deprecated. import { Scrollbars } from 'react-custom-scrollbars'; <Scrollbars ref={(ref) => { this.scrollarea = ref; }} onScroll={this.onScroll} > {messageListItems} </Scrollbars> Now, whenever a scroll action is performed, the onScroll() function is triggered. We now have to know if the scroll bar is at the bottom or not. We make use of the scroll area’s props to get the scroll offsets. The getValues() function returns an object containing different scroll offsets and scroll area dimensions. We are interested in values.top which tells about the scroll-top’s progress from 0 to 1 i.e when the scroll bar is at the top most point values.top is 0 and when it’s on the bottom most point, values.top is 1. So whenever values.top is 1, showScrollBottom is false else true. onScroll = () => { let scrollarea = this.scrollarea;…

Continue ReadingAdding a Scroll To Bottom button in SUSI WebChat
Read more about the article Setup SUSI Assistant on Raspberry Pi in under 30 minutes

Setup SUSI Assistant on Raspberry Pi in under 30 minutes

With our ever growing list of list of platforms supported by Susi AI, we now have a client that can run on Raspberry Pi and you can access it hands-free!! Here is a video that you can refer for its working. https://youtu.be/au0MAc0GF6w But it might have left you wondering how you can replicate such a setup yourself? It is fairly easy and will be done fairly easy. Just follow the following instructions. You need to have following hardware in order to have your own SUSI Assistant running on Raspberry Pi. A Raspberry Pi (prefer 2 or 3) with Raspbian Jessie OS. A stable internet connection.  ( Recommended 4 Mbps ) A USB Microphone /  USB Webcam with Microphone. You may buy one like this. A Speaker that connects through 3.5mm jack. You may buy one like this. After you get all the above items in order, you need to get access to a terminal of your Raspberry Pi. You can have that by either connecting a monitor to Raspberry Pi temporarily or by connecting to Raspberry Pi over SSH. Once this is done, next step is the installation of the dependencies. The installation of the SUSI on Raspberry is automated after dependencies are installed. Run the following command on Raspberry Pi terminal. sudo apt install git swig3.0 portaudio19-dev pulseaudio libpulse-dev unzip sox libatlas-dev libatlas-base-dev libsox-fmt-all python3 After this, you may check if your output and input devices are working alright. For this, run rec recording.wav . It will start recording audio and saving it to a file named recording.wav. Play back the file using play recording.wav If you hear your audio clearly, setup is done right else you need to configure your Audio Devices correctly.  Most of the time the configuration of Audio works out the box and devices are plug and play so you would not encounter any errors. If you are successful in configuring your devices, install extra dependencies for SUSI Hardware by running the automated install script. In your terminal run, $ git clone https://github.com/fossasia/susi_hardware.git $ cd susi_hardware $ ./install.sh This will install all the remaining dependencies. After the above step is complete, you may run configuration file generator script to choose the Text to Speech and Speech to Text service according to your wish. For doing so, you need to run $ python3 config_generator.py Follow the instructions in the script. It will ask you to configure the default service for Text to Speech and Speech to Text and other options. After the configuration is complete, you can simply run the following command to start SUSI. $ python3 main.py This will start SUSI in a continuously listening mode. You may invoke SUSI anytime, just by saying SUSI followed by a query. The query will be answered by SUSI subsequently. Since configurations for different hardware devices may vary, you may encounter some problems. In such a scenario, you may refer to the following resources to solve the issues. Resources: For Setting up Audio properly: - Debian Pulseaudio setup guide. For setting up…

Continue ReadingSetup SUSI Assistant on Raspberry Pi in under 30 minutes