Processing Text Responses in SUSI Web Chat

SUSI Web Chat client now supports emojis, images, links and special characters. However, these aren’t declared as separate action types i.e the server doesn’t explicitly tell the client that the response contains any of the above features when it sends the JSON response. So the client must parse the text response from server and add support for each of the above mentioned features instead of rendering the plain text as is, to ensure good UX. SUSI Web Chat client parses the text responses to support : HTML Special Entities Images and GIFs URLs and Mail IDs Emojis and Symbols // Proccess the text for HTML Spl Chars, Images, Links and Emojis function processText(text){ if(text){ let htmlText = entities.decode(text); let imgText = imageParse(htmlText); let replacedText = parseAndReplace(imgText); return <Emojify>{replacedText}</Emojify>; }; return text; } Let us write sample skills to test these out. Visit http://dream.susi.ai/ and enter textprocessing. You can then see few sample queries and responses at http://dream.susi.ai/p/textprocessing. Lets visit SUSI WebChat and try it out. Query : dream textprocessing Response: dreaming enabled for textprocessing Query : text with special characters Response:  &para; Here are few “Special Characters&rdquo;! All the special entities notations have been parsed and rendered accordingly! Sometimes we might need to use HTML special characters due to reasons like You need to escape HTML special characters like <, &, or ". Your keyboard does not support the required character. For example, many keyboards do not have em-dash or the copyright symbol. You might be wondering why the client needs to handle this separately as it is generally, automatically converted to relevant HTML character while rendering the HTML. SUSI Web Chat client uses reactjs which has JSX and not HTML. So JSX doesn’t support HTML special characters i.e they aren’t automatically converted to relevant characters while rendering. Hence, the client needs to handle this explicitly. We used the module, html-entities to decode all types of special HTML characters and entities. This module parses the text for HTML entities and replaces them with the relevant character for rendering when used to decode text. import {AllHtmlEntities} from 'html-entities'; const entities = new AllHtmlEntities(); let htmlText = entities.decode(text); Now that the HTML entities are processed, the client then processes the text for image links. Let us now look at how images and gifs are handled. Query : random gif Response: https://media1.giphy.com/media/AAKZ9onKpXog8/200.gif Sometimes, the text contains links for images or gifs and the user would be expecting a media type like image or gif instead of text. So we need to replace those image links with actual images to ensure good UX. This is handled using regular expressions to match image type urls and correspondingly replace them with html img tags so that the response is a image and not URL text. // Parse text for Image URLs function imageParse(stringWithLinks){ let replacePattern = new RegExp([ '((?:https?:\\/\\/)(?:[a-zA-Z]{1}', '(?:[\\w-]+\\.)+(?:[\\w]{2,5}))', '(?::[\\d]{1,5})?\\/(?:[^\\s/]+\\/)', '*(?:[^\\s]+\\.(?:jpe?g|gif|png))', '(?:\\?\\w+=\\w+(?:&\\w+=\\w+)*)?)' ].join(''),'gim'); let splits = stringWithLinks.split(replacePattern); let result = []; splits.forEach((item,key)=>{ let checkmatch = item.match(replacePattern); if(checkmatch){ result.push( <img key={key} src={checkmatch} style={{width:'95%',height:'auto'}} alt=''/>) } else{…

Continue ReadingProcessing Text Responses in SUSI Web Chat

How SUSI AI Searches the Web For You

SUSI is now capable of performing web search to answer your queries. When SUSI doesn't know how to answer your queries, it performs a web search on the client side and displays all the results as horizontally swipeable tiles with each tile giving a brief description and also providing a link to the relevant source. Lets visit SUSI WebChat and try it out. Query : Search for Google Response : <Web Search Results> How does SUSI know when to perform a websearch? It uses action types to identify if a web search is to be performed or not. The API Response is parsed to check for the action types and if a websearch action type is present, then an API call is made using the duckduckgo api with the relevant query and the results are displayed as tiles with : Category : The Topic related to the given query Text : The result from the websearch Image : Image related to the query if present Link : A url redirecting to the relevant source Parsing the actions : Let us look at the API response for a query. Sample Query: search for google Response: <API Response> "actions": [ { "type": "answer", "expression": "Here is a web search result:" }, { "type": "websearch", "query": "google" } ] Note: The API Response has been trimmed to show only the relevant content We find a websearch type action and the query to be searched as google . So we now make a API call using duckduckgo api to get our websearch results. API Call Format : api.duckduckgo.com/?q={query}&format=json API Call for query google : http://api.duckduckgo.com/?q=google&format=json And from the duckduckgo API response we generate our websearch tiles showing relevant data using the fields present in each object. This is the sample object from duckduckgo API response under the RelatedTopics , which we use to create our websearch result tiles. { "Result": "<a href=\"https:\/\/duckduckgo.com\/Google\">Google<\/a> An American multinational technology company specializing in Internet-related services and...", "Icon": { "URL": "https:\/\/duckduckgo.com\/i\/8f85c93f.png", "Height": "", "Width": "" }, "FirstURL": "https:\/\/duckduckgo.com\/Google", "Text": "Google An American multinational technology company specializing in Internet-related services and..." }, Let us look at the code for querying data from the API if (actions.indexOf('websearch')>=0) { let actionIndex = actions.indexOf('websearch'); let query = response.answers[0].actions[actionIndex].query; $.ajax({ url: 'http://api.duckduckgo.com/?format=json&q='+query, dataType: 'jsonp', crossDomain: true, timeout: 3000, async: false, success: function (data) { receivedMessage.websearchresults = data.RelatedTopics; if(data.AbstractText){ let abstractTile = { Text:'', FirstURL:'', Icon:{URL:''}, } abstractTile.Text = data.AbstractText; abstractTile.FirstURL = data.AbstractURL; abstractTile.Icon.URL = data.Image; receivedMessage.websearchresults.unshift(abstractTile); } let message =  ChatMessageUtils.getSUSIMessageData( receivedMessage, currentThreadID); ChatAppDispatcher.dispatch({ type: ActionTypes.CREATE_SUSI_MESSAGE, message }); }, error: function(errorThrown) { console.log(errorThrown); receivedMessage.text = 'Please check your internet connection'; } }); } Here, from the actions object, we get the query needed to search the web. We then make a ajax call using that query to the duckduckgo API. If the API call succeeds then we collect the required data to create tiles as array of objects and store it as websearchresults. and dispatch the message with the websearchresults which gets reflected in the message store and when…

Continue ReadingHow SUSI AI Searches the Web For You

How SUSI AI Tabulates Answers For You

SUSI is an artificial intelligence chat bot that responds to all kinds of user queries. It isn’t any regular chat bot replying in just plain text. It supports various response types which we refer to as ‘actions’. One such action is the “table” type. When the response to a user query contains a list of answers which can be grouped, it is better visualised as a table rather than plain text. Lets visit SUSI WebChat and try it out. In our example we ask SUSI for the 2009 race statistics of British Formula 1 racing driver Lewis Hamilton. Query: race stats of hamilton in f1 season 2009 Response: <table> (API response)     How does SUSI do that? Let us look at the skill teaching SUSI to give table responses. # Returns race stats as a table race summary of  * in f1 season *|race stats of  * in f1 season * !console: { "url":"http://ergast.com/api/f1/$2$/drivers/$1$/status.json", "path":"$.MRData.StatusTable.Status", "actions":[{ "type":"table", "columns":{"status":"Race Status","count":"Number Of Races"} }] } eol Here, we are telling SUSI that the data type is a table through type attribute in actions and also defining column names and under which column each value must be put using their respective keys. Using this information SUSI generates a response accordingly with the table schema and data points. How do we know when to render a table? We know it through the type attribute in the actions from the API response. "actions": [{ "type": "table", "columns": { "status": "Race Status", "count": "Number Of Races" }, "count": -1 }] }], We can see that the type is table so we now know that we have to render a table. But what is the table schema? What do we fill it with? There is a columns key under actions and from the value of the columns key we get a object whose key value pairs give us column names and what data to put under each column. Here, we have two columns - Race Status and Number Of Races And the data to put under each column is found in answers[0].data with same keys as those for each column name i.e ‘status’ and ‘count’. Sample data object from the API response: { "statusId": "2", "count": "1", "status": "Disqualified" } The value under ‘status’ key is ‘Disqualified’ and the column name for ‘status’ key is ‘Race Status’, so Disqualified is entered under Race Status column in the table. Similarly 1  is entered under Number Of Races column. We thus have a row of our table. We populate the table for each object in the data array using the same procedure. let coloumns = data.answers[0].actions[index].columns; let count = data.answers[0].actions[index].count; let table = drawTable(coloumns,data.answers[0].data,count); We also have a ’count’ attribute in the API response . This is used to denote how many rows to populate in the table. If count = -1 , then it means infinite or to display all the results. function drawTable(coloumns,tableData,count){ let parseKeys; let showColName = true; if(coloumns.constructor === Array){ parseKeys = coloumns; showColName…

Continue ReadingHow SUSI AI Tabulates Answers For You

Map Support for SUSI Webchat

SUSI chat client supports map tiles now for queries related to location. SUSI responds with an interactive internal map tile with the location pointed by a marker. It also provides you with a link to open street maps where you can get the whole view of the location using the zooming options provided and also gives the population count for that location. Lets visit SUSI WebChat and try it out. Query : Where is london Response : City of London is a place with a population of 7556900. Here is a map:https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228 Link to Openstreetmap: City of London (Hyperlinked with https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228) <Map Tile> Implementation: How do we know that a map tile is to be rendered? The actions in the API response tell the client what to render. The client loops through the actions array and renders the response for each action accordingly. "actions": [ { "type": "answer", "expression": "City of London is a place with a population of 7556900. Here is a map: https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228" }, { "type": "anchor", "link": "https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228", "text": "Link to Openstreetmap: City of London" }, { "type": "map", "latitude": "51.51279067225417", "longitude": "-0.09184009399817228", "zoom": "13" } ] Note: The API response has been trimmed to show only the relevant content. The first action element is of type answer so the client renders the text response, ‘City of London is a place with a population of 7556900. Here is a map: https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228’ The second action element is of type anchor with the text to display and the link to hyperlink specified by the text and link attributes, so the client renders the text `Link to Openstreetmap: City of London`, hyperlinked to "https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228". Finally, the third action element is of type map. Latitude, Longitude and zoom level information are also  specified using latitude, longitude and zoom attributes. The client renders a map using these attributes. I used ‘react-leaflet’ module to render the interactive map tiles. To integrate it into our project and set the required style for the map tiles, we need to load Leaflet’s CSS style sheet and we also need to include height and width for the map component.  <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet/v0.7.7/leaflet.css" /> .leaflet-container { height: 150px; width: 80%; margin: 0 auto; } case 'map': { let lat = parseFloat(data.answers[0].actions[index].latitude); let lng = parseFloat(data.answers[0].actions[index].longitude); let zoom = parseFloat(data.answers[0].actions[index].zoom); let mymap = drawMap(lat,lng,zoom); listItems.push( <li className='message-list-item' key={action+index}> <section className={messageContainerClasses}> {mymap} <p className='message-time'> {message.date.toLocaleTimeString()} </p>; </section> </li> ); break; } import { divIcon } from 'leaflet'; import { Map, Marker, Popup, TileLayer } from 'react-leaflet'; // Draw a Map function drawMap(lat,lng,zoom){ let position = [lat, lng]; const icon = divIcon({ className: 'map-marker-icon', iconSize: [35, 35] }); const map = ( <Map center={position} zoom={zoom}> <TileLayer attribution='' url='http://{s}.tile.osm.org/{z}/{x}/{y}.png'      /> <ExtendedMarker position={position} icon={icon}> <Popup> <span><strong>Hello!</strong> <br/> I am here.</span> </Popup> </ExtendedMarker> </Map> ); return map; } Here, I used a custom marker icon because the default icon provided by leaflet had an issue and was not being rendered. I used divIcon from leaflet to create a custom map marker icon. When the…

Continue ReadingMap Support for SUSI Webchat

Hyperlinking Support for SUSI Webchat

SUSI responses can contain links or email ids. Whenever we want to access those links or send a mail to those email ids, it is very inconvenient for the user to manually copy the link and check out the contents, which is a very bad UX. I used a module called ‘react-linkify’ to address this issue. ‘React-linkify’ is a React component to parse links (urls, emails, etc.) in text into clickable links. Usage: <Linkify>{text to linkify}</Linkify> Any link that appears inside the linkify component will be hyperlinked and is made clickable. It uses regular expressions and pattern matching to detect URLs and mail ids and are made clickable such that clicking on URLs opens the link in a new window and clicking a mail id opens “mailto:” . Code: export const parseAndReplace = (text) => {return <Linkify properties={{target:"_blank"}}>{text}</Linkify>;} Lets visit SUSI WebChat and try it out. Query: search internet Response: Internet The global system of interconnected computer networks that use the Internet protocol suite to... https://duckduckgo.com/Internet The link has been parsed from the response text and has been successfully hyperlinked. Clicking the links opens the respective URL in a new window. Resources Linkify Library Examples Using React-Linkify

Continue ReadingHyperlinking Support for SUSI Webchat

How to teach SUSI skills calling an External API

SUSI is an intelligent  personal assistant. SUSI can learn skills to understand and respond to user queries better. A skill is taught using rules. Writing rules is an easy task and one doesn’t need any programming background too. Anyone can start contributing. Check out these tutorials and do watch this video to get started and start teaching susi. SUSI can be taught to call external API’s to answer user queries. While writing skills we first mention string patterns to match the user's query and then tell SUSI what to do with the matched pattern. The pattern matching is similar to regular expressions and we can also retrieve the matched parameters using $<parameter number>$ notation. Example : My name is * Hi $1$! When the user inputs “My name is Uday” , it is matched with “My name is *” and “Uday” is stored in $1$. So the output given is “Hi Uday!”. SUSI can call an external API to reply to user query. An API endpoint or url when called must return a JSON or JSONP response for SUSI to be able to parse the response and retrieve the answer. Rule Format for a skill calling an external API The rule format for calling an external API is : <regular expression for pattern matching> !console: <return answer using $object$ or $required_key$> { “url”: “<API endpoint or url>”, “path”: “$.<key in the API response to find the answer>”, } eol Url is the API endpoint to be called which returns a JSON or JSONP response. The parameters to the url if any can be added using $$ notation. Path is used to help susi know where to look for the answer in the returned response. If the path points to a root element, then the answer is stored in $object$, otherwise we can query $key$ to get the answer which is a value to the key under the path. eol or end of line indicates the end of the rule. Understanding the Path Attribute Let us understand the Path attribute better through some test cases. In each of the test cases we discuss what the path should be and how to retrieve the answer for a given required answer from the json response of an API. API response in json : { “Key1” : “Value1” } Required answer : Value1 Path : “$.Key1    =>   Retrieve Answer:  $object$   API response in json : { “Key1” : [{“Key11” : “Value11”}] } Required answer : Value11 Path : $.Key1[0]   =>  Retrieve Answer: $Key11$ Path : $.Key1[0].Key11   => Retrieve Answer: $object$   API response in json : { “Key1” : {“Key11” : “Value11”} } Required answer : Value11 Path : $.Key1  => Retrieve Answer:  $Key11$ Path : $.Key1.Key11  => Retrieve Answer: $object$   API response in json : { “Key1” : { “Key11” : “Value11”, “Key12” : “Value12” } } Required answer : Value11 , Value12 Path : $.Key1  => Retrieve Answer:  $Key11$ , $Key12$ Where to write these rules? Now, since we know…

Continue ReadingHow to teach SUSI skills calling an External API