node.js

Read more about the article Showing RSS and Table Type Replies in SUSI Messenger Bots

Showing RSS and Table Type Replies in SUSI Messenger Bots

All the messengers have a “plain text” reply support. To show web search (RSS) or table type replies, either: We need a “list type” (as in Facebook messenger) or “table type” reply support built in the messenger itself.                                                               or We need to convert the RSS or table type reply by SUSI API to plain text, so that we can send it, due to the “plain text” reply support available in almost every messenger. The second point is the most favorable approach, as that way, replying with RSS or table type results is dependent only on the text support feature in the messenger. This way RSS or table type replies can be shown in messengers like REST API Gitter (which do not provide any other reply type support except text). In SUSI web app, the UI of the web search and table type results: As the SUSI web app is made in React js, it provided the app with necessary features to show the results this way. The messengers may not be having these required features. So the task is we need to convert the RSS or table type replies by SUSI API to plain text to send it to the messenger. Let’s work it out. Converting RSS results to text: First get familiar with the SUSI API reply to query “why” by visiting this url - http://api.susi.ai/susi/chat.json?q=why. The JSON object returned will be constituting an array of objects as the value of the data key like: "data": [ { "title": "Why is Oracle male?", "description": "Why is Oracle male?. http dba oracle com why is male htm. Oracle Why is masculine?. ", "link": "http://dba-oracle.com/t_why_is_oracle_male.htm" } ] If we notice carefully each object has 3 main keys namely “title”, “description” and “link”. So extracting these 3 properties from each object and binding them together into 1 string is the task we need to do. So we traverse each object (i.e. rss result) in the data array and we keep on appending the values of “title”, “description” and “link” key values to the ans variable. At the end we send this resultant string to the messenger bot as a reply.   Suppose we have the returned JSON object, in the data variable. // storing the number of rss results var metaCnt = data.answers[0].metadata.count; for(var i=0;i<metaCnt;i++){ ans += ('Title : '); ans += data.answers[0].data[i].title+', '; ans += ('Link : '); ans += data.answers[0].data[i].link+', '; ans += '\n\n'; } // send the message in ans variable to the messenger Converting table type replies to text: First get familiar with the SUSI API reply to query “why” by visiting this url - http://api.susi.ai/susi/chat.json?q=universities+in+australia. The JSON object returned will be constituting an array of objects representing universities as the value of the data key in this form: { "alpha_two_code": "AU", "name": "Australian Correspondence Schools", "country":…

Continue ReadingShowing RSS and Table Type Replies in SUSI Messenger Bots
Read more about the article Using Templates in SUSI FBbot

Using Templates in SUSI FBbot

The SUSI AI Fbbot previously showed rss and table type replies as plain text to the user. To enhance the user experience, Facebook provides with templates which can be used in it’s messenger. Using those, we show rss and table type replies wrapped up in a better U.I. creating a better user experience. The 4 basic template structures that can be used for this task are: Button template Generic template List template Receipt template List template is used in SUSI A.I. Fbbot because rss reply and table type reply both are lists of data. The basic syntax for list template with reference to Fb official docs is: "message": { "attachment": { "type": "template", "payload": { "template_type": "list", "top_element_style": "compact", "elements": [ { "title": "Classic White T-Shirt", "subtitle": "100% Cotton, 200% Comfortable", "default_action": { "type": "web_url", "url": "https://peterssendreceiveapp.ngrok.io/view?item=100" }, "buttons": [ { "title": "Buy", "type": "web_url", "url": "https://peterssendreceiveapp.ngrok.io/shop?item=100" } ] } ] } } } This code shows a reply to the user like this: If we want to show a “View more” button at the end of the list, we can add a “buttons” key and an array as its value which will have information regarding all the buttons we want to show. The code below will show a “View more” button at the end of the list: "elements": [ { // all the elements, like shirt in the above case } ], "buttons": [ { "title": "View More", "type": "postback", "payload": "payload" } ] Let’s learn how to incorporate these features to rss results in SUSI Fbbot: When sending a reply using list template, “elements” key must have an array type value which will be constituted of list items. Therefore, we need to push all the rss results into that array and set it as the value of the “elements” key. Let’s develop the code part: Fetch the JSON object from SUSI API url with query as “why” i.e. http://api.susi.ai/susi/chat.json?q=why. Let body be a variable which stores the returned JSON object. The below code fills up an array (namely elementsVal) with all the rss results: var elementsVal = []; var metaCnt = body.answers[0].metadata.count; for(var i=0;i<((metaCnt>4)?4:metaCnt);i++){ elementsVal.push( { "title": body.answers[0].data[i].title, "subtitle": body.answers[0].data[i].link } ); } Setting the elementsVal array as the value of the "elements" key: var message = { "type": "template", "payload": { "template_type": "list", "top_element_style": "compact", "elements": elementsVal } }; Sending this message as a reply to the user: sendTextMessage(sender, message); Same procedure can be used to render table type replies in the bot using the list template. Generic template provided by Facebook can also used to render the web and table type replies. This template helps in showing the results in square boxes, which can be changed using left or right arrows. Resources: By Mikhail Larionov from Facebook blogs - List templates and check box plugin By Slobodan Stojanović from smashing magazine - Develop a chat bot with node js.

Continue ReadingUsing Templates in SUSI FBbot
Read more about the article Integration of SUSI AI to Facebook

Integration of SUSI AI to Facebook

It's easy to create your own SUSI AI Facebook messenger bot. You can read the official documentation by Facebook to get started. Messenger bots use a web server to send and receive messages. You also need to have the bot be authenticated to speak with the web server and be approved by Facebook before getting public. If any problems faced, visit the susi_fbbot repository which hosts the code for SUSI Facebook Messenger bot. We will be using Node js technology to develop the FB bot. First, let’s see on how to request an answer from the SUSI API. To call Susi API and fetch an answer from it for a query (‘hi’ in this case). Let's first visit http://api.asksusi.com/susi/chat.json?q=hi from the browser. We will get a JSON object as follows: The answer can be found as the value of the key named expression. In this case, it is “Hallo!”. To fetch the answer through coding, we can use this code snippet in Node js: // including request module var request = require(‘request’); // setting options to make a successful call to Susi API. var options = { method: 'GET', url: 'http://api.asksusi.com/susi/chat.json', qs: { timezoneOffset: '-330', q: 'hi' } }; // A request to the Susi bot request(options, function (error, response, body) { if (error) throw new Error(error); // answer fetched from susi ans = (JSON.parse(body)).answers[0].actions[0].expression; } The properties required for the call are set up through a JSON object (i.e. options). Pass the options object to our request function as its 1st parameter. The response from the API will be stored in ‘body’ variable. We need to parse this body, to be able to access the properties of that body object. Hence, fetching the answer from Susi API. Now let’s dive into the code of receiving and messaging back to the user on Facebook: A generic function to send a message to the user. // the first argument is the sender id and the second is the text to send. function sendTextMessage(sender, text) { var messageData = { text:text }; // making a post request to facebook graph API to send message. request({ url: 'https://graph.facebook.com/v2.6/me/messages', qs: {access_token:token}, method: 'POST', json: { recipient: {id:sender}, message: messageData, } }, function(error, response, body) { if (error) { console.log('Error sending messages: ', error); } else if (response.body.error) { console.log('Error: ', response.body.error); } }); } According to step 9, in the below instructions we need to include this code snippet too: // for facebook verification app.get('/webhook/', function (req, res) { if (req.query['hub.verify_token'] === 'my_voice_is_my_password_verify_me') { res.send(req.query['hub.challenge']); } res.send('Error, wrong token'); }); When user messages to our bot, we need to extract the text of the message from the request body. Then we need to extract the reply from the SUSI API and send it back to the user. // when a message from a user is received. app.post('/webhook/', function (req, res) { var messaging_events = req.body.entry[0].messaging for (var i = 0; i < messaging_events.length; i++) { // fetching the current event var event = req.body.entry[0].messaging[i];…

Continue ReadingIntegration of SUSI AI to Facebook
Read more about the article Integration of SUSI AI in Twitter

Integration of SUSI AI in Twitter

We will be making a Susi messenger bot on Twitter. The messenger bot will tweet back to your tweets and reply instantly when you chat with it. Feel free to tweet to the already made SUSI AI account (mentioning @SusiAI1 in it). Follow it, to have a personal chat. Make a new account, which you want to use as the bot account. You can make one from sign up option from https://www.twitter.com. Prerequisites To create your account on -: 1. Twitter 2. Github 3. Heroku 4. Node js Setup your own Messenger Bot 1. Make a new app here, to know the access token and other properties for our application. These properties will help us communicate with Twitter. Click "modify the app permissions" link, as shown here: Select the Read, Write and Access direct messages option: Don't forget to click the update settings button at the bottom. Click the Generate My Access Token and Token Secret button. 3. Create a new heroku app here. This app will accept the requests from Twitter and Susi api. 4. Create a config variable by switching to settings page of your app.      The name of your first config variable should be HEROKU_URL and its value is the url address of the heroku app created by you.   The other config variables that need to be created will be these:   The corresponding names of these variables in the same order are:   i) Access token   ii) Access token secret   iii) Consumer key   iv) Consumer secret    We need to visit our app from here, the keys and access tokens tab will help us with the values of these variables. Let’s start with the code part of the integration of SUSI AI to Twitter. We will be using Node js to achieve this integration. First we need to require some packages: Now using the Twit module, we need to authenticate our requests, by using our environment variables as set up in step 4: Now let’s make a user stream: var stream = T.stream('user'); We will be using the capabilities of this stream, to catch events of getting tweeted or receiving a direct message by using: stream.on('tweet', functionToBeCalledWhenTweeted); stream.on('follow', functionToBeCalledWhenFollowed); stream.on('direct_message', functionToBeCalledWhenDirectMessaged); So, when a person tweets to our account like this: We can catch it with ‘tweet’ event and execute a set of instructions: stream.on('tweet', tweetEvent); function tweetEvent(eventMsg) { var replyto = eventMsg.in_reply_to_screen_name; // to store the message tweeted excluding '@SusiAI1' substring var text = eventMsg.text.substring(9); // to store the name of the tweeter var from = eventMsg.user.screen_name; if (replyto === 'SusiAI1') { var queryUrl = 'http://api.asksusi.com/susi/chat.json?q=' + encodeURI(text); var message = ''; request({ url: queryUrl, json: true }, function (err, response, data) { if (!err && response.statusCode === 200) { // fetching the answer from the data object returned message = data.answers[0].actions[0].expression + data; } else { message = 'Oops, Looks like Susi is taking a break'; console.log(err); } console.log(message); // If the message length is more than tweet limit if(message.length > 140){ tweetIt('@'…

Continue ReadingIntegration of SUSI AI in Twitter
Read more about the article Integration of Susi AI to Gitter

Integration of Susi AI to Gitter

This blog post discusses the development of Susi Messenger bot on Gitter. It replies instantly to the messages sent to it, using the Susi API. The Streaming API notifies us when a user messages to the SUSI chat room. The REST API helps to message back with a reply from SUSI API, to the SUSI chat room. Feel free to message to the already made SUSI AI account on Gitter and have a chat with it. Prerequisites Basic knowledge about calling API’s and fetching data or posting data to the API. Node.js language. Github Heroku Figure - Architecture for running SUSI AI on different messaging services. This blog post will walk you through each of the steps required to integrate SUSI AI to Gitter: Setup SUSI AI Bot on Gitter Create a Github or twitter account with a username having 'Susi' as its substring because this is the name that will be shown with the reply string we will get from Susi AI. Now you need to sign in to Gitter with a twitter or Github account from here. Create your community by visiting this page. After writing your community name press next, invite the people you want to be in this room and press next. You will be redirected to your communities lobby. This lobby is the chat room to which we will deploy our SUSI AI. Now visit the Gitter developer page, press sign in on the top right. You will be redirected to your apps page. Copy the personal access token written there as shown in this image: (The area colored black will have your access token). 5. On a new tab, in your browser visit   https://api.gitter.im/v1/rooms?access_token=YOUR_ACCESS_TOKEN, with YOUR_ACCESS_TOKEN replaced by the token we just copied. A JSON object will be shown on our browser screen. You will see the value of 'name' key as YOUR_COMMUNITY_NAME/Lobby. Copy the id of this chat room, as we will need it later. You can refer to the image below, you will have your chat room id in the area colored black. Create a new heroku app here. This app will accept the requests from Gitter and Susi api. Set the config variables for this heroku app in the setting tab of your account. Set ROOM_ID to the id of the chat room and TOKEN to the personal access token, we copied in steps 4 and 5. These were the formalities to be done to have our chat bot account on Gitter. Let’s jump to the code part of how this integration will be done: To use the two config variables set in Heroku, we need these two lines in our Node js code: var roomId = process.env.ROOM_ID; var token = process.env.TOKEN; We need to set up an options variable with our access token and room id in it: // Setting the options variable to use it in the https request block var options = { hostname: 'stream.gitter.im', port: 443, path: '/v1/rooms/' + roomId + '/chatMessages', method: 'GET', headers: {'Authorization': 'Bearer '…

Continue ReadingIntegration of Susi AI to Gitter
Read more about the article Displaying selective logs in Yaydoc with downloadable detailed logs

Displaying selective logs in Yaydoc with downloadable detailed logs

Yaydoc, our automatic documentation generation and deployment project, at the crux of it consists of bash scripts. These bash scripts perform various actions, including documentation generation, deployment to Github and deployment to Heroku. Since the inception of the User Interface of the Web UI, we have been spitting out the output of the bash script directly to the console output block without any filter. We understand that the output contains a lot of jargon that provides no essential information to the user. Hence, to improve the user experience of our platform, we decided to separate the Detailed logs and show only basic logs outlining the flow of the processes. To implement this, we append all the logs to a unique file, sending only basic logs to stdout and stderr. Since we attend to store logs and display them in the console block of Web UI simultaneously we use the tee command, piping it with echo commands. function print_log { if [ -n “$LOGFILE” ]; then echo -e $1 | tee -a ${LOGFILE} else echo -e $1 fi } ${LOGFILE}  is a unique file that has the same name as the preview directory and the compressed repository. After storing the logs in the file, the contents of the file are outputted using the cat command and is then shown to the user in a modal which is after the Detailed Logs button is clicked. $(“#btnLogs”).click(function () { socket.emit(‘retrieve-detailed-logs’, data); .... }); socket.on(‘retrieve-detailed-logs’, function (data) { var process = spawn(‘cat’, [‘temp/’+data.email+’/’+data.uniqueId+’.txt’]); process.stdout.setEncoding(‘utf-8’); process.stdout.on(‘data’, function (data)) { socket.emit(‘file-content’, data); } }); To keep the indentation of the logs, we display the content inside the HTML pre tags. Along with displaying the detailed logs, we also let our two additional functionalities. These are ‘Copy to Clipboard’ and ‘Download as text file’. The ‘copy to clipboard’ functionality is achieved using the clipboard.js jQuery library while the `Download` functionality is achieved using res.download(file) function of ExpressJS response. socket.on(‘file-content’, function (data) { new Clipboard(‘#copy-button’); $(‘#detailed-logs’).html(data); $(‘#detailed-logs-modal’).modal(‘show’); }); Resources: https://clipboardjs.com - A modern approach to copy text to clipboard https://socket.io/ - The fastest and most reliable real-time engine

Continue ReadingDisplaying selective logs in Yaydoc with downloadable detailed logs
Read more about the article Create Scraper in Javascript for Loklak Scraper JS

Create Scraper in Javascript for Loklak Scraper JS

Loklak Scraper JS is the latest repository in Loklak project. It is one of the interesting projects because of expected benefits of Javascript in web scraping. It has a Node Javascript engine and is used in Loklak Wok project as bundled package. It has potential to be used in different repositories and enhance them. Scraping in Python is easy (at least for Pythonistas) as one needs to just import Request library and BeautifulSoup library (lxml as better option), write some lines of code using Request library to get webpage and some lines of bs4 to walk through html and scrape data. This sums up to about less than a hundred lines of coding, where as Javascript coding isn't easily readable (at least to me) as compared to Python. But it has an advantage, it can easily deal with Javascript in the pages we are scraping. This is one of the motive, Loklak Scraper JS repository was created and we contributed and worked on it. I recently coded a Javascript scraper in loklak_scraper_js repository. While coding, I found it’s libraries similar to the libraries, I use to code in Python. Therefore, this blog is for Pythonistas how they can start scraping in Javascript as they finish reading and also contribute to Loklak Scraper JS. First, replace Python interpreter, Request and Beautifulsoup library with Node JS interpreter, Request and Cheerio JS library. 1) Node JS Interpreter: Node JS Interpreter is used to interpret Javascript files. This is different from Python as it deals with the project instead of a module in case of Python. The most compatible Node for most of the libraries is 6.0.0 , where as latest version available(as I checked) is 8.0.0 TIP: use `--save` with npm like here while installing a library. 2) Request Library :- This is used to load webpage to be processed. Similar to one in Python. Request-promise library, a wrapper around Request with implementation of Bluebird library, improves readability and makes code cleaner (how?).   3) Cheerio Library:- A Pythonista (a rookie one) can call it twin of BeautifulSoup Library. But this is faster and is Javascript. It's selector implementation is nearly identical to jQuery's. Let us code a basic Javascript scraper. I will take TimeAndDate scraper from loklak_scraper_js as example here. It inputs place and outputs its local time. Step#1: fetching HTML from webpage with the help of Request library. We input url to Request function to fetch the webpage and is saved to `html` variable. This scrapeTimeAndDate() function scrapes data from html url = "http://www.timeanddate.com/worldclock/results.html?query=London"; request(url, function(error, response, body) { if(error) { console.log("Error: " + error); process.exit(-1); } html = body; scrapeTimeAndDate() });   Step#2: To scrape important data from html using Cheerio JS list of date and time of locations is embedded in table tag, So we will iterate through <td> and extract text. a) Load html to Cheerio as we do in beautifulsoup In Python soup = BeautifulSoup(html,'html5lib')   In Cheerio JS $ = cheerio.load(html);   b) This line finds…

Continue ReadingCreate Scraper in Javascript for Loklak Scraper JS

Doing asynchronous tasks serially using ‘async’ in node.js

In the open-event-webapp generator we need to perform a lot of asynchronous tasks in the background like - Downloading images and audio assets Downloading the jsons from the endpoints Generating the html from handelbar templates and so on . . Sometimes tasks depend on previous tasks, and in such cases we need to perform them serially. Also there are tasks like image downloads, that would be better if done parallelly. To achieve both these purposes, there is an awesome node.js library called async that helps achieve this. To perform asynchronous tasks serially (one task, then another task), we can use something like this -   async.series([ (done) => { someAsyncFunction(function () { done () }) }, //(done) => {..}, (done) => {..} more tasks here (done) => { someAsyncFunction(function () { done () }) }); } ]); Basically async takes an array of functions. Each function contains a callback that you need to call when the internal task is finished. The 2nd task starts, only after the done() callback of first task is executed. An example of it's usage can be seen in the open-event-webapp project here

Continue ReadingDoing asynchronous tasks serially using ‘async’ in node.js