Settings Controller UI using Static Table View

Dynamic Table Views are used at places where there may be any kind of reusability of cells. This means that there would exist cells that would have the same UI elements but would differ in the content being displayed. Initially the Settings Controller was built using UICollectionViewController which is completely dynamic but later I realized that the cells will remain static every time so there is no use of dynamic cells to display the UI hence, I switched to static table view cells. Using Static Table View is very easy. In this blog, I will explain how the implementation of the same was achieved in SUSI iOS app.

Let’s start by dragging and dropping a UITableViewController into the storyboard file.

The initial configuration of the UITableView has content as Dynamic Prototypes but we need Static cells so we choose them and make the sections count to 5 to suit our need. Also, to make the UI better, we choose the style as Grouped.

Now for each section, we have the control of statically adding UI elements so, we add all the settings with their corresponding section headers and obtain the following UI.

       

After creating this UI, we can refer any UI element independently be it in any of the cells. So here we create references to each of the UISlider and UISwitch so that we can trigger an action whenever the value of anyone of them changes to get their present state.

To create an action, simply create a function and add `@IBAction` in front so that they can be linked with the UI elements in the storyboard and then click and drag the circle next to the function to UI element it needs to be added. After successful linking, hovering over the same circle would reveal all the UI elements which trigger that function. Below is a method with the @IBAction identifier indicating it can be linked with the UI elements in the storyboard. This method is executed whenever any slider or switch value changes, which then updates the UserDefaults value as well sends an API request to update the setting for the user on the server.

@IBAction func settingChanged(sender: AnyObject?) {
        var params = [String: AnyObject]()
        var key: String = ""

        if let senderTag = sender?.tag {
            if senderTag == 0 {
                key = ControllerConstants.UserDefaultsKeys.enterToSend
            } else if senderTag == 1 {
                key = ControllerConstants.UserDefaultsKeys.micInput
            } else if senderTag == 2 {
                key = ControllerConstants.UserDefaultsKeys.hotwordEnabled
            } else if senderTag == 3 {
                key = ControllerConstants.UserDefaultsKeys.speechOutput
            } else if senderTag == 4 {
                key = ControllerConstants.UserDefaultsKeys.speechOutputAlwaysOn
            } else if senderTag == 5 {
                key = ControllerConstants.UserDefaultsKeys.speechRate
            } else if senderTag == 6 {
                key = ControllerConstants.UserDefaultsKeys.speechPitch
            }

            if let slider = sender as? UISlider {
                UserDefaults.standard.set(slider.value, forKey: key)
            } else {
                UserDefaults.standard.set(!UserDefaults.standard.bool(forKey: key), forKey: key)
            }

            params[ControllerConstants.key] = key as AnyObject
            params[ControllerConstants.value] = UserDefaults.standard.bool(forKey: key) as AnyObject

            if let delegate = UIApplication.shared.delegate as? AppDelegate, let user = delegate.currentUser {
                params[Client.UserKeys.AccessToken] = user.accessToken as AnyObject
                params[ControllerConstants.count] = 1 as AnyObject

                Client.sharedInstance.changeUserSettings(params) { (_, message) in
                    DispatchQueue.main.async {
                        self.view.makeToast(message)
                    }
                }
            }
        }
    }

References

Continue Reading

Making SUSI Alexa skill as an express app

Previously SUSI Alexa skill was deployed using AWS Lambda service (Refer to this blog). Each SUSI.AI Bot should be deployed on Google cloud using Kubernetes. To accomplish that, we need to remove the dependency of the SUSI Alexa skill from AWS Lambda service. We need to make it an express app, to be able to deploy it to Google cloud. Let’s start with on how to achieve it:

SUSI Alexa skill:

We require three files to make the skill as an express app. The main entry point for the skill would be server.js file, which will serve the incoming request using two helper files alexa.js and handlers.js.

Server.js:

This file acts as the main entry point for the incoming request. We handle two type of requests using it, that are:

  1. Launch request
  2. Intent request

Launch request is triggered when a person utters “Alexa, open susi chat” , “Alexa, start susi chat”, “Alexa, launch susi chat” etc. This request is responded with an introductory phrase about SUSI.AI. To catch this request:

if (type === "LaunchRequest") {
        var endpoint = "http://api.susi.ai/susi/chat.json?q="+"Welcome"; // ENDPOINT GOES HERE
        
        http.get(endpoint, (response1) => {
            var body = "";
            response1.on("data", (chunk) => { body += chunk; });
            response1.on("end", () => {
                var viewCount;
                viewCount = JSON.parse(body).answers[0].actions[0].expression;
                endpoint = "http://api.susi.ai/susi/chat.json?q="+"Get+started"; // ENDPOINT GOES HERE
                body = "";
                http.get(endpoint, (response2) => {
                    response2.on("data", (chunk) => { body += chunk; });
                    response2.on("end", () => {
                        viewCount += JSON.parse(body);.answers[0].actions[0].expression;
                        response.say(viewCount,false);
                    });
                });
            });
        });
    }

Intent request gets triggered, when any other phrase is uttered by the user except Launch related phrases. We check if the intent triggered has a corresponding handler to handle the request. If the handler is found in handlers.js file, we call it passing the required arguments to the handler function. Let’s see how handlers make this step possible.

Handler.js:

This file decides on what function to run when a particular type of intent is triggered. As we have just one intent for our SUSI Alexa skill i.e. callSusiApi, we have just one function in our handlers.js file. During its execution, the first step we do is extract the query value:

let query = slots.query.value;

Depending upon the query value, we run its corresponding code. For example, in case of a generic query (i.e. any query except stop, cancel and help):

var endpoint = "http://api.susi.ai/susi/chat.json?q="+query; // ENDPOINT GOES HERE

http.get(endpoint, (response1) => {
    var body = "";
    response1.on("data", (chunk) => { body += chunk; });
    response1.on("end", () => {
        var data = JSON.parse(body);
        if(data.answers[0].actions[1]){
            // handle rss and table type results
        }
        else
        {
            viewCount = data.answers[0].actions[0].expression;
        }
        response.say(viewCount,true);
    });
});

At the end of the function we respond to the user with an answer to his/her query using:

response.say(viewCount,true);

Alexa.js:

When we get a request from the user, we pass that request and response object to this file. This file helps us wrap the required request properties into an object and return that back to the server file, which was the entry point for the request. Now, we can easily extract the properties in server file and work with those:

We extract the properties like this:

let session = req.body.session,
        intent,
        slots;
session.attributes = session.attributes || {};

if (req.body.request.intent) {
    intent = req.body.request.intent.name;
    slots = req.body.request.intent.slots;
}

Then we return the object back at the end:

return {
        type: req.body.request.type,
        intent: intent,
        slots: slots,
        session: session,
        response: {
            say: (text, shouldEndSession) => say(text, shouldEndSession),
            ask: (text, shouldEndSession) => say(text, shouldEndSession)
        }
    };

Great, we have made the SUSI Alexa skill as an express app. The next step is to do some changes in the configuration tab of our skill:

  1. Instead of Amazon resource number, we fill our webhook address here: 

  2. A new property shows up that is SSL certificate. As we are using Heroku for webhook services, we select the second option as shown below: 

  3. It’s time to test the skill: 

    This repository by Salesforce helped me a lot in making the SUSI skill as an express app.

    Resources:

    1. Developing Alexa Skills Locally with Node.js by Josh Skeen from Bignerdranch.
    2. Amazon Alexa Skills: Create a Custom Skill by Simon Coope from SJCNET.
Continue Reading

Reset Password Functionality in SUSI iOS

Reset Password as the name suggests is one of the features in the SUSI iOS app which allows a user to change his/her password when they are logged in. This feature was added because a user would want to change his password sometimes to prevent unauthorized access or make his account security stronger. We can find many popular apps online such as Facebook, Gmail, which allow the user to reset their password. The way this is done is pretty simple and all we need from the user is his current and the new password he/she wants to set. In this blog post, I am going to explain step by step how this is implemented in the iOS client.

Implementation

The option to Reset Password is provided to the user under the Settings Controller. On selecting the row, the user is presented with another view which asks the user for his/her current password, new password, and another field to confirm the newly entered password.

First, the user needs to provide his current password followed by the new password. The user’s current password is required just to authenticate that the account’s owner is requesting the password change. The new password field is followed by another field called confirm password just to make sure there isn’t any typo.

Now when the field is filled, the user clicks the `Reset password` button at the bottom. What happens here is, first, the fields are validated to ensure the correct length of the passwords followed by an API request to update the same. The endpoint for the same is as below:

http://api.susi.ai/aaa/changepassword.json?changepassword=user_email&password=current _pass&newpassword=new_pass&access_token=user_access_token

This endpoint requires 3 things:

  • Current Password
  • New Password
  • User’s email
  • Access Token obtained at the time of login
func validatePassword() -> [Bool:String] {
        if let newPassword = newPasswordField.text,
            let confirmPassword = confirmPasswordField.text {
            if newPassword.characters.count > 5 {
                if newPassword == confirmPassword {
                    return [true: ""]
                } else {
                    return [false: ControllerConstants.passwordDoNotMatch]
                }
            } else {
                return [false: ControllerConstants.passwordLengthShort]
            }
        }
        return [false: Client.ResponseMessages.ServerError]
    }

Initially, we were not saving the user’s email, so we added the user’s email to the User’s object which is saved at the time of login.

if var userData = results {
userData[Client.UserKeys.EmailOfAccount] = user.email
UserDefaults.standard.set(userData, forKey: ControllerConstants.UserDefaultsKeys.user)
self.saveUserGlobally(user: currentUser)
}

At last, the API call is made which is implemented as below:

let params = [
  Client.UserKeys.AccessToken: user.accessToken,
  Client.UserKeys.EmailOfAccount: user.emailID,
  Client.UserKeys.Password: currentPasswordField.text ?? "",
  Client.UserKeys.NewPassword: newPasswordField.text ?? ""
]
Client.sharedInstance.resetPassword(params as [String : AnyObject], { (_, message) in
  DispatchQueue.main.async {
    self.view.makeToast(message)
    self.setUIActive(active: false)
  }
})

Below is the final UI.

Reference

Continue Reading

Showing sample queries in SUSI.AI Bots

We need to give the user a good start to their chat with SUSI.AI. Engaging the users with some good skills at the start of the conversation, can leave a good impression about SUSI.AI. In SUSI messenger bots, we show up with some sample queries to try, during the conversation with SUSI.AI. In this blog, SUSI_Tweetbot and SUSI_FBbot are used as examples.

These queries are shown as quick replies i.e. the user can click on any of these sample queries and get an answer from SUSI.AI.  

Facebook:

When the user clicks on the “Start chatting” button, we send a descriptive message on what can the user ask to SUSI.AI .

Code snippet used for this step is:

var queryUrl = 'http://api.susi.ai/susi/chat.json?q='+'Start+chatting';
var startMessage = '';
// Wait until done and reply
request({
        url: queryUrl,
        json: true
}, function (error, response, body) {
if (!error && response.statusCode === 200) {
        startMessage = body.answers[0].actions[0].expression;
    }
else{
    startMessage = errMessage;
    }
sendTextMessage(sender, startMessage, 0);

Just a text message is not much engaging. To further enhance the experience of the user, we show some quick reply options to the user. We have finalized some skills to show to the user:

Due to the character limit for the text shown on buttons, we try to show short queries as shown in the above picture. This way the user gets an idea about what type of queries can be asked.

Generic template, help us achieve this feature in SUSI_FBbot.

The code snippet used:

var messageT = {
               "type": "template",
               "payload": {
                "template_type": "generic",
                "elements": [{
                                    "title": 'You can try the following:',
                                    "buttons": [{                                               
                                               "type":"postback",
                                               "title":"What is FOSSASIA?",                                  
                                               "payload":"What is FOSSASIA?"            
                                            }]
                            }]
                }
            };
sendTextMessage(sender, messageT, 1);

As seen in the code above, each button has a corresponding postback text. So that whenever that button is clicked the postback text is sent to our chat automatically:

This postback text acts as a query to SUSI API which fetches the response from the server and shows it back to the user.

Twitter:

As SUSI.AI bots must be generic among all the messenger platforms available , we will inculcate the same skills available in SUSI_FBbot to SUSI_Tweetbot. The quick reply feature provided by Twitter devs help us to accomplish this task at hand.

As in SUSI_FBbot a descriptive message is shown to the users first and then some quick reply options following it.

Message_create event helps in adding quick replies:

var msg = {
               "event": {
               "type": "message_create",
               "message_create": {
                   "target": {
                       "recipient_id": senderId
                    },
                    "message_data": {
                        "text": "You can try the following:",
                        "quick_reply": {
                            "type": "options",
                            "options": [{
                                "label": "What is FOSSASIA?",
                                "metadata": "external_id_4"
                            }]
                        }
                    }
                }
           }
    };
T.post('direct_messages/events/new', msg, sent);

One thing to keep in mind while coding is to send the quick reply message after the initial descriptive message i.e. the code used to send quick replies should be written inside the function, which sends the descriptive message first and aafter that step is complete it runs the code for quick replies. If we accidentally write quick reply code outside that function, it’s highly likely to find bugs in the replies by SUSI.AI.

Resources

  1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
  2. Link Ads to Messenger, Enhanced Mobile Websites, Payments and More by Seth Rosenberg from Facebook developers blog
Continue Reading

Making SUSI.AI reach more users through messenger bots

SUSI.AI learns from the queries asked to it by the users. More are the number of queries asked, the better is the learning by SUSI.AI. More are the number of users involved with SUSI.AI, better is the amount of content available to SUSI to learn from. Now, the challenge in front of us is to indulge more users with SUSI.AI. In this blog post, SUSI Tweetbot and SUSI FBbot are used as examples to show how we increase our user base through messenger bots.

Twitter:

Twitter has a 328 million user base according to this article’s data. Integration of SUSI.AI to just Twitter makes it available to around 300 million users. Even if some percentage of these users start using SUSI.AI, increase in the user base of SUSI.AI could be exponential. Increasing the user base is advantageous as it provides with more training data for SUSI.AI to learn from.

Sharing by public tweet:

Integrating to it is just the first step towards increasing SUSI.AI’s user base. Next step is to reach the users and indulge them into chatting with SUSI.AI.

Suppose a user asked something to SUSI.AI and really liked the reply from it. He/she wants to share it with his/her followers. This can prove to be a golden opportunity for us, to increase the reach of SUSI.AI. This way we can indulge their friends to try SUSI.AI and have an amazing time with it.

It becomes clear that sharing messages is an indispensable feature and can help us a lot. Twitter doesn’t provide sharing with friends through direct messages but with a feature much better than it. We can share that message as a public tweet and cover more users including the followers of the user.

 

To show this button we use Call to action support by twitter:

"message_data": {
          "text": txt,
          "ctas": [{
                      "type": "web_url",
                      "label": "Share with your followers",
                      "url": ""
          }]
}

The url key in the above code must have a value that redirects to a U.I. that allows to publicly tweet this reply by SUSI.AI.

Using this “https://twitter.com/intent/tweet?text=” as the url value shows a new page with an empty tweet message, as the text query in the url has no value. We set the text field with a value such that we end up like this:

and after tweeting it:

Sending a direct message link with the tweet:

Twitter provides with a lot of features when sending direct messages. Shifting a user from tweets to direct messages is beneficial in a way that we can efficiently tell the user about the capabilities of SUSI.AI and show important links to him/her like of its repository, web chat client etc.

When a user tweets to the SUSI.AI page with a query, we reply with a tweet back to the user. Along with that, we provide a link to privately message SUSI.AI account if the user wants to.

This way if user ends up visiting SUSI.AI in a chat window:

To achieve this in SUSI Tweetbot, Twitter provides with a direct message url. This url – https://twitter.com/messages/compose?recipient_id= redirects us to the chat window of the account having that recipient id, passed as a query string. In our case the url turns out to be – https://twitter.com/messages/compose?recipient_id=871446601000202244 as “871446601000202244” is the recipient id of @SusiAI1 account on twitter.

If we send this url as a text in our “tweet back” to the user, Twitter beautifully shows it as a clickable button with the label as “send a private message” as shown above.

Hence we call the tweet function like this:

tweetIt('@' + from + ' ' + message + date +"\nhttps://twitter.com/messages/compose?recipient_id=871446601000202244");

Facebook:

As we all know Facebook is the giant of social networking sites. Integrating SUSI.AI to Facebook is beneficial for us.

Unlike Twitter, in Facebook we can share messages with other friends through direct messaging to them. The last topic of this blog post walks you through on adding sharing feature in SUSI FBbot:

We also take advantage from the SUSI FBbot to make SUSI.AI better. We can direct the users using SUSI messenger bots to SUSI.AI repository and show them all the required information on how to contribute to the project.

The best way to do this is by showing a “How to contribute” button when the user clicks on “Get started” in the messenger bot.

This blog post will help you showing buttons along with the replies (by SUSI.AI).

When the user clicks this button, we send two messages back to the user as shown:

This way through bots, we have somehow got the user to visit SUSI.AI repository and contribute to it or indulge him/her in the discussion, through our Gitter channel on what can be the next steps in improving SUSI.AI.

Resources:

  1. Link Ads to Messenger, Enhanced Mobile Websites, Payments and More by Seth Rosenberg from Facebook developers blog.
  2. Drive discovery of bots and other customer experiences in direct messages by  Travis Lull from Twitter blog.
Continue Reading

Skill Editor in SUSI Skill CMS

SUSI Skill CMS is a web application built on ReactJS framework for creating and editing SUSI skills easily. It follows an API centric approach where the SUSI Server acts as an API server. In this blogpost we will see how to add a component which can be used to create a new skill SUSI Skill CMS.

For creating any skill in SUSI we need four parameters i.e model, group, language, skill name. So we need to ask these 4 parameters from the user. For input purposes we have a common card component which has dropdowns for selecting models, groups and languages, and a text field for skill name input.

<SelectField
    floatingLabelText="Model"
    value={this.state.modelValue}
    onChange={this.handleModelChange}
>
    {models}
</SelectField>
<SelectField
    floatingLabelText="Group"
    value={this.state.groupValue}
    onChange={this.handleGroupChange}
>
    {groups}
</SelectField>
<SelectField
    floatingLabelText="Language"
    value={this.state.languageValue}
    onChange={this.handleLanguageChange}
>
    {languages}
</SelectField>
<TextField
    floatingLabelText="Enter Skill name"
    floatingLabelFixed={true}
    hintText="My SUSI Skill"
    onChange={this.handleExpertChange}
/>
<RaisedButton label="Save" backgroundColor="#4285f4" labelColor="#fff" style={{marginLeft:10}} onTouchTap={this.saveClick} />

This is the card component where we get the user input. We have API endpoints on SUSI Server for getting the list of models, groups and languages. Using those APIs we inflate the dropdowns.
Then the user needs to edit the skill. For editing of skills we have used Ace Editor. Ace is an code
editor written in pure javascript. It matches the features native editors like Sublime and TextMate.

To use Ace we need to install the component.

npm install react-ace --save                        

This command will install the dependency and update the package.json file in our project with this dependency.

To use this editor we need to import AceEditor and place it in the render function of our react class.

<AceEditor
    mode=" markup"
    theme={this.state.editorTheme}
    width="100%"
    fontSize={this.state.fontSizeCode}
    height= "400px"
    value={this.state.code}
    name="skill_code_editor"
    onChange={this.onChange}
    editorProps={{$blockScrolling: true}}
/>

Now we have a page that looks something like this

Now we need to handle the click event when a user clicks on the save button.

First we check if the user is logged in or not. For this we check if we have the required cookies and the access token of the user.

 if(!cookies.get('loggedIn')) {
            notification.open({
                message: 'Not logged In',
                description: 'Please login and then try to create/edit a skill',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
            return 0;
        }

If the user is not logged in then we show him a error notification and asks him to login.

Then we check if he has filled all the required fields like name of the skill etc. and after that we call an API Endpoint on SUSI Server that will finally store the skill in the skill_data_repo.

let url= “http://api.susi.ai/cms/modifySkill.json”
$.ajax({
    url:url,
    dataType: 'jsonp',
    jsonp: 'callback',
    crossDomain: true,
    success: function (data) {
        console.log(data);
        if(data.accepted===true){
            notification.open({
                message: 'Accepted',
                description: 'Your Skill has been uploaded to the server',
                icon: <Icon type="check-circle" style={{ color: '#00C853' }} />,
            });
           }
    }
});

In the success function of ajax call we check if accepted parameter is true from the server or not. If accepted is true then we show user a notification with a message that “Your Skill has been uploaded to the server”.

To see this component running please visit http://skills.susi.ai/skillEditor.

Resources

Material-UI: http://www.material-ui.com/

Ace Editor: https://github.com/securingsincity/react-ace

Ajax: http://api.jquery.com/jquery.ajax/

Universal Cookies: https://www.npmjs.com/package/universal-cookie

Continue Reading

Uploading Images to SUSI Server

SUSI Skill CMS is a web app to create and modify SUSI Skills. It needs API Endpoints to function and SUSI Server makes it possible. In this blogpost, we will see how to add a servlet to SUSI Server to upload images and files.

The CreateSkillService.java file is the servlet which handles the process of creating new Skills. It requires different user roles to be implemented and hence it extends the AbstractAPIHandler.

Image upload is only possible via a POST request so we will first override the doPost method in this servlet.

  @Override
  protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
  resp.setHeader("Access-Control-Allow-Origin", "*"); // enable CORS

resp.setHeader enables the CORS for the servlet. This is required as POST requests must have CORS enables from the server. This is an important security feature that is provided by the browser.

        Part file = req.getPart("image");
        if (file == null) {
            json.put("accepted", false);
            json.put("message", "Image not given");
        }

Image upload to servers is usually a Multipart Request. So we get the part which is named as “image” in the form data.

When we receive the image file, then we check if the image with the same name exists on the server or not.

Path p = Paths.get(language + File.separator + “images/” + image_name);

        if (image_name == null || Files.exists(p)) {
                json.put("accepted", false);
                json.put("message", "The Image name not given or Image with same name is already present ");
            }

If the same file is present on the server then we return an error to the user requesting to give a unique filename to upload.

Image image = ImageIO.read(filecontent);
BufferedImage bi = this.createResizedCopy(image, 512, 512, true);
if(!Files.exists(Paths.get(language.getPath() + File.separator + "images"))){
   new File(language.getPath() + File.separator + "images").mkdirs();
           }
ImageIO.write(bi, "jpg", new File(language.getPath() + File.separator + "images/" + image_name));

Then we read the content for the image in an Image object. Then we check if images directory exists or not. If there is no image directory in the skill path specified then create a folder named “images”.

We usually prefer square images at the Skill CMS. So we create a resized copy of the image of 512×512 dimensions and save that copy to the directory we created above.

BufferedImage createResizedCopy(Image originalImage, int scaledWidth, int scaledHeight, boolean preserveAlpha) {
        int imageType = preserveAlpha ? BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB;
        BufferedImage scaledBI = new BufferedImage(scaledWidth, scaledHeight, imageType);
        Graphics2D g = scaledBI.createGraphics();
        if (preserveAlpha) {
            g.setComposite(AlphaComposite.Src);
        }
        g.drawImage(originalImage, 0, 0, scaledWidth, scaledHeight, null);
        g.dispose();
        return scaledBI;
    }

The function above is used to create a  resized copy of the image of specified dimensions. If the image was a PNG then it also preserves the transparency of the image while creating a copy.

Since the SUSI server follows an API centric approach, all servlets respond in JSON.

       resp.setContentType("application/json");
       resp.setCharacterEncoding("UTF-8");
       resp.getWriter().write(json.toString());’

At last, we set the character encoding and the character set of the output. This helps the clients to parse the data easily.

To see this endpoint in live send a POST request at http://api.susi.ai/cms/createSkill.json.

Resources

Apache Docs: https://commons.apache.org/proper/commons-fileupload/using.html

Multipart POST Request Tutorial: http://www.codejava.net/java-se/networking/upload-files-by-sending-multipart-request-programmatically

Java File Upload tutorial: https://ursaj.com/upload-files-in-java-with-servlet-api

Jetty Project: https://github.com/jetty-project/

Continue Reading
Close Menu