How to Implement Feedback System in SUSI iOS

The SUSI iOS app provides responses for various queries but the response is always not accurate. To improve the response, we make use of the feedback system, which is the first step towards implementing Machine Learning on the SUSI Server. The way this works is that for every query, we present the user with an option to upvote or downvote the response and based on that a positive or negative feedback is saved on the server. In this blog, I will explain how this feedback system was implemented in the SUSI iOS app.

Steps to implement:

We start by adding the UI which is two buttons, one with a thumbs up and the other with a thumbs down image.

textBubbleView.addSubview(thumbUpIcon)
textBubbleView.addSubview(thumbDownIcon)
textBubbleView.addConstraintsWithFormat(format: "H:[v0]-4-[v1(14)]-2-[v2(14)]-8-|", views: timeLabel, thumbUpIcon, thumbDownIcon)
textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbUpIcon)
textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbDownIcon)
thumbUpIcon.isUserInteractionEnabled = true
thumbDownIcon.isUserInteractionEnabled = true

Here, we add the subviews and assign constraints so that these buttons align to the bottom right next to each other. Also, we enable the user interaction for these buttons.

We know that the user can rate the response by pressing either of the buttons added above. To do that we make an API call to the endpoint below:

BASE_URL+'/cms/rateSkill.json?'+'model='+model+'&group='+group+'&skill='+skill+’&language’+language+’&rating=’+rating

Here, the BASE_URL is the url of the server, the other three params model, group, language and skill are retrieved by parsing the skill location parameter we get with the response. The rating is positive or negative based on which button was pressed by the user. The skill param in the response looks like this:

skills:
[
"/susi_skill_data/models/general/entertainment/en/quotes.txt"
]

Let’s write the method that makes the API call and responds to the UI that it was successful.

if let accepted = response[ControllerConstants.accepted] as? Bool {
  if accepted {
    completion(true, nil)
    return
  }
  completion(false, ResponseMessages.ServerError)
  return
}

Here after receiving a response from the server, we check if the `accepted` variable is true or not. Based on that, we pass `true` or `false` to the completion handler. Below the response we actually receive by making the request.

{
session: {
identity: {
type: "host",
name: "23.105.140.146",
anonymous: true
}
},
accepted: true,
message: "Skill ratings updated"
}

Finally, let’s update the UI after the request has been successful.

if sender == thumbUpIcon {
thumbDownIcon.tintColor = UIColor(white: 0.1, alpha: 0.7)
thumbUpIcon.isUserInteractionEnabled = false
thumbDownIcon.isUserInteractionEnabled = true
feedback = "positive"
} else {
thumbUpIcon.tintColor = UIColor(white: 0.1, alpha: 0.7)
thumbDownIcon.isUserInteractionEnabled = false
thumbUpIcon.isUserInteractionEnabled = true
feedback = "negative"
}
sender.tintColor = UIColor.hexStringToUIColor(hex: "#2196F3")

Here, we check the sender (the thumbs up or down button) and based on that pass the rating (positive or negative) and update the color of the button.

Below is the app in action with the feedback system.

Resources:

Continue ReadingHow to Implement Feedback System in SUSI iOS

Advanced functionality in SUSI Tweetbot

SUSI AI is integrated to Twitter (blog). During the initial phase, SUSI Tweetbot had basic UI and functionalities like just “plain text” replies. Twitter provides with many more features like quick replies i.e. presenting to the user with some choices to choose from or visiting SUSI server repository by just clicking buttons during the chat etc.

All these features are provided to enhance the user experience with our chatbot on Twitter.

This blog post walks you through on adding these functionalities to the SUSI Tweetbot:

  1. Quick replies
  2. Buttons

    Quick replies:

    This feature provides options to the user to choose from.

    The user doesn’t need to type the next query but rather select a quick reply from the options available. This speeds up the process and makes it easy for the user. Also, it helps developers know all the possible queries which can come next, from the user. Hence, it helps in efficient coding on how to handle those queries.In SUSI Tweetbot this feature is used to welcome a new user to the SUSI A.I.’s chat window, as shown in the image above. The user can select any option among “Get started” and “Start chatting”.The “Get started” option is basically for introduction of SUSI A.I. to the user. While, “Start chatting” when clicked shows the user of what all queries the user can try.Let’s come to the code part on how to show these options and what events happen when a user selects one of the options.

    To show the Welcome message, we call SUSI API with the query as string “Welcome” and store the reply in message variable. The code snippet used:

var queryUrl = 'http://api.susi.ai/susi/chat.json?q=Welcome';
var message = '';
request({
    url: queryUrl,
    json: true
}, function (err, response, data) {
    if (!err && response.statusCode === 200) {
        message = data.answers[0].actions[0].expression;
    } 
    else {
        // handle error
    }
});

To show options with the message:

var msg =  {
        "welcome_message" : {
                    "message_data": {
                        "text": message,
                        "quick_reply": {
                              "type": "options",
                              "options": [
                                {
                                  "label": "Get started",
                                  "metadata": "external_id_1"
                                },
                                {
                                  "label": "Start chatting",
                                  "metadata": "external_id_2"
                                }
                              ]
                            }
                    }
                      }
       };
T.post('direct_messages/welcome_messages/new', msg, sent);

The line T.post() makes a POST request to the Twitter API, to register the welcome message with Twitter for our chatbot. The return value from this request includes a welcome message id in it corresponding to this welcome message.

We set up a welcome message rule for this welcome message using it’s id. By setting up the rule is to set this welcome message as the default welcome message shown to new users. Twitter also provides with custom welcome messages, information about which can be found in their official docs.

The welcome message rule is set up by sending the welcome message id as a key in the request body:

var welcomeId = data.welcome_message.id;
var welcomeRule = {
            "welcome_message_rule": {
                "welcome_message_id": welcomeId
            }
};
T.post('direct_messages/welcome_messages/rules/new', welcomeRule, sent);

Now, we are all set to show the new users with a welcome message.

Buttons:

Let’s go a bit further. If the user clicks on the option “Get started”, we want to show a basic introduction of SUSI A.I. to the user. Along with that we provide some buttons to visit the SUSI A.I. repository or experience chatting with SUSI A.I. on the web client.

The procedure followed to show buttons is almost the same as followed in case of options.This doc by Twitter proves to be helpful to get familiar with buttons.

As soon as a person clicks on “Get started” option, Twitter sends a message to our bot with the query as “Get started”.

For the message part, we call SUSI API with the query as string “Get started” and store the reply in a message variable. The code snippet used:

var queryUrl = 'http://api.susi.ai/susi/chat.json?q=Get+started';
var message = '';
request({
    url: queryUrl,
    json: true
}, function (err, response, data) {
    if (!err && response.statusCode === 200) {
        message = data.answers[0].actions[0].expression;
    } 
    else {
        // handle error
    }
});

Both the buttons to be shown with the message should have a corresponding url. So that after clicking the button a person is redirected to that url in a new browser tab.

To show buttons a simple message won’t help, we need to create an event. This event constitutes of our message and buttons. The buttons are referred to as call-to-action i.e. CTAs by Twitter dev’s. The maximum number of buttons in an event can not be more than three in number.

The code used to make an event in our case:

var msg = {
        "event": {
                "type": "message_create",
                "message_create": {
                              "target": {
                                "recipient_id": sender
                              },
                              "message_data": {
                                "text": message,
                                "ctas": [
                                  {
                                    "type": "web_url",
                                    "label": "View Repository",
                                    "url": "https://www.github.com/fossasia/susi_server"
                                  },
                                  {
                                    "type": "web_url",
                                    "label": "Chat on the web client",
                                    "url": "http://chat.susi.ai"
                                  }
                                ]
                              }
                            }
            }
        };

T.post('direct_messages/events/new', msg, sent);

The line T.post() makes a POST request to the Twitter API, to send this event to the concerned recipient guiding them on how to get started with the bot.

Resources:

  1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
  2. Drive discovery of bots and other customer experiences in direct messages by Travis Lull from Twitter blog.
Continue ReadingAdvanced functionality in SUSI Tweetbot

Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

In this blog post, I will be explaining how to encode and decode images and save them in UserDefaults so that the image persists even if it is removed from the Photos app. It happens a number of times that images are removed from the gallery by the users which results in the app loosing the image. So, to avoid this, we save the image by encoding it in a data object and save it inside UserDefaults. In SUSI iOS app we simply select an image from the image picker, encode it and save it in UserDefaults. To set the image, we simply fetch the image data from the UserDefaults and decode it to an image.

There are two ways we can do the encoding and decoding process:

  • Using Data object
  • Using Base64 string

For the scope of this tutorial, we will use the Data object.

Implementation Steps

  1. To use the image picker, we need to add permissions to `Info.plist` file.
<key>NSLocationWhenInUseUsageDescription</key>
<string>Susi is requesting to get your current location</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>Susi needs to request your gallery access to select wallpaper</string>
  1. Select image from gallery

First, we present an alert which gives an option to select the image from the gallery.

// Show wallpaper options to set wallpaper or clear wallpaper
func showWallpaperOptions() {
  let imageDialog = UIAlertController(title: ControllerConstants.wallpaperOptionsTitle, message: nil, preferredStyle: UIAlertControllerStyle.alert)
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsPickAction, style: .default, handler: { (_: UIAlertAction!) in
  imageDialog.dismiss(animated: true, completion: nil)
  self.showImagePicker()
  }))
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsNoWallpaperAction, style: .default, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
    self.removeWallpaperFromUserDefaults()
  }))
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
  }))
  self.present(imageDialog, animated: true, completion: nil)
}

Here, we create and UIAlertController with three options to select, one which presents the image picker controller, the second one removes the background wallpaper and the third dismisses the alert.

  1. Set the image as background view
// Callback when image is selected from gallery
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
  dismiss(animated: true, completion: nil)
  let chosenImage = info[UIImagePickerControllerOriginalImage] as? UIImage
  if let image = chosenImage {
    setBackgroundImage(image: image)
  }
}

We use the `didFinishPickingMediaWithInfo` delegate method to set the image as background. First we get the image using the the `info` dictionary using the `UIImagePickerControllerOriginalImage` key.

  1. Save the image in UserDefaults (encoding)
// Save image selected by user to user defaults
func saveWallpaperInUserDefaults(image: UIImage!) {
  let imageData = UIImageJPEGRepresentation(image!, 1.0)
  let defaults = UserDefaults.standard
  defaults.set(imageData, forKey: userDefaultsWallpaperKey)
}

We first convert the image to a data object using the `UIImageJPEGRepresentation` method followed by saving the data object in UserDefaults with the key `wallpaper`.

  1. Decode the data object back to UIImage 

Now whenever we need to decode the image, we simply get the data object from the UserDefaults and use it to display the image.

// Check if user defaults have an image data saved else return nil/Any
func getWallpaperFromUserDefaults() -> Any? {
  let defaults = UserDefaults.standard
  return defaults.object(forKey: userDefaultsWallpaperKey)
}

Below is the output when an image is selected and displayed as a background.

Resources:

Continue ReadingEncoding and Decoding Images as Data in UserDefaults in SUSI iOS

Using The Dark and Light Theme in SUSI iOS

SUSI being an AI for interactive chat bots, provides answers to the users in the intelligent way. So, to make the SUSI iOS app more user friendly, the option of switching between themes was introduced. This also enables the user switch between themes based on the environment around. Any user can switch between the light and dark themes easily from the settings.

We start by declaring an enum called `theme` which contains two strings namely, dark and light.

enum theme: String {
    case light
    case dark
}

We can update the color scheme based on the theme selected very easily by checking the currently active theme and based on that check, we update the color scheme. To check the currently active theme, we define a variable in the `AppDelegate` which holds the value.

var activeTheme: String?

Below is the way the color scheme of the LoginViewController is set.

var activeTheme: String?func setupTheme() {
  let image = UIImage(named: ControllerConstants.susi)?.withRenderingMode(.alwaysTemplate)
  susiLogo.image = image
  susiLogo.tintColor = .white
  UIApplication.shared.statusBarStyle = .lightContent
  let activeTheme = AppDelegate().activeTheme
  if activeTheme == theme.light.rawValue {
    view.backgroundColor = UIColor.lightThemeBackground()
  } else if activeTheme == theme.dark.rawValue {
    view.backgroundColor = UIColor.darkThemeBackground()
  }
}

Here, we first get the image and set the rendering mode to `alwaysTemplate` so that we can change the tint color of the image. Next, we assign the image to the `IBOutlet` and change the tint color to `white`. We also change the status bar style to `lightContent`. Next, we check the active theme and change the view’s background color accordingly. For this method to execute, we call it inside, `viewDidLoad` so that the theme loads up as the view loads.

Next, lets add this option of switching between themes inside the `SettingsViewController`. We add a cell with `titleLabel` as `Change Theme` and use the collectionView’s delegate method of `didSelect` to show an alert. This alert contains three options, Dark theme, Light Theme and Cancel. Let’s code that method which presents the alert.

func themeToggleAlert() {
  let imageDialog = UIAlertController(title: ControllerConstants.toggleTheme, message: nil, preferredStyle: UIAlertControllerStyle.alert)
  imageDialog.addAction(UIAlertAction(title: theme.dark.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
    AppDelegate().activeTheme = theme.dark.rawValue
    self.settingChanged(sender: self.imagePicker)
    self.setupTheme()
  }))
  imageDialog.addAction(UIAlertAction(title: theme.light.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
    AppDelegate().activeTheme = theme.light.rawValue
    self.settingChanged(sender: self.imagePicker)
    self.setupTheme()
  }))
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
  }))
  self.present(imageDialog, animated: true, completion: nil)
}

Here, we assign the alert view’s title and add 3 actions and their respective completion handlers. If we see inside these completion handlers, we can notice that we first dismiss the alert followed by updating the activeTheme variable in AppDelegate and call the `settingChanged` function which updates the user’s settings on the server. Finally, we update the color scheme.

Now, if we build and run the app and change the theme from the settings, we will notice that on returning to the chat view, the color scheme is not updated. The reason here is that we are setting up the theme on viewDidLoad which loads only once and is not executed until the controller is presented again. Here, we make use of the `viewDidAppear` method which executes every time the view appears.

override func viewDidAppear(_ animated: Bool) {
  super.viewDidAppear(animated)
  setupTheme()
}

To persist the selected theme, we used the UserDefaults to save the theme which got assigned everytime to the `activeTheme` variable when the app loads up.

UserDefaults.standard.set(AppDelegate().activeTheme, forKey: ControllerConstants.UserDefaultsKeys.theme)

On app launch, we assigned this User Default the value of the light theme as a default.

Below is the final output:

References:

Continue ReadingUsing The Dark and Light Theme in SUSI iOS

Advanced functionality in SUSI FBbot

SUSI AI is integrated to Facebook (blog). During the initial phase, SUSI FBbot had basic UI and functionalities like just “plain text” replies. Facebook provides many more features like replies enclosed in templates (blog link), sharing the replies by SUSI A.I. with friends, get started button or a persistent menu to show quick reply options to the user etc. All these features to enhance the user experience with SUSI AI chatbot.

This blog post walks you through on adding these functionalities to the SUSI FBbot:

Adding Get Started button

A Get Started button is added to the SUSI FBbot to give the user a brief introduction about SUSI AI and what the user can try next.

Clicking on the get started button , will send the message as “Get Started” to the SUSI FBbot:

The reply message, provides the user with options to visit SUSI A.I. repository or to just start chatting.

To have this button in our bot, we use this code snippet:

// Add a get started button to the messenger
request({
    url: 'https://graph.facebook.com/v2.6/me/messenger_profile',
    qs: {access_token:token},
    method: 'POST',
    json: { 
      "get_started":{
        "payload":"GET_STARTED_PAYLOAD"
      }
    }
}, function(error, response, body) {
    // handle errors and response here
})

When a user clicks this button, a postback is sent to the webhook of SUSI FBbot with payload as “GET_STARTED_PAYLOAD”. When we receive such postback, we reply with a message as shown above using generic template.

Adding a persistent menu to the bot

If not at the start, while chatting with SUSI AI for sometime, it is quite possible that the user becomes curious to visit the repository of SUSI A.I. . So we need a quick access to the “Visit repository” button all the time. Persistent menu, helps us with this feature:

This way it is accessible at each point of time. Some other buttons can also be added to the menu like “Latest News” as shown in the image above.

To have a persistent menu for the SUSI FBbot, the following code snippet is used:

request({
        url: 'https://graph.facebook.com/v2.6/me/messenger_profile',
        qs: {access_token:token},
        method: 'POST',
        json: {
                "persistent_menu":[{
                    "locale":"default", 
                        "composer_input_disabled":false,
                        "call_to_actions":[{
                            "type":"web_url",
                            "title":"Visit Repository",
                            "url":"https://github.com/fossasia/susi_server",
                            "webview_height_ratio":"full"
                        }]
                 }]
            }
    }, function(error, response, body) {
        // handle errors and response
    })

We can add more buttons to the menu. JSON object having the required properties of that button can be appended to the key “call_to_actions” to do so.

Adding a messenger code to join SUSI FBbot

To enable Facebook users to chat with SUSI AI by scanning a code through messenger. This feature is added to the bot by making the following POST request:

request({
        url: 'https://graph.facebook.com/v2.6/me/messenger_codes',
        qs: {access_token:token},
        method: 'POST',
        json: {
                type: "standard",
                image_size: 1000
        }
    }, function(error, response, body) {
        // handle errors and response.
});

Adding message sharing feature

To increase the reach of SUSI A.I. to more users on Facebook, message sharing proves to be a big boon. The reply by SUSI A.I. to users can be shared with their friends. Along with the message we can also send a promotional message(related to SUSI A.I.), to the people with which the message was shared.

This sharing can end up having more users trying SUSI A.I., leading to increase the user base of SUSI AI and its popularity.

We can allow sharing of just the message(i.e. the reply) or a promotional message with it. In case of just the reply:

Clicking the share button, will share just the reply with another person. To add capabilities of sharing the reply along with one more message(prompting to try SUSI A.I.), some changes to the code are done:

We need to set the buttons property in generic template like:

buttons : [
            {
                "type":"element_share",
                    "share_contents": { 
                      "attachment": {
                        "type": "template",
                        "payload": {
                          "template_type": "generic",
                          "elements": [
                            {
                              "title": "I had an amazing chat with SUSI.",
                              "buttons": [
                                {
                                  "type": "web_url",
                                  "url": "https://m.me/asksusisu", 
                                  "title": "Chat with SUSI AI"
                                }
                              ]
                            }
                          ]
                        }
                      }
                   }
            } 
       ];

This way when a user shares the message with other, an extra message is sent with the original message, tempting the user to try a chat with SUSI A.I.:

Resources:

  1. By Seth Rosenberg from Facebook developers blogLink Ads to Messenger, Enhanced Mobile Websites, Payments and More.
  2. By Slobodan Stojanović from smashing magazineDevelop a chat bot with node js.
Continue ReadingAdvanced functionality in SUSI FBbot

Adding Description to the Susi AI Skills

Susi skill CMS is an editor to write and edit skill easily. It follows an API-centric approach where the Susi server acts as API server and a web front-end act as the client for the API and provides the user interface. A skill is a set of intents. One text file represents one skill, it may contain several intents which all belong together. All the skills are stored in Susi Skill Data repository and the schema is as following.

Using this, one can access any skill based on four tuples parameters model, group, language, skill. To know what a skill is about we needed to add a !description operator which identifies the text as a description for the skill. Let’s check out how to achieve it.Susi Skill class provides parser methods for the set of intents, given as text files.

 public static JSONObject readEzDSkill(BufferedReader br) throws JSONException {}
if (line.startsWith("!") && (thenpos = line.indexOf(':')) > 0) {
        String head = line.substring(1, thenpos).trim().toLowerCase();
       String tail = line.substring(thenpos + 1).trim();
if (head.equals("description")) {
   description =tail;
    }
}
 if (description.length() > 0) intent.put("description", description); 

The method readEzDSkill parses the skill txt file, it checks if a line starts with ‘!description’ (‘bang operator with description’) it then stores the content in string variable description.
If a description is found in a skill, it is recorded and put into Json Array of intents.

private final Map<String, Set<String>> skillDescriptions; 
 if (intent.getDescription() !=null) {
  Set<String> descriptions = this.skillDescriptions.get(intent.getSkill());
  if (descriptions == null) {
     descriptions = new LinkedHashSet<>();
     this.skillDescriptions.put(intent.getSkill(), descriptions);
   }
   descriptions.add(intent.getDescription());
}

SusiMind class  process this json and stores the description in a map of skill path and description. This map is used by DescriptionSkillService to list descriptions for all the skills given its model, group and language. For adding the description servlet we need to inherit the service class from AbstractAPIHandler and implement APIhandler interface.In Susi Server, an abstract class AbstractAPIHandler extending HttpServelets and implementing API handler interface is provided.

 @Override
    public BaseUserRole getMinimalBaseUserRole() { return BaseUserRole.ANONYMOUS; }

    @Override
    public JSONObject getDefaultPermissions(BaseUserRole baseUserRole) {
        return null;
    }

    @Override
    public String getAPIPath() {
        return "/cms/getDescriptionSkill.json";
    }

The getAPIPath() methods sets the API endpoint path, it gets appended to base path which is 127.0.0.1:4000/cms/getDescriptionSkill.json for local host. The getMinimalBaseRole method tells the minimum Userrole required to access this servlet it can also be ADMIN, USER. In our case it is Anonymous. A User need not log in to access this endpoint.
Next, we implement serviceimpl method which gives us the desired response in JSON format.

@Override
    public ServiceResponse serviceImpl(Query call, HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) {
        String model = call.get("model", "");
        String group = call.get("group", "");
        String language = call.get("language", "");
        JSONObject descriptions = new JSONObject(true);
            for (Map.Entry<String, Set<String>> entry : DAO.susi.getSkillDescriptions().entrySet()) {
                String path = entry.getKey();
  if ((model.length() == 0 || path.indexOf("/" + model + "/") > 0) &&(group.length() == 0 || path.indexOf("/" + group + "/") > 0) &&(language.length() == 0 || path.indexOf("/" + language + "/") > 0)) {
      descriptions.put(path, entry.getValue());
   }
            }
            JSONObject json = new JSONObject(true)
                    .put("model", model)
                    .put("group", group)
                    .put("language", language)
                    .put("descriptions", descriptions);
        return new ServiceResponse(json);
    }

We can get the required parameters through a call.get() method where the first parameter is the key for which we want to get the value and second parameter is the default value. If the path contains the desired language, group and model, we return it as a response otherwise an error message is displayed. To check the response go to http://api.susi.ai/cms/getDescriptionSkill.json?model=general&group=knowledge&language=en or http://127.0.0.1:4000/cms/getDescriptionSkill.json.

This is how getDescriptionSkill service works. To add a description to the skill visit susi_skill_data, the storage place for susi skills. For more information and complete code take a look at Susi server and join gitter chat channel for discussions.

Resources

Continue ReadingAdding Description to the Susi AI Skills

Use of Flux Architecture to Switch between Themes in SUSI Web Chat

While we were developing the SUSI Web Chat we got a requirement to build a dark theme. And to build a way that user can switch between dark and light theme.

SUSI Web Chat application is made according to the Flux architecture, So we had to build sub components according to that architecture.

What is flux:

Flux is not a framework. It is an Architecture/pattern that we can use to build applications using react and some other frameworks. Below figure shows the way how that architecture works and how it communicate.

How flux works:

Flux has four types of components. Those are views, actions, dispatcher and stores. We use JSX to build and integrate views into our JavaScript code.

When someone triggers an event from view, it triggers an action and that action sends particular action details  such as Actiontype, action name  and data to dispatcher. Dispatcher broadcasts those details to every store which are registered with the particular dispatcher. That means every store gets all the action details and data which are broadcasting from dispatchers which they are registered.

Let’s say we have triggered an action from view that is going to change the value of the store. Those action details are coming to dispatcher. Then dispatcher broadcasts those data to every store that registered with it. Matching action will trigger and update its value. If there is any change happened in store, view automatically updates corresponding view.

How themes are changing:

We have a store called “SettingStore.js”. This “SettingStore” contains theme values like current theme. We store other settings of the application such as: Mic input settings, Custom server details, Speech Output details, Default Language, etc.it is responsible to provide these details to corresponding view.

let CHANGE_EVENT = 'change';
class SettingStore extends EventEmitter {
   constructor() {
       super();
       this.theme = true; 
   }

We use “this.theme = true” in constructor to switch light theme as the default theme.

getTheme() { //provides current value of theme
       return this.theme;
   }

This method returns the value of the current theme when it triggers.

   changeTheme(themeChanges) {
       this.theme = !this.theme;
       this.emit(CHANGE_EVENT);
   }

We use “changeTheme” method to change the selected theme.

   handleActions(action) {
       switch (action.type) {
           case ActionTypes.THEME_CHANGED: {
               this.changeTheme(action.theme);
               break;
           }
           default: {
               // do nothing
           }
       }
   }
}

This switch case is the place that store gets actions distributed from the dispatcher and executes relevant method.

const settingStore = new SettingStore();
ChatAppDispatcher.register(settingStore.handleActions.bind(settingStore));
export default settingStore;

Here we registered our store(SettingStore) to “ChatAppDispatcher” .

This is how Store works.
Now we need to get the default light theme to the view. For that we have to call ”getTheme()” function. We can use the value it returns to update the state of the application.
Now we are going to change the theme. To do that we have to trigger “changeTheme” method of “Settingstrore” from view ( MessageSection.react.js ).
We trigger below method on click of the “Change Theme” button. It triggers the action called “themeChanged”.

 themeChanger(event) {
   Actions.themeChanged(!this.state.darkTheme);
 }

Previous method executes “themeChanged()” function of the actions.js file.

export function themeChanged(theme) {
 ChatAppDispatcher.dispatch({
   type: ActionTypes.THEME_CHANGED,
   theme //data came from parameter
 });
};

In above function we collect data from the view and send those data, method details to dispatcher.
Dispatcher sends those details to each and every registered store. In our case we have “SettingStore” and update the state to new dark theme.
This is how themes are changing in SUSI AI Web Chat application. Check this link to see the preview.

Resources:

  • Read About Flux: https://facebook.github.io/flux/
  • GitHub repository: https://github.com/fossasia/chat.susi.ai
  • Live Web Chat: http://chat.susi.ai/
Continue ReadingUse of Flux Architecture to Switch between Themes in SUSI Web Chat

How to Store and Retrieve User Settings from SUSI Server in SUSI iOS

Any user using the SUSI iOS client can set preferences like enabling or disabling the hot word recognition or enabling input from the microphone. These settings need to be stored, in order to be used across all platforms such as web, Android or iOS. Now, in order to store these settings and maintain a synchronization between all the clients, we make use of the SUSI server. The server provides an endpoint to retrieve these settings when the user logs in.

First, we will focus on storing settings on the server followed by retrieving settings from the server. The endpoint to store settings is as follows:

http://api.susi.ai/aaa/changeUserSettings.json?key=key&value=value&access_token=ACCESS_TOKEN

This takes the key value pair for storing a settings and an access token to identify the user as parameters in the GET request. Let’s start by creating the method that takes input the params, calls the API to store settings and returns a status specifying if the executed successfully or not.

 let url = getApiUrl(UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.ipAddress) as! String, Methods.UserSettings)

        _ = makeRequest(url, .get, [:], parameters: params, completion: { (results, message) in
            if let _ = message {
                completion(false, ResponseMessages.ServerError)
            } else if let results = results {
                guard let response = results as? [String : AnyObject] else {
                    completion(false, ResponseMessages.ServerError)
                    return
                }
                if let accepted = response[ControllerConstants.accepted] as? Bool, let message = response[Client.UserKeys.Message] as? String {
                    if accepted {
                        completion(true, message)
                        return
                    }
                    completion(false, message)
                    return
                }
            }
        })

Let’s understand this function line by line. First we generate the URL by supplying the server address and the method. Then, we pass the URL and the params in the `makeRequest` method which has a completion handler returning a results object and an error object. Inside the completion handler, check for any error, if it exists mark the request completed with an error else check for the results object to be a dictionary and a key `accepted`, if this key is `true` our request executed successfully and we mark the request to be executed successfully and finally return the method. After making this method, it needs to be called in the view controller, we do so by the following code.

Client.sharedInstance.changeUserSettings(params) { (_, message) in
  DispatchQueue.global().async {
    self.view.makeToast(message)
  }
}

The code above takes input params containing the user token and key-value pair for the setting that needs to be stored. This request runs on a background thread and displays a toast message with the result of the request.

Now that the settings have been stored on the server, we need to retrieve these settings every time the user logs in the app. Below is the endpoint for the same:

http://api.susi.ai/aaa/listUserSettings.json?access_token=ACCESS_TOKEN

This endpoint accepts the user token which is generated when the user logs in which is used to uniquely identify the user and his/her settings are returned. Let’s create the method that would call this endpoint and parse and save the settings data in the iOS app’s User Defaults.

if let _ = message {
  completion(false, ResponseMessages.ServerError)
} else if let results = results {
  guard let response = results as? [String : AnyObject] else {
    completion(false, ResponseMessages.ServerError)
    return
  }
  guard let settings = 
response[ControllerConstants.Settings.settings.lowercased()] as? [String:String] else {
    completion(false, ResponseMessages.ServerError)
    return
  }
  for (key, value) in settings {
    if value.toBool() != nil {
      UserDefaults.standard.set(value.toBool()!, forKey: key)
    } else {
      UserDefaults.standard.set(value, forKey: key)
    }
  }
  completion(true, response[Client.UserKeys.Message] as? String ?? "error")
}

Here, the creation of the URL is same as we created above the only difference being the method passed. We parse the settings key value into a dictionary followed by a loop which loop’s through all the keys and stores the value in the User Defaults for that key. We simply call this method just after user log in as follows:

Client.sharedInstance.fetchUserSettings(params as [String : AnyObject]) { (success, message) in
  DispatchQueue.global().async {
    print("User settings fetch status: \(success) : \(message)")
  }
}

That’s all for this tutorial where we learned how to store and retrieve settings on the SUSI Server.

References

Continue ReadingHow to Store and Retrieve User Settings from SUSI Server in SUSI iOS

Save Chat Messages using Realm in SUSI iOS

Fetching data from the server each time causes a network load which makes the app depend on the server and the network in order to display data. We use an offline database to store chat messages so that we can show messages to the user even if network is not present which makes the user experience better. Realm is used as a data storage solution due to its ease of usability and also, since it’s faster and more efficient to use. So in order to save messages received from the server locally in a database in SUSI iOS, we are using Realm and the reasons for using the same are mentioned below.

The major upsides of Realm are:

  • It’s absolutely free of charge,
  • Fast, and easy to use.
  • Unlimited use.
  • Work on its own persistence engine for speed and performance

Below are the steps to install and use Realm in the iOS Client:

Installation:

  • Install Cocoapods
  • Run `pod repo update` in the root folder
  • In your Podfile, add use_frameworks! and pod ‘RealmSwift’ to your main and test targets.
  • From the command line run `pod install`
  • Use the `.xcworkspace` file generated by Cocoapods in the project folder alongside `.xcodeproj` file

After installation we start by importing `Realm` in the `AppDelegate` file and start configuring Realm as below:

func initializeRealm() {
        var config = Realm.Configuration(schemaVersion: 1,
            migrationBlock: { _, oldSchemaVersion in
                if (oldSchemaVersion < 0) {
                    // Nothing to do!
                }
        })
        config.fileURL = config.fileURL?.deletingLastPathComponent().appendingPathComponent("susi.realm")
        Realm.Configuration.defaultConfiguration = config
}

Next, let’s head over to creating a few models which will be used to save the data to the DB as well as help retrieving that data so that it can be easily used. Since Susi server has a number of action types, we will cover some of the action types, their model and how they are used to store and retrieve data. Below are the currently available data types, that the server supports.

enum ActionType: String {
  case answer
  case websearch
  case rss
  case table
  case map 
  case anchor
}

Let’s start with the creation of the base model called `Message`. To make it a RealmObject, we import `RealmSwift` and inherit from `Object`

class Message: Object {
  dynamic var queryDate = NSDate()
  dynamic var answerDate = NSDate()
  dynamic var message: String = ""
  dynamic var fromUser = true
  dynamic var actionType = ActionType.answer.rawValue
  dynamic var answerData: AnswerAction?
  dynamic var mapData: MapAction?
  dynamic var anchorData: AnchorAction?
}

Let’s study these properties of the message one by one.

  • `queryDate`: saves the date-time the query was made
  • `answerDate`: saves the date-time the query response was received
  • `message`: stores the query/message that was sent to the server
  • `fromUser`: a boolean which keeps track who created the message
  • `actionType`: stores the action type
  • `answerData`, `rssData`, `mapData`, `anchorData` are the data objects that actually store the respective action’s data

To initialize this object, we need to create a method that takes input the data received from the server.

// saves query and answer date
if let queryDate = data[Client.ChatKeys.QueryDate] as? String,
let answerDate = data[Client.ChatKeys.AnswerDate] as? String {
  message.queryDate = dateFormatter.date(from: queryDate)! as NSDate
  message.answerDate = dateFormatter.date(from: answerDate)! as NSDate}if let type = action[Client.ChatKeys.ResponseType] as? String,
  let data = answers[0][Client.ChatKeys.Data] as? [[String : AnyObject]] {
  if type == ActionType.answer.rawValue {
     message.message = action[Client.ChatKeys.Expression] as! String
     message.actionType = ActionType.answer.rawValue
    message.answerData = AnswerAction(action: action)
  } else if type == ActionType.map.rawValue {
    message.actionType = ActionType.map.rawValue
    message.mapData = MapAction(action: action)
  } else if type == ActionType.anchor.rawValue {
    message.actionType = ActionType.anchor.rawValue
    message.anchorData = AnchorAction(action: action)
    message.message = message.anchorData!.text
  }
}

Since, the response from the server for a particular query might contain numerous action types, we create loop inside a method to capture all those action types and save each one of them. Since, there are multiple action types thus we need a list containing all the messages created for the action types. For each action in the loop, corresponding data is saved into their specific objects.

Let’s discuss the individual action objects now.

  • AnswerAction
class AnswerAction: Object {
  dynamic var expression: String = ""
  convenience init(action: [String : AnyObject]) {
    self.init()
    if let expression = action[Client.ChatKeys.Expression] as? String {
      self.expression = expression
    }
  }
}

 This is the simplest action type implementation. It contains a single property `expression` which is a string type. For initializing it, we take the action object and extract the expression key-value and save it.

if type == ActionType.answer.rawValue {
  message.message = action[Client.ChatKeys.Expression] as! String
  message.actionType = ActionType.answer.rawValue
  // pass action object and save data in `answerData`
  message.answerData = AnswerAction(action: action)
}

Above is the way an answer action is checked and data saved inside the `answerData` variable.

2)   MapAction

class MapAction: Object {
  dynamic var latitude: Double = 0.0
  dynamic var longitude: Double = 0.0
  dynamic var zoom: Int = 13

  convenience init(action: [String : AnyObject]) {
    self.init()
    if let latitude = action[Client.ChatKeys.Latitude] as? String,
    let longitude = action[Client.ChatKeys.Longitude] as? String,
    let zoom = action[Client.ChatKeys.Zoom] as? String {
      self.longitude = Double(longitude)!
      self.latitude = Double(latitude)!
      self.zoom = Int(zoom)!
    }
  }
}

This action implementation contains three properties, `latitude` `longitude` `zoom`. Since the server responds the values inside a string, each of them need to be converted to their respective type using force-casting. Default values are provided for each property in case some illegal value comes from the server.

3)   AnchorAction

class AnchorAction: Object {
  dynamic var link: String = ""
  dynamic var text: String = ""

  convenience init(action: [String : AnyObject]) {
    self.init()if let link = action[Client.ChatKeys.Link] as? String,
    let text = action[Client.ChatKeys.Text] as? String {
      self.link = link
      self.text = text
    }
  }
}

Here, the link to the openstreetmap website is saved in order to retrieve the image for displaying.

Finally, we need to call the API and create the message object and use the `write` clock of a realm instance to save it into the DB.

if success {
  self.collectionView?.performBatchUpdates({
    for message in messages! {
    // real write block
      try! self.realm.write {
        self.realm.add(message)
        self.messages.append(message)
        let indexPath = IndexPath(item: self.messages.count - 1, section: 0)
        self.collectionView?.insertItems(at: [indexPath])
      }
   }
}, completion: { (_) in
    self.scrollToLast()
  })
}

list of message items and inserted into the collection view.Below is the output of the Realm Browser which is a UI for viewing the database.

References:

Continue ReadingSave Chat Messages using Realm in SUSI iOS

Skill History Component in Susi Skill CMS

SUSI Skill CMS is an editor to write and edit skill easily. It is built on ReactJS framework and follows an API centric approach where the Susi server acts as API server. Using Skill CMS we can browse history of a skill, where we get commit ID, commit message and name the author who made the changes to that skills. In this blog post, we will see how to add skill revision history component in Susi Skill CMS.

One text file represents one skill, it may contain several intents which all belong together. Susi skills are stored in susi_skill_data repository. We can access any skill based on four tuples parameters model, group, language, skill.

<Menu.Item key="BrowseRevision">
 <Icon type="fork" />
   Browse Skills Revision
 <Link to="/browseHistory"></Link>
</Menu.Item>

First let’s create a option in sidebar menu, and link it “/browseHistory” route created at index.js 

 <Route path="/browseHistory" component={BrowseHistory} />

Next we will be adding skill versioning using endpoints provided by Susi Server, to select a skill we will create a drop down list, for this we will be using Select Field a component of  Material UI.

request('http://cors-anywhere.herokuapp.com/api.susi.ai/cms/getModel.json').then((data) => {
    console.log(data.data);
    data = data.data;
    for (let i = 0; i < data.length; i++) {
        models.push(<MenuItem value={i} key={data[i]} primaryText={`${data[i]}`}/>);
    }

    console.log(models);
});
 <SelectField
   floatingLabelText="Expert"
   style={{width: '50px'}}
   value={this.state.value}
   onChange={this.handleChange}>
      {experts}
  </SelectField>

We store the models, groups and languages in array using the endpoints api.susi.ai/cms/getModel.json, api.susi.ai/cms/getGroups.json, api.susi.ai/cms/getAllLanguages.json set the values in respective select fields. The request functions takes the url as string and the parses the json and fetches the object containing data or error depending on the response from the server. Once run your project using

npm start

And you would be able to see the drop down list working

Next, we will use Material UI tables for displaying the organized data. For using  table component we need to import table, it’s body, header and row column from Material ui class. 

import {Table, TableBody, TableHeader, TableHeaderColumn, TableRow, TableRowColumn} from "material-ui/Table";

We then make our header Columns, in our case it’s three, namely Commit ID, Commit Message, Author name.

 <TableRow>
  <TableHeaderColumn tooltip="Commit ID">Commit ID</TableHeaderColumn>
  <TableHeaderColumn tooltip="Commit Message">Commit Message</TableHeaderColumn>
  <TableHeaderColumn tooltip="Author Name">Author Name</TableHeaderColumn>
 </TableRow>

To get the history of modification of a skill, we will use endpoint “http://api.susi.ai/cms/getSkillHistory.json”. It uses JGit for managing version control in skill data repository. JGit is a library which implements the Git functionality in Java. An endpoint is accessed based on userRoles, which can be Admin, Privilege, User, Anonymous. In our case it is Anonymous. Thus a User need not to log in to access this endpoint.

 url = "http://api.susi.ai/cms/getSkillHistory.json?model="+models[this.state.modelValue].key+"&group="+groups[this.state.groupValue].key+"&language="+languages[this.state.languageValue].key+"&skill="+this.state.expertValue;

After getting the url, we will next make a ajax network call to get the modification history of skill., if the method returns success, we get the desired data in table array and display it in rows through its render() method, which checks if the data is set in state -if so, it renders the contents otherwise we display the error occurred while processing the request.

 $.ajax({
            url: url,
            jsonpCallback: 'pccd',
            dataType: 'jsonp',
            jsonp: 'callback',
            crossDomain: true,
            success: function(data) {
                data = data.commits;
                let array = [];
                for(let i=0;i<data.length;i++){
                    array.push(data[i]);
                }
{tableData.map((row, index) => (
  <TableRow key={index}>
    <TableRowColumn>{row.commitID}</TableRowColumn>
    <TableRowColumn>{row.commit_message}</TableRowColumn>
    <TableRowColumn>{row.author}</TableRowColumn>
   </TableRow>
))}

Test the final output  on http://skills.susi.ai/browseHistory or http://localhost:3000/browseHistory , select the model, group , language and skill and get the history of that skill.


Next time when you need drop down list or tables to organize your data, do check out https://github.com/callemall/material-ui for examples, which can help in providing good outlines to you apps. For contributions to susi_skill_cms, join our chat channel  on gitter: https://gitter.im/fossasia/susi_server and browse https://github.com/fossasia/susi_skill_cms for complete code.

Resources

Continue ReadingSkill History Component in Susi Skill CMS