Author: Chashmeet Singh

This author has written 16 articles
Read more about the article Implementing Skill Listing in SUSI iOS

Implementing Skill Listing in SUSI iOS

Skills are basically a set of rules which respond to the user’s queries through any client app. All the skills are defined in the SUSI Skill Data repo where the user’s queries are matched with the already present skills and the server responds accordingly. Apps like Alexa, Google Assistant have an interface to view skills with a set of sample queries that can be used therefore, we are adding the same skill display UI in the SUSI iOS app. Implementation All the skills are arranged into categories or groups, so we first need to fetch all those groups followed by fetching skills from each group. For listing all the groups, we use the endpoint below: http://api.susi.ai/cms/getGroups.json Which returns all the groups in the groups object like below: { "session": {"identity": { "type": "host", "name": "67.214.191.117", "anonymous": true }}, "accepted": true, "groups": [ "Social", "News", "Food and Drink", "Travel and Transportation", "Connected Car", "Movies and TV", "Problem Solving", "Knowledge", "Business and Finance", "Productivity", "Games, Trivia and Accessories", "Lifestyle", "Health and Fitness", "Music and Audio", "Shopping", "Communication", "Novelty and Humour", "Utilities", "Sports", "Weather" ], "message": "Success: Fetched group list" } After the groups have been fetched, we need to get the skills for each group. Here, we use the endpoint below: http://api.susi.ai/cms/getSkillList.json?group=GROUP_NAME Since, we have a number of groups, we need to make the above API call as many times as the group count is. A sample call would look like: http://api.susi.ai/cms/getSkillList.json?group=News which would fetch all the skills in the News group. { "accepted": true, "model": "general", "group": "News", "language": "en", "skills": {"news": { "image": "images/news.png", "author_url": "https://github.com/AliAyub007", "examples": [ "News", "latest news", "most viewed articles in science today", "most viewed articles in science in the last week", "most viewed articles in science in the last month", "most shared articles in science today", "can you tell me last week's most shared articles in science", "do you know most shared articles in arts in the last month", "most emailed articles today in arts", "most emailed articles in science in the last week", "can you tell me most emailed articles in science in the last month", "articles in science", "show me articles", "most emailed articles", "most shared articles", "tell me news in tech world" ], "developer_privacy_policy": null, "author": "Ali Ayub Khan", "skill_name": "NEWS", "dynamic_content": true, "terms_of_use": null, "descriptions": "A skill to give news.", "skill_rating": null }}, "message": "Success: Fetched skill list", "session": {"identity": { "type": "host", "name": "23.94.137.239", "anonymous": true }} } Each such json object gives us the following values: Model Group Language Image Path Author’s URL Author’s Name A list of sample queries Skill Name Licence and terms of use Rating Description Implementation in SUSI iOS The UI for listing skills is a little complex as it consists of a UITableView where each UITableViewCell consists of a UILabel(group name) and a UICollectionVIew (horizontal scroll). Let’s see the step by step process to implement the skill listing. First, we need to fetch all the groups available using the endpoint above. // get all…

Continue ReadingImplementing Skill Listing in SUSI iOS
Read more about the article Settings Controller UI using Static Table View

Settings Controller UI using Static Table View

Dynamic Table Views are used at places where there may be any kind of reusability of cells. This means that there would exist cells that would have the same UI elements but would differ in the content being displayed. Initially the Settings Controller was built using UICollectionViewController which is completely dynamic but later I realized that the cells will remain static every time so there is no use of dynamic cells to display the UI hence, I switched to static table view cells. Using Static Table View is very easy. In this blog, I will explain how the implementation of the same was achieved in SUSI iOS app. Let’s start by dragging and dropping a UITableViewController into the storyboard file. The initial configuration of the UITableView has content as Dynamic Prototypes but we need Static cells so we choose them and make the sections count to 5 to suit our need. Also, to make the UI better, we choose the style as Grouped. Now for each section, we have the control of statically adding UI elements so, we add all the settings with their corresponding section headers and obtain the following UI.         After creating this UI, we can refer any UI element independently be it in any of the cells. So here we create references to each of the UISlider and UISwitch so that we can trigger an action whenever the value of anyone of them changes to get their present state. To create an action, simply create a function and add `@IBAction` in front so that they can be linked with the UI elements in the storyboard and then click and drag the circle next to the function to UI element it needs to be added. After successful linking, hovering over the same circle would reveal all the UI elements which trigger that function. Below is a method with the @IBAction identifier indicating it can be linked with the UI elements in the storyboard. This method is executed whenever any slider or switch value changes, which then updates the UserDefaults value as well sends an API request to update the setting for the user on the server. @IBAction func settingChanged(sender: AnyObject?) {         var params = [String: AnyObject]()         var key: String = ""         if let senderTag = sender?.tag {             if senderTag == 0 {                 key = ControllerConstants.UserDefaultsKeys.enterToSend             } else if senderTag == 1 {                 key = ControllerConstants.UserDefaultsKeys.micInput             } else if senderTag == 2 {                 key = ControllerConstants.UserDefaultsKeys.hotwordEnabled             } else if senderTag == 3 {                 key = ControllerConstants.UserDefaultsKeys.speechOutput             } else if senderTag == 4 {                 key = ControllerConstants.UserDefaultsKeys.speechOutputAlwaysOn             } else if senderTag == 5 {                 key = ControllerConstants.UserDefaultsKeys.speechRate             } else if senderTag == 6 {                 key = ControllerConstants.UserDefaultsKeys.speechPitch             }             if let slider = sender as? UISlider {                 UserDefaults.standard.set(slider.value, forKey: key)             } else {                 UserDefaults.standard.set(!UserDefaults.standard.bool(forKey: key), forKey: key)             }             params[ControllerConstants.key] = key as AnyObject…

Continue ReadingSettings Controller UI using Static Table View
Read more about the article Reset Password Functionality in SUSI iOS

Reset Password Functionality in SUSI iOS

Reset Password as the name suggests is one of the features in the SUSI iOS app which allows a user to change his/her password when they are logged in. This feature was added because a user would want to change his password sometimes to prevent unauthorized access or make his account security stronger. We can find many popular apps online such as Facebook, Gmail, which allow the user to reset their password. The way this is done is pretty simple and all we need from the user is his current and the new password he/she wants to set. In this blog post, I am going to explain step by step how this is implemented in the iOS client. Implementation The option to Reset Password is provided to the user under the Settings Controller. On selecting the row, the user is presented with another view which asks the user for his/her current password, new password, and another field to confirm the newly entered password. First, the user needs to provide his current password followed by the new password. The user’s current password is required just to authenticate that the account’s owner is requesting the password change. The new password field is followed by another field called confirm password just to make sure there isn’t any typo. Now when the field is filled, the user clicks the `Reset password` button at the bottom. What happens here is, first, the fields are validated to ensure the correct length of the passwords followed by an API request to update the same. The endpoint for the same is as below: http://api.susi.ai/aaa/changepassword.json?changepassword=user_email&password=current _pass&newpassword=new_pass&access_token=user_access_token This endpoint requires 3 things: Current Password New Password User’s email Access Token obtained at the time of login func validatePassword() -> [Bool:String] {         if let newPassword = newPasswordField.text,             let confirmPassword = confirmPasswordField.text {             if newPassword.characters.count > 5 {                 if newPassword == confirmPassword {                     return [true: ""]                 } else {                     return [false: ControllerConstants.passwordDoNotMatch]                 }             } else {                 return [false: ControllerConstants.passwordLengthShort]             }         }         return [false: Client.ResponseMessages.ServerError]     } Initially, we were not saving the user’s email, so we added the user’s email to the User’s object which is saved at the time of login. if var userData = results { userData[Client.UserKeys.EmailOfAccount] = user.email UserDefaults.standard.set(userData, forKey: ControllerConstants.UserDefaultsKeys.user) self.saveUserGlobally(user: currentUser) } At last, the API call is made which is implemented as below: let params = [ Client.UserKeys.AccessToken: user.accessToken, Client.UserKeys.EmailOfAccount: user.emailID, Client.UserKeys.Password: currentPasswordField.text ?? "", Client.UserKeys.NewPassword: newPasswordField.text ?? "" ] Client.sharedInstance.resetPassword(params as [String : AnyObject], { (_, message) in DispatchQueue.main.async { self.view.makeToast(message) self.setUIActive(active: false) } }) Below is the final UI. Reference Alamofire Docs for making network calls: https://github.com/Alamofire/Alamofire Tutorial on how to create static table view cells: https://www.appcoda.com/ios-static-table-view-storyboard/ Pull Request for API reference: https://github.com/fossasia/susi_server/pull/352

Continue ReadingReset Password Functionality in SUSI iOS
Read more about the article How to Implement Feedback System in SUSI iOS

How to Implement Feedback System in SUSI iOS

The SUSI iOS app provides responses for various queries but the response is always not accurate. To improve the response, we make use of the feedback system, which is the first step towards implementing Machine Learning on the SUSI Server. The way this works is that for every query, we present the user with an option to upvote or downvote the response and based on that a positive or negative feedback is saved on the server. In this blog, I will explain how this feedback system was implemented in the SUSI iOS app. Steps to implement: We start by adding the UI which is two buttons, one with a thumbs up and the other with a thumbs down image. textBubbleView.addSubview(thumbUpIcon) textBubbleView.addSubview(thumbDownIcon) textBubbleView.addConstraintsWithFormat(format: "H:[v0]-4-[v1(14)]-2-[v2(14)]-8-|", views: timeLabel, thumbUpIcon, thumbDownIcon) textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbUpIcon) textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbDownIcon) thumbUpIcon.isUserInteractionEnabled = true thumbDownIcon.isUserInteractionEnabled = true Here, we add the subviews and assign constraints so that these buttons align to the bottom right next to each other. Also, we enable the user interaction for these buttons. We know that the user can rate the response by pressing either of the buttons added above. To do that we make an API call to the endpoint below: BASE_URL+'/cms/rateSkill.json?'+'model='+model+'&group='+group+'&skill='+skill+’&language’+language+’&rating=’+rating Here, the BASE_URL is the url of the server, the other three params model, group, language and skill are retrieved by parsing the skill location parameter we get with the response. The rating is positive or negative based on which button was pressed by the user. The skill param in the response looks like this: skills: [ "/susi_skill_data/models/general/entertainment/en/quotes.txt" ] Let’s write the method that makes the API call and responds to the UI that it was successful. if let accepted = response[ControllerConstants.accepted] as? Bool { if accepted { completion(true, nil) return } completion(false, ResponseMessages.ServerError) return } Here after receiving a response from the server, we check if the `accepted` variable is true or not. Based on that, we pass `true` or `false` to the completion handler. Below the response we actually receive by making the request. { session: { identity: { type: "host", name: "23.105.140.146", anonymous: true } }, accepted: true, message: "Skill ratings updated" } Finally, let’s update the UI after the request has been successful. if sender == thumbUpIcon { thumbDownIcon.tintColor = UIColor(white: 0.1, alpha: 0.7) thumbUpIcon.isUserInteractionEnabled = false thumbDownIcon.isUserInteractionEnabled = true feedback = "positive" } else { thumbUpIcon.tintColor = UIColor(white: 0.1, alpha: 0.7) thumbDownIcon.isUserInteractionEnabled = false thumbUpIcon.isUserInteractionEnabled = true feedback = "negative" } sender.tintColor = UIColor.hexStringToUIColor(hex: "#2196F3") Here, we check the sender (the thumbs up or down button) and based on that pass the rating (positive or negative) and update the color of the button. Below is the app in action with the feedback system. Resources: Tutorial on Raywanderlich discusses the way custom collection view cells are created Another tutorial on creating custom cells Alamofire Docs: explains usage of Alamofire to make API calls Material.io icons: source of the icons used

Continue ReadingHow to Implement Feedback System in SUSI iOS
Read more about the article Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

In this blog post, I will be explaining how to encode and decode images and save them in UserDefaults so that the image persists even if it is removed from the Photos app. It happens a number of times that images are removed from the gallery by the users which results in the app loosing the image. So, to avoid this, we save the image by encoding it in a data object and save it inside UserDefaults. In SUSI iOS app we simply select an image from the image picker, encode it and save it in UserDefaults. To set the image, we simply fetch the image data from the UserDefaults and decode it to an image. There are two ways we can do the encoding and decoding process: Using Data object Using Base64 string For the scope of this tutorial, we will use the Data object. Implementation Steps To use the image picker, we need to add permissions to `Info.plist` file. <key>NSLocationWhenInUseUsageDescription</key> <string>Susi is requesting to get your current location</string> <key>NSPhotoLibraryUsageDescription</key> <string>Susi needs to request your gallery access to select wallpaper</string> Select image from gallery First, we present an alert which gives an option to select the image from the gallery. // Show wallpaper options to set wallpaper or clear wallpaper func showWallpaperOptions() { let imageDialog = UIAlertController(title: ControllerConstants.wallpaperOptionsTitle, message: nil, preferredStyle: UIAlertControllerStyle.alert) imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsPickAction, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) self.showImagePicker() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsNoWallpaperAction, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) self.removeWallpaperFromUserDefaults() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) })) self.present(imageDialog, animated: true, completion: nil) } Here, we create and UIAlertController with three options to select, one which presents the image picker controller, the second one removes the background wallpaper and the third dismisses the alert. Set the image as background view // Callback when image is selected from gallery func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) { dismiss(animated: true, completion: nil) let chosenImage = info[UIImagePickerControllerOriginalImage] as? UIImage if let image = chosenImage { setBackgroundImage(image: image) } } We use the `didFinishPickingMediaWithInfo` delegate method to set the image as background. First we get the image using the the `info` dictionary using the `UIImagePickerControllerOriginalImage` key. Save the image in UserDefaults (encoding) // Save image selected by user to user defaults func saveWallpaperInUserDefaults(image: UIImage!) {   let imageData = UIImageJPEGRepresentation(image!, 1.0)   let defaults = UserDefaults.standard   defaults.set(imageData, forKey: userDefaultsWallpaperKey) } We first convert the image to a data object using the `UIImageJPEGRepresentation` method followed by saving the data object in UserDefaults with the key `wallpaper`. Decode the data object back to UIImage  Now whenever we need to decode the image, we simply get the data object from the UserDefaults and use it to display the image. // Check if user defaults have an image data saved else return nil/Any func getWallpaperFromUserDefaults() -> Any? {   let defaults = UserDefaults.standard   return defaults.object(forKey: userDefaultsWallpaperKey) } Below is the output when an image is…

Continue ReadingEncoding and Decoding Images as Data in UserDefaults in SUSI iOS
Read more about the article Using The Dark and Light Theme in SUSI iOS

Using The Dark and Light Theme in SUSI iOS

SUSI being an AI for interactive chat bots, provides answers to the users in the intelligent way. So, to make the SUSI iOS app more user friendly, the option of switching between themes was introduced. This also enables the user switch between themes based on the environment around. Any user can switch between the light and dark themes easily from the settings. We start by declaring an enum called `theme` which contains two strings namely, dark and light. enum theme: String {     case light     case dark } We can update the color scheme based on the theme selected very easily by checking the currently active theme and based on that check, we update the color scheme. To check the currently active theme, we define a variable in the `AppDelegate` which holds the value. var activeTheme: String? Below is the way the color scheme of the LoginViewController is set. var activeTheme: String?func setupTheme() { let image = UIImage(named: ControllerConstants.susi)?.withRenderingMode(.alwaysTemplate) susiLogo.image = image susiLogo.tintColor = .white UIApplication.shared.statusBarStyle = .lightContent let activeTheme = AppDelegate().activeTheme if activeTheme == theme.light.rawValue { view.backgroundColor = UIColor.lightThemeBackground() } else if activeTheme == theme.dark.rawValue { view.backgroundColor = UIColor.darkThemeBackground() } } Here, we first get the image and set the rendering mode to `alwaysTemplate` so that we can change the tint color of the image. Next, we assign the image to the `IBOutlet` and change the tint color to `white`. We also change the status bar style to `lightContent`. Next, we check the active theme and change the view’s background color accordingly. For this method to execute, we call it inside, `viewDidLoad` so that the theme loads up as the view loads. Next, lets add this option of switching between themes inside the `SettingsViewController`. We add a cell with `titleLabel` as `Change Theme` and use the collectionView’s delegate method of `didSelect` to show an alert. This alert contains three options, Dark theme, Light Theme and Cancel. Let’s code that method which presents the alert. func themeToggleAlert() { let imageDialog = UIAlertController(title: ControllerConstants.toggleTheme, message: nil, preferredStyle: UIAlertControllerStyle.alert) imageDialog.addAction(UIAlertAction(title: theme.dark.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) AppDelegate().activeTheme = theme.dark.rawValue self.settingChanged(sender: self.imagePicker) self.setupTheme() })) imageDialog.addAction(UIAlertAction(title: theme.light.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) AppDelegate().activeTheme = theme.light.rawValue self.settingChanged(sender: self.imagePicker) self.setupTheme() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) })) self.present(imageDialog, animated: true, completion: nil) } Here, we assign the alert view’s title and add 3 actions and their respective completion handlers. If we see inside these completion handlers, we can notice that we first dismiss the alert followed by updating the activeTheme variable in AppDelegate and call the `settingChanged` function which updates the user’s settings on the server. Finally, we update the color scheme. Now, if we build and run the app and change the theme from the settings, we will notice that on returning to the chat view, the color scheme is not updated. The reason here is that we are setting up the theme on…

Continue ReadingUsing The Dark and Light Theme in SUSI iOS
Read more about the article How to Store and Retrieve User Settings from SUSI Server in SUSI iOS

How to Store and Retrieve User Settings from SUSI Server in SUSI iOS

Any user using the SUSI iOS client can set preferences like enabling or disabling the hot word recognition or enabling input from the microphone. These settings need to be stored, in order to be used across all platforms such as web, Android or iOS. Now, in order to store these settings and maintain a synchronization between all the clients, we make use of the SUSI server. The server provides an endpoint to retrieve these settings when the user logs in. First, we will focus on storing settings on the server followed by retrieving settings from the server. The endpoint to store settings is as follows: http://api.susi.ai/aaa/changeUserSettings.json?key=key&value=value&access_token=ACCESS_TOKEN This takes the key value pair for storing a settings and an access token to identify the user as parameters in the GET request. Let’s start by creating the method that takes input the params, calls the API to store settings and returns a status specifying if the executed successfully or not. let url = getApiUrl(UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.ipAddress) as! String, Methods.UserSettings)         _ = makeRequest(url, .get, [:], parameters: params, completion: { (results, message) in             if let _ = message {                 completion(false, ResponseMessages.ServerError)             } else if let results = results {                 guard let response = results as? [String : AnyObject] else {                     completion(false, ResponseMessages.ServerError)                     return                 }                 if let accepted = response[ControllerConstants.accepted] as? Bool, let message = response[Client.UserKeys.Message] as? String {                     if accepted {                         completion(true, message)                         return                     }                     completion(false, message)                     return                 }             }         }) Let’s understand this function line by line. First we generate the URL by supplying the server address and the method. Then, we pass the URL and the params in the `makeRequest` method which has a completion handler returning a results object and an error object. Inside the completion handler, check for any error, if it exists mark the request completed with an error else check for the results object to be a dictionary and a key `accepted`, if this key is `true` our request executed successfully and we mark the request to be executed successfully and finally return the method. After making this method, it needs to be called in the view controller, we do so by the following code. Client.sharedInstance.changeUserSettings(params) { (_, message) in DispatchQueue.global().async { self.view.makeToast(message) } } The code above takes input params containing the user token and key-value pair for the setting that needs to be stored. This request runs on a background thread and displays a toast message with the result of the request. Now that the settings have been stored on the server, we need to retrieve these settings every time the user logs in the app. Below is the endpoint for the same: http://api.susi.ai/aaa/listUserSettings.json?access_token=ACCESS_TOKEN This endpoint accepts the user token which is generated when the user logs in which is used to uniquely identify the user and his/her settings are returned. Let’s create the method that would call this endpoint and parse and save the settings data in the…

Continue ReadingHow to Store and Retrieve User Settings from SUSI Server in SUSI iOS
Read more about the article Save Chat Messages using Realm in SUSI iOS

Save Chat Messages using Realm in SUSI iOS

Fetching data from the server each time causes a network load which makes the app depend on the server and the network in order to display data. We use an offline database to store chat messages so that we can show messages to the user even if network is not present which makes the user experience better. Realm is used as a data storage solution due to its ease of usability and also, since it’s faster and more efficient to use. So in order to save messages received from the server locally in a database in SUSI iOS, we are using Realm and the reasons for using the same are mentioned below. The major upsides of Realm are: It’s absolutely free of charge, Fast, and easy to use. Unlimited use. Work on its own persistence engine for speed and performance Below are the steps to install and use Realm in the iOS Client: Installation: Install Cocoapods Run `pod repo update` in the root folder In your Podfile, add use_frameworks! and pod 'RealmSwift' to your main and test targets. From the command line run `pod install` Use the `.xcworkspace` file generated by Cocoapods in the project folder alongside `.xcodeproj` file After installation we start by importing `Realm` in the `AppDelegate` file and start configuring Realm as below: func initializeRealm() {         var config = Realm.Configuration(schemaVersion: 1,             migrationBlock: { _, oldSchemaVersion in                 if (oldSchemaVersion < 0) {                     // Nothing to do!                 }         })         config.fileURL = config.fileURL?.deletingLastPathComponent().appendingPathComponent("susi.realm")         Realm.Configuration.defaultConfiguration = config } Next, let’s head over to creating a few models which will be used to save the data to the DB as well as help retrieving that data so that it can be easily used. Since Susi server has a number of action types, we will cover some of the action types, their model and how they are used to store and retrieve data. Below are the currently available data types, that the server supports. enum ActionType: String { case answer case websearch case rss case table case map case anchor } Let’s start with the creation of the base model called `Message`. To make it a RealmObject, we import `RealmSwift` and inherit from `Object` class Message: Object { dynamic var queryDate = NSDate() dynamic var answerDate = NSDate() dynamic var message: String = "" dynamic var fromUser = true dynamic var actionType = ActionType.answer.rawValue dynamic var answerData: AnswerAction? dynamic var mapData: MapAction? dynamic var anchorData: AnchorAction? } Let’s study these properties of the message one by one. `queryDate`: saves the date-time the query was made `answerDate`: saves the date-time the query response was received `message`: stores the query/message that was sent to the server `fromUser`: a boolean which keeps track who created the message `actionType`: stores the action type `answerData`, `rssData`, `mapData`, `anchorData` are the data objects that actually store the respective action’s data To initialize this object, we need to create a method that takes input the data received from the server. // saves…

Continue ReadingSave Chat Messages using Realm in SUSI iOS
Read more about the article Custom UI Implementation for Web Search and RSS actions in SUSI iOS Using Kingfisher for Image Caching

Custom UI Implementation for Web Search and RSS actions in SUSI iOS Using Kingfisher for Image Caching

The SUSI Server is an AI powered server which is capable of responding to intelligent answers based on user’s queries. The queries to the susi server are obtained either as a websearch using the application or as an RSS feed. Two of the actions are websearch and RSS. These actions as the name suggests respond to queries based on search results from the web which are rendered in the clients. In order to use use these action types and display them in the SUSI iOS client, we need to first parse the actions looking for these action types and then creating a custom UI for them to display them. To start with, we need to make send the query to the server to receive an intelligent response from the server. This response is parsed into different action types supported by the server and saved into relevant objects. Here, we check the action types by looping through the answers array containing the actions and based on that, we save the data for that action. if type == ActionType.rss.rawValue {    message.actionType = ActionType.rss.rawValue    message.rssData = RSSAction(data: data, actionObject: action) } else if type == ActionType.websearch.rawValue {    message.actionType = ActionType.websearch.rawValue    message.message = action[Client.ChatKeys.Query] as? String ?? "" } Here, we parsed the data response from the server and looked for the rss and websearch action type followed by which we saved the data we received from the server for each of the action types in their own objects. Next, when a message object is created, we insert it into the dataSource item by appending it and use the `insertItems(at: [IndexPath])` method of collection view to insert them into the views at a particular index. Before adding them, we need to create a Custom UI for them. This UI will consist of a Collection View which is scrollable in the horizontal direction inside a CollectionView Cell. To start with this, we create a new class called `WebsearchCollectionView` which will be a `UIView` consisting of a `UICollectionView`. We start by adding a collection view into the UIView inside the `init` method by overriding it. Declare a collection view using flow layout and scroll direction set to `horizontal`. Also, hide the scroll indicators and assign the delegate and datasource to `self`. Now to populate this collection view, we need to specify the number of items that will show up. For this, we make use of the `message` variable declared. We use the `websearchData` in case of websearch action and `rssData` otherwise. Now to specify the number of cells, we use the below method which returns the number of rss or websearch action objects and defaults to 0 such cells. func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {     if let rssData = message?.rssData {         return rssData.count     } else if let webData = message?.websearchData {         return webData.count     }     return 0 } We display the title, description and image for each object for which we need to create a UI for the cells. Let’s start by creating…

Continue ReadingCustom UI Implementation for Web Search and RSS actions in SUSI iOS Using Kingfisher for Image Caching
Read more about the article Hotword Recognition in SUSI iOS

Hotword Recognition in SUSI iOS

Hot word recognition is a feature by which a specific action can be performed each time a specific word is spoken. There is a service called Snowboy which helps us achieve this for various clients (for ex: iOS, Android, Raspberry pi, etc.). It is basically a DNN based hotword recognition toolkit. In this blog, we will learn how to integrate the snowboy hotword detection wrapper in the SUSI iOS client. This service can be used in any open source project but for using it commercially, a commercial license needs to be obtained. Following are the files that need to be added to the project which are provided by the service itself: snowboy-detect.h libsnowboy-detect.a and a trained model file which can be created using their online service: snowboy.kitt.ai. For the sake of this blog, we will be using the hotword “Susi”, the model file can be found here. The way how snowboy works is that speech is recorded for a few seconds and this data is detected with an already trained model by a specific hotword, now if snowboy returns a 1 means word has been successfully detected else wasn’t. We start with creation of a wrapper class in Objective-C which can be found wrapper and the bridging header in case this needs to be added to a Swift project. The wrapper contains methods for setting sensitivity, audio gain and running the detection using the buffer. It is a wrapper class built on top of the snowboy-detect.h header file. Let’s initialize the service and run it. Below are the steps followed to enable hotword recognition and print out whether it successfully detected the hotword or not: Create a ViewController class with extensions AVAudioRecorderDelegate AVAudioPlayerDelegate since we will be recording speech. Import AVFoundation Create a basic layout containing a label which detects whether hotword detected or not and create corresponding `IBOutlet` in the ViewController and a button to trigger the start and stop of recognition. Create the following variables: let WAKE_WORD = "Susi" // hotword used let RESOURCE = Bundle.main.path(forResource: "common", ofType: "res") let MODEL = Bundle.main.path(forResource: "susi", ofType: "umdl") //path where the model file is stored var wrapper: SnowboyWrapper! = nil // wrapper instance for running detection var audioRecorder: AVAudioRecorder! // audio recorder instance var audioPlayer: AVAudioPlayer! var soundFileURL: URL! //stores the URL of the temp reording file var timer: Timer! //timer to fire a function after an interval var isStarted = false // variable to check if audio recorder already started In `viewDidLoad` initialize the wrapper and set sensitivity and audio gain. Recognition best happens when sensitivity is set to `0.5` and audio gain is set to `1.0` according to the docs. override func viewDidLoad() { super.viewDidLoad() wrapper = SnowboyWrapper(resources: RESOURCE, modelStr: MODEL) wrapper.setSensitivity("0.5") wrapper.setAudioGain(1.0) } Create an `IBAction` for the button to start recognition. This action will be used to start or stop the recording in which the action toggles based on the `isStarted` variable. When true, recording is stopped and the timer invalidated else a timer is started…

Continue ReadingHotword Recognition in SUSI iOS