2k17

Read more about the article Implementing Skill Listing in SUSI iOS

Implementing Skill Listing in SUSI iOS

Skills are basically a set of rules which respond to the user’s queries through any client app. All the skills are defined in the SUSI Skill Data repo where the user’s queries are matched with the already present skills and the server responds accordingly. Apps like Alexa, Google Assistant have an interface to view skills with a set of sample queries that can be used therefore, we are adding the same skill display UI in the SUSI iOS app. Implementation All the skills are arranged into categories or groups, so we first need to fetch all those groups followed by fetching skills from each group. For listing all the groups, we use the endpoint below: http://api.susi.ai/cms/getGroups.json Which returns all the groups in the groups object like below: { "session": {"identity": { "type": "host", "name": "67.214.191.117", "anonymous": true }}, "accepted": true, "groups": [ "Social", "News", "Food and Drink", "Travel and Transportation", "Connected Car", "Movies and TV", "Problem Solving", "Knowledge", "Business and Finance", "Productivity", "Games, Trivia and Accessories", "Lifestyle", "Health and Fitness", "Music and Audio", "Shopping", "Communication", "Novelty and Humour", "Utilities", "Sports", "Weather" ], "message": "Success: Fetched group list" } After the groups have been fetched, we need to get the skills for each group. Here, we use the endpoint below: http://api.susi.ai/cms/getSkillList.json?group=GROUP_NAME Since, we have a number of groups, we need to make the above API call as many times as the group count is. A sample call would look like: http://api.susi.ai/cms/getSkillList.json?group=News which would fetch all the skills in the News group. { "accepted": true, "model": "general", "group": "News", "language": "en", "skills": {"news": { "image": "images/news.png", "author_url": "https://github.com/AliAyub007", "examples": [ "News", "latest news", "most viewed articles in science today", "most viewed articles in science in the last week", "most viewed articles in science in the last month", "most shared articles in science today", "can you tell me last week's most shared articles in science", "do you know most shared articles in arts in the last month", "most emailed articles today in arts", "most emailed articles in science in the last week", "can you tell me most emailed articles in science in the last month", "articles in science", "show me articles", "most emailed articles", "most shared articles", "tell me news in tech world" ], "developer_privacy_policy": null, "author": "Ali Ayub Khan", "skill_name": "NEWS", "dynamic_content": true, "terms_of_use": null, "descriptions": "A skill to give news.", "skill_rating": null }}, "message": "Success: Fetched skill list", "session": {"identity": { "type": "host", "name": "23.94.137.239", "anonymous": true }} } Each such json object gives us the following values: Model Group Language Image Path Author’s URL Author’s Name A list of sample queries Skill Name Licence and terms of use Rating Description Implementation in SUSI iOS The UI for listing skills is a little complex as it consists of a UITableView where each UITableViewCell consists of a UILabel(group name) and a UICollectionVIew (horizontal scroll). Let’s see the step by step process to implement the skill listing. First, we need to fetch all the groups available using the endpoint above. // get all…

Continue ReadingImplementing Skill Listing in SUSI iOS
Read more about the article Settings Controller UI using Static Table View

Settings Controller UI using Static Table View

Dynamic Table Views are used at places where there may be any kind of reusability of cells. This means that there would exist cells that would have the same UI elements but would differ in the content being displayed. Initially the Settings Controller was built using UICollectionViewController which is completely dynamic but later I realized that the cells will remain static every time so there is no use of dynamic cells to display the UI hence, I switched to static table view cells. Using Static Table View is very easy. In this blog, I will explain how the implementation of the same was achieved in SUSI iOS app. Let’s start by dragging and dropping a UITableViewController into the storyboard file. The initial configuration of the UITableView has content as Dynamic Prototypes but we need Static cells so we choose them and make the sections count to 5 to suit our need. Also, to make the UI better, we choose the style as Grouped. Now for each section, we have the control of statically adding UI elements so, we add all the settings with their corresponding section headers and obtain the following UI.         After creating this UI, we can refer any UI element independently be it in any of the cells. So here we create references to each of the UISlider and UISwitch so that we can trigger an action whenever the value of anyone of them changes to get their present state. To create an action, simply create a function and add `@IBAction` in front so that they can be linked with the UI elements in the storyboard and then click and drag the circle next to the function to UI element it needs to be added. After successful linking, hovering over the same circle would reveal all the UI elements which trigger that function. Below is a method with the @IBAction identifier indicating it can be linked with the UI elements in the storyboard. This method is executed whenever any slider or switch value changes, which then updates the UserDefaults value as well sends an API request to update the setting for the user on the server. @IBAction func settingChanged(sender: AnyObject?) {         var params = [String: AnyObject]()         var key: String = ""         if let senderTag = sender?.tag {             if senderTag == 0 {                 key = ControllerConstants.UserDefaultsKeys.enterToSend             } else if senderTag == 1 {                 key = ControllerConstants.UserDefaultsKeys.micInput             } else if senderTag == 2 {                 key = ControllerConstants.UserDefaultsKeys.hotwordEnabled             } else if senderTag == 3 {                 key = ControllerConstants.UserDefaultsKeys.speechOutput             } else if senderTag == 4 {                 key = ControllerConstants.UserDefaultsKeys.speechOutputAlwaysOn             } else if senderTag == 5 {                 key = ControllerConstants.UserDefaultsKeys.speechRate             } else if senderTag == 6 {                 key = ControllerConstants.UserDefaultsKeys.speechPitch             }             if let slider = sender as? UISlider {                 UserDefaults.standard.set(slider.value, forKey: key)             } else {                 UserDefaults.standard.set(!UserDefaults.standard.bool(forKey: key), forKey: key)             }             params[ControllerConstants.key] = key as AnyObject…

Continue ReadingSettings Controller UI using Static Table View
Read more about the article Reset Password Functionality in SUSI iOS

Reset Password Functionality in SUSI iOS

Reset Password as the name suggests is one of the features in the SUSI iOS app which allows a user to change his/her password when they are logged in. This feature was added because a user would want to change his password sometimes to prevent unauthorized access or make his account security stronger. We can find many popular apps online such as Facebook, Gmail, which allow the user to reset their password. The way this is done is pretty simple and all we need from the user is his current and the new password he/she wants to set. In this blog post, I am going to explain step by step how this is implemented in the iOS client. Implementation The option to Reset Password is provided to the user under the Settings Controller. On selecting the row, the user is presented with another view which asks the user for his/her current password, new password, and another field to confirm the newly entered password. First, the user needs to provide his current password followed by the new password. The user’s current password is required just to authenticate that the account’s owner is requesting the password change. The new password field is followed by another field called confirm password just to make sure there isn’t any typo. Now when the field is filled, the user clicks the `Reset password` button at the bottom. What happens here is, first, the fields are validated to ensure the correct length of the passwords followed by an API request to update the same. The endpoint for the same is as below: http://api.susi.ai/aaa/changepassword.json?changepassword=user_email&password=current _pass&newpassword=new_pass&access_token=user_access_token This endpoint requires 3 things: Current Password New Password User’s email Access Token obtained at the time of login func validatePassword() -> [Bool:String] {         if let newPassword = newPasswordField.text,             let confirmPassword = confirmPasswordField.text {             if newPassword.characters.count > 5 {                 if newPassword == confirmPassword {                     return [true: ""]                 } else {                     return [false: ControllerConstants.passwordDoNotMatch]                 }             } else {                 return [false: ControllerConstants.passwordLengthShort]             }         }         return [false: Client.ResponseMessages.ServerError]     } Initially, we were not saving the user’s email, so we added the user’s email to the User’s object which is saved at the time of login. if var userData = results { userData[Client.UserKeys.EmailOfAccount] = user.email UserDefaults.standard.set(userData, forKey: ControllerConstants.UserDefaultsKeys.user) self.saveUserGlobally(user: currentUser) } At last, the API call is made which is implemented as below: let params = [ Client.UserKeys.AccessToken: user.accessToken, Client.UserKeys.EmailOfAccount: user.emailID, Client.UserKeys.Password: currentPasswordField.text ?? "", Client.UserKeys.NewPassword: newPasswordField.text ?? "" ] Client.sharedInstance.resetPassword(params as [String : AnyObject], { (_, message) in DispatchQueue.main.async { self.view.makeToast(message) self.setUIActive(active: false) } }) Below is the final UI. Reference Alamofire Docs for making network calls: https://github.com/Alamofire/Alamofire Tutorial on how to create static table view cells: https://www.appcoda.com/ios-static-table-view-storyboard/ Pull Request for API reference: https://github.com/fossasia/susi_server/pull/352

Continue ReadingReset Password Functionality in SUSI iOS
Read more about the article How to Implement Feedback System in SUSI iOS

How to Implement Feedback System in SUSI iOS

The SUSI iOS app provides responses for various queries but the response is always not accurate. To improve the response, we make use of the feedback system, which is the first step towards implementing Machine Learning on the SUSI Server. The way this works is that for every query, we present the user with an option to upvote or downvote the response and based on that a positive or negative feedback is saved on the server. In this blog, I will explain how this feedback system was implemented in the SUSI iOS app. Steps to implement: We start by adding the UI which is two buttons, one with a thumbs up and the other with a thumbs down image. textBubbleView.addSubview(thumbUpIcon) textBubbleView.addSubview(thumbDownIcon) textBubbleView.addConstraintsWithFormat(format: "H:[v0]-4-[v1(14)]-2-[v2(14)]-8-|", views: timeLabel, thumbUpIcon, thumbDownIcon) textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbUpIcon) textBubbleView.addConstraintsWithFormat(format: "V:[v0(14)]-2-|", views: thumbDownIcon) thumbUpIcon.isUserInteractionEnabled = true thumbDownIcon.isUserInteractionEnabled = true Here, we add the subviews and assign constraints so that these buttons align to the bottom right next to each other. Also, we enable the user interaction for these buttons. We know that the user can rate the response by pressing either of the buttons added above. To do that we make an API call to the endpoint below: BASE_URL+'/cms/rateSkill.json?'+'model='+model+'&group='+group+'&skill='+skill+’&language’+language+’&rating=’+rating Here, the BASE_URL is the url of the server, the other three params model, group, language and skill are retrieved by parsing the skill location parameter we get with the response. The rating is positive or negative based on which button was pressed by the user. The skill param in the response looks like this: skills: [ "/susi_skill_data/models/general/entertainment/en/quotes.txt" ] Let’s write the method that makes the API call and responds to the UI that it was successful. if let accepted = response[ControllerConstants.accepted] as? Bool { if accepted { completion(true, nil) return } completion(false, ResponseMessages.ServerError) return } Here after receiving a response from the server, we check if the `accepted` variable is true or not. Based on that, we pass `true` or `false` to the completion handler. Below the response we actually receive by making the request. { session: { identity: { type: "host", name: "23.105.140.146", anonymous: true } }, accepted: true, message: "Skill ratings updated" } Finally, let’s update the UI after the request has been successful. if sender == thumbUpIcon { thumbDownIcon.tintColor = UIColor(white: 0.1, alpha: 0.7) thumbUpIcon.isUserInteractionEnabled = false thumbDownIcon.isUserInteractionEnabled = true feedback = "positive" } else { thumbUpIcon.tintColor = UIColor(white: 0.1, alpha: 0.7) thumbDownIcon.isUserInteractionEnabled = false thumbUpIcon.isUserInteractionEnabled = true feedback = "negative" } sender.tintColor = UIColor.hexStringToUIColor(hex: "#2196F3") Here, we check the sender (the thumbs up or down button) and based on that pass the rating (positive or negative) and update the color of the button. Below is the app in action with the feedback system. Resources: Tutorial on Raywanderlich discusses the way custom collection view cells are created Another tutorial on creating custom cells Alamofire Docs: explains usage of Alamofire to make API calls Material.io icons: source of the icons used

Continue ReadingHow to Implement Feedback System in SUSI iOS
Read more about the article Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

In this blog post, I will be explaining how to encode and decode images and save them in UserDefaults so that the image persists even if it is removed from the Photos app. It happens a number of times that images are removed from the gallery by the users which results in the app loosing the image. So, to avoid this, we save the image by encoding it in a data object and save it inside UserDefaults. In SUSI iOS app we simply select an image from the image picker, encode it and save it in UserDefaults. To set the image, we simply fetch the image data from the UserDefaults and decode it to an image. There are two ways we can do the encoding and decoding process: Using Data object Using Base64 string For the scope of this tutorial, we will use the Data object. Implementation Steps To use the image picker, we need to add permissions to `Info.plist` file. <key>NSLocationWhenInUseUsageDescription</key> <string>Susi is requesting to get your current location</string> <key>NSPhotoLibraryUsageDescription</key> <string>Susi needs to request your gallery access to select wallpaper</string> Select image from gallery First, we present an alert which gives an option to select the image from the gallery. // Show wallpaper options to set wallpaper or clear wallpaper func showWallpaperOptions() { let imageDialog = UIAlertController(title: ControllerConstants.wallpaperOptionsTitle, message: nil, preferredStyle: UIAlertControllerStyle.alert) imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsPickAction, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) self.showImagePicker() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsNoWallpaperAction, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) self.removeWallpaperFromUserDefaults() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) })) self.present(imageDialog, animated: true, completion: nil) } Here, we create and UIAlertController with three options to select, one which presents the image picker controller, the second one removes the background wallpaper and the third dismisses the alert. Set the image as background view // Callback when image is selected from gallery func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) { dismiss(animated: true, completion: nil) let chosenImage = info[UIImagePickerControllerOriginalImage] as? UIImage if let image = chosenImage { setBackgroundImage(image: image) } } We use the `didFinishPickingMediaWithInfo` delegate method to set the image as background. First we get the image using the the `info` dictionary using the `UIImagePickerControllerOriginalImage` key. Save the image in UserDefaults (encoding) // Save image selected by user to user defaults func saveWallpaperInUserDefaults(image: UIImage!) {   let imageData = UIImageJPEGRepresentation(image!, 1.0)   let defaults = UserDefaults.standard   defaults.set(imageData, forKey: userDefaultsWallpaperKey) } We first convert the image to a data object using the `UIImageJPEGRepresentation` method followed by saving the data object in UserDefaults with the key `wallpaper`. Decode the data object back to UIImage  Now whenever we need to decode the image, we simply get the data object from the UserDefaults and use it to display the image. // Check if user defaults have an image data saved else return nil/Any func getWallpaperFromUserDefaults() -> Any? {   let defaults = UserDefaults.standard   return defaults.object(forKey: userDefaultsWallpaperKey) } Below is the output when an image is…

Continue ReadingEncoding and Decoding Images as Data in UserDefaults in SUSI iOS
Read more about the article Using The Dark and Light Theme in SUSI iOS

Using The Dark and Light Theme in SUSI iOS

SUSI being an AI for interactive chat bots, provides answers to the users in the intelligent way. So, to make the SUSI iOS app more user friendly, the option of switching between themes was introduced. This also enables the user switch between themes based on the environment around. Any user can switch between the light and dark themes easily from the settings. We start by declaring an enum called `theme` which contains two strings namely, dark and light. enum theme: String {     case light     case dark } We can update the color scheme based on the theme selected very easily by checking the currently active theme and based on that check, we update the color scheme. To check the currently active theme, we define a variable in the `AppDelegate` which holds the value. var activeTheme: String? Below is the way the color scheme of the LoginViewController is set. var activeTheme: String?func setupTheme() { let image = UIImage(named: ControllerConstants.susi)?.withRenderingMode(.alwaysTemplate) susiLogo.image = image susiLogo.tintColor = .white UIApplication.shared.statusBarStyle = .lightContent let activeTheme = AppDelegate().activeTheme if activeTheme == theme.light.rawValue { view.backgroundColor = UIColor.lightThemeBackground() } else if activeTheme == theme.dark.rawValue { view.backgroundColor = UIColor.darkThemeBackground() } } Here, we first get the image and set the rendering mode to `alwaysTemplate` so that we can change the tint color of the image. Next, we assign the image to the `IBOutlet` and change the tint color to `white`. We also change the status bar style to `lightContent`. Next, we check the active theme and change the view’s background color accordingly. For this method to execute, we call it inside, `viewDidLoad` so that the theme loads up as the view loads. Next, lets add this option of switching between themes inside the `SettingsViewController`. We add a cell with `titleLabel` as `Change Theme` and use the collectionView’s delegate method of `didSelect` to show an alert. This alert contains three options, Dark theme, Light Theme and Cancel. Let’s code that method which presents the alert. func themeToggleAlert() { let imageDialog = UIAlertController(title: ControllerConstants.toggleTheme, message: nil, preferredStyle: UIAlertControllerStyle.alert) imageDialog.addAction(UIAlertAction(title: theme.dark.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) AppDelegate().activeTheme = theme.dark.rawValue self.settingChanged(sender: self.imagePicker) self.setupTheme() })) imageDialog.addAction(UIAlertAction(title: theme.light.rawValue.capitalized, style: .default, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) AppDelegate().activeTheme = theme.light.rawValue self.settingChanged(sender: self.imagePicker) self.setupTheme() })) imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in imageDialog.dismiss(animated: true, completion: nil) })) self.present(imageDialog, animated: true, completion: nil) } Here, we assign the alert view’s title and add 3 actions and their respective completion handlers. If we see inside these completion handlers, we can notice that we first dismiss the alert followed by updating the activeTheme variable in AppDelegate and call the `settingChanged` function which updates the user’s settings on the server. Finally, we update the color scheme. Now, if we build and run the app and change the theme from the settings, we will notice that on returning to the chat view, the color scheme is not updated. The reason here is that we are setting up the theme on…

Continue ReadingUsing The Dark and Light Theme in SUSI iOS
Read more about the article How to Store and Retrieve User Settings from SUSI Server in SUSI iOS

How to Store and Retrieve User Settings from SUSI Server in SUSI iOS

Any user using the SUSI iOS client can set preferences like enabling or disabling the hot word recognition or enabling input from the microphone. These settings need to be stored, in order to be used across all platforms such as web, Android or iOS. Now, in order to store these settings and maintain a synchronization between all the clients, we make use of the SUSI server. The server provides an endpoint to retrieve these settings when the user logs in. First, we will focus on storing settings on the server followed by retrieving settings from the server. The endpoint to store settings is as follows: http://api.susi.ai/aaa/changeUserSettings.json?key=key&value=value&access_token=ACCESS_TOKEN This takes the key value pair for storing a settings and an access token to identify the user as parameters in the GET request. Let’s start by creating the method that takes input the params, calls the API to store settings and returns a status specifying if the executed successfully or not. let url = getApiUrl(UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.ipAddress) as! String, Methods.UserSettings)         _ = makeRequest(url, .get, [:], parameters: params, completion: { (results, message) in             if let _ = message {                 completion(false, ResponseMessages.ServerError)             } else if let results = results {                 guard let response = results as? [String : AnyObject] else {                     completion(false, ResponseMessages.ServerError)                     return                 }                 if let accepted = response[ControllerConstants.accepted] as? Bool, let message = response[Client.UserKeys.Message] as? String {                     if accepted {                         completion(true, message)                         return                     }                     completion(false, message)                     return                 }             }         }) Let’s understand this function line by line. First we generate the URL by supplying the server address and the method. Then, we pass the URL and the params in the `makeRequest` method which has a completion handler returning a results object and an error object. Inside the completion handler, check for any error, if it exists mark the request completed with an error else check for the results object to be a dictionary and a key `accepted`, if this key is `true` our request executed successfully and we mark the request to be executed successfully and finally return the method. After making this method, it needs to be called in the view controller, we do so by the following code. Client.sharedInstance.changeUserSettings(params) { (_, message) in DispatchQueue.global().async { self.view.makeToast(message) } } The code above takes input params containing the user token and key-value pair for the setting that needs to be stored. This request runs on a background thread and displays a toast message with the result of the request. Now that the settings have been stored on the server, we need to retrieve these settings every time the user logs in the app. Below is the endpoint for the same: http://api.susi.ai/aaa/listUserSettings.json?access_token=ACCESS_TOKEN This endpoint accepts the user token which is generated when the user logs in which is used to uniquely identify the user and his/her settings are returned. Let’s create the method that would call this endpoint and parse and save the settings data in the…

Continue ReadingHow to Store and Retrieve User Settings from SUSI Server in SUSI iOS
Read more about the article Save Chat Messages using Realm in SUSI iOS

Save Chat Messages using Realm in SUSI iOS

Fetching data from the server each time causes a network load which makes the app depend on the server and the network in order to display data. We use an offline database to store chat messages so that we can show messages to the user even if network is not present which makes the user experience better. Realm is used as a data storage solution due to its ease of usability and also, since it’s faster and more efficient to use. So in order to save messages received from the server locally in a database in SUSI iOS, we are using Realm and the reasons for using the same are mentioned below. The major upsides of Realm are: It’s absolutely free of charge, Fast, and easy to use. Unlimited use. Work on its own persistence engine for speed and performance Below are the steps to install and use Realm in the iOS Client: Installation: Install Cocoapods Run `pod repo update` in the root folder In your Podfile, add use_frameworks! and pod 'RealmSwift' to your main and test targets. From the command line run `pod install` Use the `.xcworkspace` file generated by Cocoapods in the project folder alongside `.xcodeproj` file After installation we start by importing `Realm` in the `AppDelegate` file and start configuring Realm as below: func initializeRealm() {         var config = Realm.Configuration(schemaVersion: 1,             migrationBlock: { _, oldSchemaVersion in                 if (oldSchemaVersion < 0) {                     // Nothing to do!                 }         })         config.fileURL = config.fileURL?.deletingLastPathComponent().appendingPathComponent("susi.realm")         Realm.Configuration.defaultConfiguration = config } Next, let’s head over to creating a few models which will be used to save the data to the DB as well as help retrieving that data so that it can be easily used. Since Susi server has a number of action types, we will cover some of the action types, their model and how they are used to store and retrieve data. Below are the currently available data types, that the server supports. enum ActionType: String { case answer case websearch case rss case table case map case anchor } Let’s start with the creation of the base model called `Message`. To make it a RealmObject, we import `RealmSwift` and inherit from `Object` class Message: Object { dynamic var queryDate = NSDate() dynamic var answerDate = NSDate() dynamic var message: String = "" dynamic var fromUser = true dynamic var actionType = ActionType.answer.rawValue dynamic var answerData: AnswerAction? dynamic var mapData: MapAction? dynamic var anchorData: AnchorAction? } Let’s study these properties of the message one by one. `queryDate`: saves the date-time the query was made `answerDate`: saves the date-time the query response was received `message`: stores the query/message that was sent to the server `fromUser`: a boolean which keeps track who created the message `actionType`: stores the action type `answerData`, `rssData`, `mapData`, `anchorData` are the data objects that actually store the respective action’s data To initialize this object, we need to create a method that takes input the data received from the server. // saves…

Continue ReadingSave Chat Messages using Realm in SUSI iOS
Read more about the article Hotword Recognition in SUSI iOS

Hotword Recognition in SUSI iOS

Hot word recognition is a feature by which a specific action can be performed each time a specific word is spoken. There is a service called Snowboy which helps us achieve this for various clients (for ex: iOS, Android, Raspberry pi, etc.). It is basically a DNN based hotword recognition toolkit. In this blog, we will learn how to integrate the snowboy hotword detection wrapper in the SUSI iOS client. This service can be used in any open source project but for using it commercially, a commercial license needs to be obtained. Following are the files that need to be added to the project which are provided by the service itself: snowboy-detect.h libsnowboy-detect.a and a trained model file which can be created using their online service: snowboy.kitt.ai. For the sake of this blog, we will be using the hotword “Susi”, the model file can be found here. The way how snowboy works is that speech is recorded for a few seconds and this data is detected with an already trained model by a specific hotword, now if snowboy returns a 1 means word has been successfully detected else wasn’t. We start with creation of a wrapper class in Objective-C which can be found wrapper and the bridging header in case this needs to be added to a Swift project. The wrapper contains methods for setting sensitivity, audio gain and running the detection using the buffer. It is a wrapper class built on top of the snowboy-detect.h header file. Let’s initialize the service and run it. Below are the steps followed to enable hotword recognition and print out whether it successfully detected the hotword or not: Create a ViewController class with extensions AVAudioRecorderDelegate AVAudioPlayerDelegate since we will be recording speech. Import AVFoundation Create a basic layout containing a label which detects whether hotword detected or not and create corresponding `IBOutlet` in the ViewController and a button to trigger the start and stop of recognition. Create the following variables: let WAKE_WORD = "Susi" // hotword used let RESOURCE = Bundle.main.path(forResource: "common", ofType: "res") let MODEL = Bundle.main.path(forResource: "susi", ofType: "umdl") //path where the model file is stored var wrapper: SnowboyWrapper! = nil // wrapper instance for running detection var audioRecorder: AVAudioRecorder! // audio recorder instance var audioPlayer: AVAudioPlayer! var soundFileURL: URL! //stores the URL of the temp reording file var timer: Timer! //timer to fire a function after an interval var isStarted = false // variable to check if audio recorder already started In `viewDidLoad` initialize the wrapper and set sensitivity and audio gain. Recognition best happens when sensitivity is set to `0.5` and audio gain is set to `1.0` according to the docs. override func viewDidLoad() { super.viewDidLoad() wrapper = SnowboyWrapper(resources: RESOURCE, modelStr: MODEL) wrapper.setSensitivity("0.5") wrapper.setAudioGain(1.0) } Create an `IBAction` for the button to start recognition. This action will be used to start or stop the recording in which the action toggles based on the `isStarted` variable. When true, recording is stopped and the timer invalidated else a timer is started…

Continue ReadingHotword Recognition in SUSI iOS
Read more about the article Using CoreLocation in SUSI iOS

Using CoreLocation in SUSI iOS

The SUSI Server responds with intelligent answers to the user’s queries. To make these answers better, the server makes use of the user’s location which is sent as a parameter to the query request each time. To implement this feature in the SUSI iOS client, we use the CoreLocation framework provided by Apple which helps us to get the user’s location coordinates and add them as a parameter to each request made. In order to start with using the CoreLocation framework, we first import it inside the view controller. import CoreLocation Now, we create a variable of type CLLocationManager which will help us to use the actual functionality. // Location Manager var locationManager = CLLocationManager() The location manager has some delegate methods which give an option to get the maximum accuracy for a user’s location.  To set that, we need the controller to conform to the CLLocationManagerDelegate, so we create an extension of the view controller conforming to this. extension MainViewController: CLLocationManagerDelegate {    // use functionality } Next, we set the manager delegate. locationManager.delegate = self And create a method to ask for using the user’s location and set the delegate properties. func configureLocationManager() {        locationManager.delegate = self        if CLLocationManager.authorizationStatus() == .notDetermined || CLLocationManager.authorizationStatus() == .denied {            self.locationManager.requestWhenInUseAuthorization()        }        locationManager.distanceFilter = kCLDistanceFilterNone        locationManager.desiredAccuracy = kCLLocationAccuracyBest } Here, we ask for the user location if it was previously denied or is not yet determined and following that, we set the `distanceFilter` as kCLDistanceFilterNone  and `desiredAccuray` as kCLLocationAccuracyBest.. Finally, we are left with starting to update the location which we do by: locationManager.startUpdatingLocation() We call this method inside viewDidLoad to start updation of the location when the view first loads. The complete extension looks like below: extension MainViewController: CLLocationManagerDelegate {    // Configures Location Manager    func configureLocationManager() {        locationManager.delegate = self        if CLLocationManager.authorizationStatus() == .notDetermined || CLLocationManager.authorizationStatus() == .denied {            self.locationManager.requestWhenInUseAuthorization()        }        locationManager.distanceFilter = kCLDistanceFilterNone        locationManager.desiredAccuracy = kCLLocationAccuracyBest        locationManager.startUpdatingLocation()    } } Now, it’s very easy to use the location manager and get the coordinates and add it to the params for each request. if let location = locationManager.location {    params[Client.ChatKeys.Latitude] = location.coordinate.latitude as AnyObject    params[Client.ChatKeys.Longitude] = location.coordinate.longitude as AnyObject } Now the params which is a dictionary object is added to each request made so that the user get’s the most accurate results for each query he makes. References: https://developer.apple.com/documentation/corelocation/cllocationmanager https://developer.apple.com/documentation/corelocation/cllocationmanagerdelegate https://www.raywenderlich.com/136165/core-location-geofencing-tutorial

Continue ReadingUsing CoreLocation in SUSI iOS