Susi

Read more about the article Displaying Skills Feedback on SUSI iOS

Displaying Skills Feedback on SUSI iOS

SUSI allows the user to rate the SUSI Skills with the five-star rating system. SUSI offer a good feedback system where the user can post feedback to any skill by using iOS, Android, and Web clients. In Skill Detail, there is a skill feedback text field where the user can write feedback about SUSI Skill. We display the users posted feedbacks on Skill Detail screen. In this post, we will see how the displaying skills feedback feature implemented on SUSI iOS. Implementation - We are displaying three feedback on Skill Detail screen, to see all feedback, there is a “See All Review” option, by clicking user is directed to a new screen where he/she can see all feedback related to particular skill. We use the endpoint below for getting skill feedback from server side – https://api.susi.ai/cms/getSkillFeedback.json With the following params: Model Group Language Skill Name The API endpoint above return the all the feedback array related to particular susi skill. We store feedbacks in an array of Feedback object, which holds three value: Feedback String - Feedback string posted by the user Email - Email address of feedback poster user Time Stamp - Time of posting feedback class Feedback: NSObject { var feedbackString: String = "" var email: String = "" var timeStamp: String = "" ... } To display feedbacks, we are using UITableView with two prototype cells, one for feedbacks and one for “See All Review” option. There can be different cases eg. when the total number of feedback for skill is less than three or three. When the feedback count is three or less than three, there is no need to show “See All Review” option. Also, tableView height is different for different feedback count. For varying tableView height, we have created an outlet for tableView height constraints and vary accordingly. @IBOutlet weak var feedbackTableHeighConstraint: NSLayoutConstraint! Now, let’s see how the number of cells, height for cells and different cells presented according to feedback count with UITableViewDelegate and UITableViewDataSource methods. Handling number of tableView rows - func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int { guard let feedbacks = feedbacks, feedbacks.count > 0 else { feedbackTableHeighConstraint.constant = 0.0 return 0 } if feedbacks.count < 4 { feedbackTableHeighConstraint.constant = CGFloat(72 * feedbacks.count) return feedbacks.count } else { feedbackTableHeighConstraint.constant = 260.0 return 4 } } Where feedbacks is the array of Feedback object which holds the feedbacks we are getting from the server side for a skill. var feedbacks: [Feedback]? In the above method, we see that how we are handling the number of cells case. Now let’s see how to handle which cells to be present on basis of the number of cells case - func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell { guard let feedbacks = feedbacks, feedbacks.count > 0 else { feedbackTableHeighConstraint.constant = 0.0 return UITableViewCell() } if feedbacks.count < 4 { if let cell = tableView.dequeueReusableCell(withIdentifier: "feedbackDisplayCell", for: indexPath) as? FeedbackDisplayCell { cell.feedback = feedbacks[indexPath.row - Int(indexPath.row/2)] return cell } } else…

Continue ReadingDisplaying Skills Feedback on SUSI iOS
Read more about the article Allowing user to submit ratings for skills in SUSI iOS

Allowing user to submit ratings for skills in SUSI iOS

Rating is a great way to get feedback from the user. Generally, the 5-Star rating system used to get feedback. The Five-Star Quality Rating System was developed as an easy-to-understand rating system for users. In SUSI iOS we are using the Star Rating field to get feedback of SUSI skills. We enable the user to rate the skills from one to five star. In the rating submission, we get the number of stars user picked. The stars help show if you would recommend the skill to others. The values start at 1 (being the lowest) and go to 5 (being the highest), as seen below - Server-Side Implementation - fiveStarRatings API is using to submit users rating. Whenever the user taps the star fiveStarRatings being called: func submitRating(_ params: [String: AnyObject], _ completion: @escaping(_ updatedRatings: Ratings?, _ success: Bool, _ error: String?) -> Void) { let url = getApiUrl(UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.ipAddress) as! String, Methods.fiveStarRateSkill) _ = makeRequest(url, .get, [:], parameters: params, completion: { (results, message) in // handle request .... }) } The following params we send in the request: Model Group Language Skill Stars Access Token After successfully rating submission we get updated ratings for each star of the particular skill. The following response we get from the server after successfully submitted rating: { "ratings": { "one_star": 0, "four_star": 1, "five_star": 0, "total_star": 1, "three_star": 0, "avg_star": 4, "two_star": 0 }, "session": {"identity": { "type": "email", "name": "abc@a.com", "anonymous": false }}, "accepted": true, "message": "Skill ratings updated" } We use ratings object from fiveStarRatings API to update the ratings displayed on charts and labels and also, we use ratings object to update Skill model which we passed from SkillListingController to SkillDetailController so the user can see updated rating chart when clicking back to skill. func updateSkill(with ratings: Ratings) {..} If the user already submitted ratings for a skill, we are using getRatingByUser API with same params as in fiveStarRatings except ratings to get already submitted user rating and we display that ratings as initial ratings in UI. Implementation of Submit Rating UI - RatingView is behind the submit rating UI. FloatRatingViewDelegate protocol is implemented to get ratings is updating or get updated. @objc public protocol FloatRatingViewDelegate { /// Returns the rating value when touch events end @objc optional func floatRatingView(_ ratingView: RatingView, didUpdate rating: Double) /// Returns the rating value as the user pans @objc optional func floatRatingView(_ ratingView: RatingView, isUpdating rating: Double) } Rating updated on rating display chart: Now see, how we handle the case when Skill is not rated before and the user first time rate the skill. There is a label that shows the “Skill not rated yet” when a skill is not rated. When the user rates the skill, the label is hidden and chart bar and the label is shown. if self.ratingsBackViewHeightConstraint.constant == 0 { self.ratingsBackViewHeightConstraint.constant = 128.0 self.contentType.topAnchor.constraint(equalTo: self.ratingBackView.bottomAnchor, constant: 16).isActive = true self.ratingsBackStackView.isHidden = false self.topAvgRatingStackView.isHidden = false self.notRatedLabel.isHidden = true }   Resources - Swift Protocol: https://docs.swift.org/swift-book/LanguageGuide/Protocols.html SUSI Skills: https://skills.susi.ai/ SUSI Server Link:…

Continue ReadingAllowing user to submit ratings for skills in SUSI iOS
Read more about the article Change Text-to-Speech Voice Language of SUSI in SUSI iOS

Change Text-to-Speech Voice Language of SUSI in SUSI iOS

SUSI iOS app now enables the user to change the text-to-speech voice language within the app. Now, the user can select any language of their choice from the list of 37 languages list. To change the text-to-speech voice language, go to, Settings > Change SUSI’s Voice, choose the language of your choice. Let see here how this feature implemented. Apple’s AVFoundation API is used to implement the text-to-speech feature in SUSI iOS. AVFoundation API offers 37 voice languages which can be used for text-to-speech voice accent. AVFoundation’s AVSpeechSynthesisVoice API can be used to select a voice appropriate to the language of the text to be spoken or to select a voice exhibiting a particular local variant of that language (such as Australian or South African English). To print the list of all languages offered by AVFoundation: import AVFoundation print(AVSpeechSynthesisVoice.speechVoices()) Or the complete list of supported languages can be found at Languages Supported by VoiceOver. When the user clicks Change SUSI’s voice in settings, a screen is presented with the list of available languages with the language code. Dictionary holds the list of available languages with language name and language code and used as Data Source for tableView. var voiceLanguagesList: [Dictionary<String, String>] = [] When user choose the language and click on done, we store language chosen by user in UserDefaults: UserDefaults.standard.set(voiceLanguagesList[selectedVoiceLanguage][ControllerConstants.ChooseLanguage.languageCode], forKey: ControllerConstants.UserDefaultsKeys.languageCode) UserDefaults.standard.set(voiceLanguagesList[selectedVoiceLanguage][ControllerConstants.ChooseLanguage.languageName], forKey: ControllerConstants.UserDefaultsKeys.languageName) Language name with language code chosen by user displayed in settings so the user can know which language is currently being used for text-to-speech voice. To select a voice for use in speech, we obtain an AVSpeechSynthesisVoice instance using one of the methods in Finding Voices and then set it as the value of the voice property on an AVSpeechUtterance instance containing text to be spoken. Earlier stored language code in UserDefaults shared instance used for setting the text-to-speech language for AVSpeechSynthesisVoice. if let selectedLanguage = UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.languageCode) as? String { speechUtterance.voice = AVSpeechSynthesisVoice(language: selectedLanguage) } AVSpeechUtterance is responsible for a chunk of text to be spoken, along with parameters that affect its speech. Resources - UserDefaults: https://developer.apple.com/documentation/foundation/userdefaults AVSpeechSynthesisVoice: https://developer.apple.com/documentation/avfoundation/avspeechsynthesisvoice AVFoundation: https://developer.apple.com/av-foundation/ SUSI iOS Link: https://github.com/fossasia/susi_iOS

Continue ReadingChange Text-to-Speech Voice Language of SUSI in SUSI iOS
Read more about the article Skill Ratings Over Time

Skill Ratings Over Time

The SUSI SKill CMS provides an option to rate and review a skill. These feedbacks help the skill creators to improve the skills. Also, the ratings and reviews can be updated by the reviewer. But the CMS only provides the current rating of a skill. What if a user or a developer wants to see how that skill has performed over time? Are there any improvements in the skill or not? For that, we need the skill ratings over time ! Server side implementation Create a ratingsOverTime.json file to store the monthly average rating of the skills and make a JSONTray object for that in src/ai/susi/DAO.java file. The JSON file contains the timestamp for every month, the average ratings on a skill in that month and the total number of ratings in that month. public static JsonTray ratingsOverTime; Path ratingsOverTime_per = susi_skill_rating_dir.resolve("ratingsOverTime.json"); Path ratingsOverTime_vol = susi_skill_rating_dir.resolve("ratingsOverTime_session.json"); ratingsOverTime = new JsonTray(ratingsOverTime_per.toFile(), ratingsOverTime_vol.toFile(), 1000000); OS.protectPath(ratingsOverTime_per); OS.protectPath(ratingsOverTime_vol); Now whenever a user rates a skill, the data in ratingsOverTime.json needs to be updated. For this fetch the overall rating data of the current month. Multiply the average rating with the total number of ratings (count) of that month. sum = average_rating X number_of_ratings Then add the rating given by the current user to this sum and divide by count + 1 to again get the new average rating. Also increment the total number of ratings by 1. new_sum = sum + rating_by_user new_avg = new_sum/(count+1) number_of_ratings =  number_of_ratings + 1 float totalRating = skillRating * ratingCount; float newAvgRating = (totalRating + skill_stars)/(ratingCount + 1); ratingObject.put("rating", newAvgRating); ratingObject.put("count", ratingCount + 1); Now we have got the ratings over time stored in ratingsOverTime.json file. An API to access this data is also required. So create an API GetRatingOverTime.java returns the ratings over time of a particular skill. The API has the following attributes : Endpoint : /cms/getRatingsOverTime.json Minimum user role : anonymous Parameters : model, group, language and skill JSONArray skillRatings = languageName.getJSONArray(skill_name); result.put("skill_name", skill_name); result.put("ratings_over_time", skillRatings); return new ServiceResponse(result); It fetches the data corresponding to the skill from ratingsOverTime.json and returns it to the CMS. Add this API to SusiServer.java //Skill ratings over time GetRatingsOverTime.class References 5 Star Skill Rating System https://docs.google.com/document/d/1uzi3WoLV18L9gV5S1-rf4JlH4GC8AP28DRk3hpqNPpA/edit?usp=sharing

Continue ReadingSkill Ratings Over Time
Read more about the article Creating Feedback Logs for Analysis

Creating Feedback Logs for Analysis

The thumbs up and thumbs down feedback on the clients is meant for the improvement of the skills in SUSI.AI. So we need to scope the feedback system to a particular interaction rather than skill as a whole. The feedback logs can be used for various kinds of analysis and machine learning. Server side implementation Components of Feedback Log: User ID - For identification of a feedback given by a particular user. For consistency in data, the user should not be able to change the feedback over the same interaction. Interaction: User query SUSI Reply Client location - The response of a skill may not be interesting for the users of a particular country. That means the skill should give localised results. Skill path - The path on the server where the skill is stored. Create a feedbackLogs.json file to store the logs of feedback given by the user and make a JSONTray object for that in src/ai/susi/DAO.java file. The JSON file contains the above mentioned components. public static JsonTray feedbackLogs; Path feedbackLogs_per = susi_skill_rating_dir.resolve("feedbackLogs.json"); Path feedbackLogs_vol = susi_skill_rating_dir.resolve("feedbackLogs_session.json"); feedbackLogs = new JsonTray(feedbackLogs_per.toFile(), feedbackLogs_vol.toFile(), 1000000); OS.protectPath(feedbackLogs_per); OS.protectPath(feedbackLogs_vol); Create FeedbackLogService.java file that acts as an API to create the feedback logs. The API accepts the feedback data from the client and stores it into the json file using DAO object. The user should be logged in to give feedback on an interaction. So keep the minimum user role as USER to access the API. JSONObject feedbackLogObject = new JSONObject(); feedbackLogObject.put("timestamp", timestamp); feedbackLogObject.put("uuid", idvalue); feedbackLogObject.put("feedback", skill_rate); feedbackLogObject.put("user_query", user_query); feedbackLogObject.put("susi_reply", susi_reply); feedbackLogObject.put("country_name", country_name); feedbackLogObject.put("country_code", country_code); feedbackLogObject.put("skill_path", skill_path); The API is accessible at /cms/feedbackLog.json endpoint. Send feedback log from Web Client The feedback API should be called only if the user is logged in. When the user presses the feedback buttons fetch the required data for log (access token, user query, susi response, country and user feedback) and POST them on the feedbackLog.json API. let rateEndPoint = BASE_URL + '/cms/feedbackLog.json?model=' + skill.model + '&group=' + skill.group + '&language=' + skill.language + '&skill=' + skill.skill + '&rating=' + rating + '&access_token=' + accessToken + '&user_query=' + interaction.userQuery + '&susi_reply=' + interaction.susiReply + '&country_name=' + country.countryName + '&country_code=' + country.countryCode ; $.ajax({ url: rateEndPoint, success: function(response) { console.log('Skill rated successfully'); } }) References FreeGeoIP- http://freegeoip.net/json/

Continue ReadingCreating Feedback Logs for Analysis
Read more about the article STOP action implementation in SUSI iOS

STOP action implementation in SUSI iOS

You may have experienced, you can stop Google home or Amazon Alexa during the ongoing task. The same feature is available for SUSI too. Now, SUSI can respond to ‘stop’ action and stop ongoing tasks (e.g. SUSI is narrating a story and if the user says STOP, it stops narrating the story). 'stop' action is introduced to enable the user to make SUSI stop anything it's doing. Video demonstration of how stop action work on SUSI iOS App can be found here. Stop action is implemented on SUSI iOS, Web chat, and Android. Here we will see how it is implemented in SUSI iOS. When you ask SUSI to stop, you get following actions object from server side: "actions": [{"type": "stop"}] Full JSON response can be found here. When SUSI respond with ‘stop’ action, we create a new action type ‘stop’ and assign `Message` object `actionType` to ‘stop’. Adding ‘stop’ to action type: enum ActionType: String { ... // other action types case stop } Assigning to the message object: if type == ActionType.stop.rawValue { message.actionType = ActionType.stop.rawValue message.message = ControllerConstants.stopMessage message.answerData = AnswerAction(action: action) } A new collectionView cell is created to respond user with “stoped” text. Registering the stopCell: collectionView?.register(StopCell.self, forCellWithReuseIdentifier: ControllerConstants.stopCell) Add cell to the chat screen: if message.actionType == ActionType.stop.rawValue { if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.stopCell, for: indexPath) as? StopCell { cell.message = message let message = ControllerConstants.stopMessage let estimatedFrame = self.estimatedFrame(message: message) cell.setupCell(estimatedFrame, view.frame) return cell } } AVFoundation’s AVSpeechSynthesizer API is used to stop the action: func stopSpeakAction() { speechSynthesizer.stopSpeaking(at: AVSpeechBoundary.immediate) } This method immediately stops the speak action. Final Output: Resources -  About SUSI: https://chat.susi.ai/overview JSON response for ‘stop’ action: https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=susi+stop AVSpeechSynthesisVoice: https://developer.apple.com/documentation/avfoundation/avspeechsynthesisvoice AVFoundation: https://developer.apple.com/av-foundation/ SUSI iOS Link: https://github.com/fossasia/susi_iOS SUSI Android Link: https://github.com/fossasia/susi_android SUSI Web Chat Link: https://chat.susi.ai/

Continue ReadingSTOP action implementation in SUSI iOS
Read more about the article Displaying skill rating for each skill on skill page of SUSI SKILL CMS

Displaying skill rating for each skill on skill page of SUSI SKILL CMS

SUSI exhibits several skills which are managed by the SUSI Skill CMS, it essentially is a client which allows users to create/update skills conveniently since for each skill it is important to have the functionality of rating system so developers can get to know which skills are performing better than the rest and consequently improve them, thus a skill rating system which allows the users to give positive or negative feedback for each skill is implemented on the server. Fetching skill_rating from the server Fetch skill data for which ratings are to be displayed through ajax calls API Endpoint - /cms/getSkillMetadata.json?… Parse the received metadata object to get positive and negative ratings for that skill if(skillData.skill_rating) { let positive_rating = skillData.skill_rating.positive; let negative_rating = skillData.skill_rating.negative; } Sample API response { "skill_metadata": { "model": "general", "group": "Knowledge", "language": "en", "developer_privacy_policy": null, "descriptions": "Want to know about fossasia, just ask susi to tell that, Susi tells about the SUSI.AI creators", "image": "images/creator_info.png", "author": "madhav rathi", "author_url": "https://github.com/madhavrathi", "author_email": null, "skill_name": "Creator Info", "terms_of_use": null, "dynamic_content": false, "examples": [ "Who created you?", "what is fossasia?" ], "skill_rating": { "negative": "0", "positive": "0", "stars": { "one_star": 0, "four_star": 0, "five_star": 0, "total_star": 0, "three_star": 0, "avg_star": 0, "two_star": 0 }, "feedback_count": 0 }, "creationTime": "2018-03-17T16:38:29Z", "lastAccessTime": "2018-06-15T15:51:50Z", "lastModifiedTime": "2018-03-17T16:38:29Z" }, "accepted": true, "message": "Success: Fetched Skill's Metadata", "session": {"identity": { "type": "host", "name": "162.158.166.37_d80fb5c9", "anonymous": true }} } Set the react state of the component to store positive and negative rating. this.setState({ positive_rating, negative_rating }) Use react-icons to fetch like and dislike icon components from font-awesome. npm i -S react-icons Import the corresponding icons in the SkillPage component import { FaThumbsOUp, FaThumbsODown } from 'react-icons/lib/fa/' Display the rating count along with their icons <div className="rating"> <div className="positive"> <FaThumbsOUp /> {this.state.positive_rating} </div> <div className="negative"> <FaThumbsODown /> {this.state.negative_rating} </div> </div> Example References react-icons npm package Blog on icons as react components - https://medium.com/@david.gilbertson/icons-as-react-components-de3e33cb8792

Continue ReadingDisplaying skill rating for each skill on skill page of SUSI SKILL CMS
Read more about the article Adding “All” in Skill Categories

Adding “All” in Skill Categories

The SUSI SKill CMS has various filters to explore the skills of interest.  For example skill category and skill language. The skills are stored in the susi_skill_data Github repo in the following structure: susi_skill_data/models/<model_name>/<group_name>/<language_name>/<skill_name>.txt NOTE: group and category are same terms and can be used interchangeably So when a category filter is applied the skills from the corresponding directory are returned. susi_skill_data/models/<model_name>/<group_name>/<language_name>/<skill_name>.txt But there’s no directory called “All”,  so how to get skills of all groups? For this, we need to loop through all the directories present in the model. Server side implementation Create a helper function that returns a list of all the folders present in a directory. The function accepts the parent directory name and an empty list. First, fetch all the items (files and folders) present in that directory and store them in an array. Then apply a filter over the array to check if the element is a directory and doesn’t start with a dot(.) i.e., it's not hidden. Add the filtered array to the list. private void listFoldersForFolder(final File folder, ArrayList<String> fileList) { File[] filesInFolder = folder.listFiles(); if (filesInFolder != null) { Arrays.stream(filesInFolder) .filter(fileEntry -> fileEntry.isDirectory() && !fileEntry.getName().startsWith(".")) .forEach(fileEntry -> fileList.add(fileEntry.getName() + "")); } } Fetch the group name form the request and add a check if the CMS is asking for all skill. Otherwise, return the skills of a particular group only. String group_name = call.get("group", "Knowledge"); if (group_name.equals("All")) { // Return the list of all skills } else { // Return the list of a skills in a particular group only } To fetch the list of all skills, call the listFoldersForFolders() function with the model name and an empty list as arguments. The function adds all the directories, present in that model directory, to folderList. File allGroup = new File(String.valueOf(model)); ArrayList<String> folderList = new ArrayList<String>(); listFoldersForFolder(allGroup, folderList); Then loop over all the groups present in the list to get all the skills present in that group. This process is the same as the existing process of getting skills of a particular category. Just keep adding the skill list to a global array. CMS side implementation The list of categories is first fetched from the API and then added to the dropdown menu. Since the API doesn’t return “All” in it, so we need to push it to the list manually. groups.push(<MenuItem value="All" key="All" primaryText="All" />); References StackOverflow : https://stackoverflow.com/questions/5694385/getting-the-filenames-of-all-files-in-a-folder

Continue ReadingAdding “All” in Skill Categories
Read more about the article Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad

Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad

You may have experienced Apple HomPad, Google Home, Alexa etc or read about smart speakers that offer interactive action over voice commands. The smart speaker uses the hot word for activation. They utilize Wi-Fi, Bluetooth, and other wireless protocols. SUSI.AI is also coming with Open Source smart speaker that can do various actions like playing music etc over voice commands. To use SUSI Smart Speaker, you have to connect it to the SUSI iOS or Android App. You can manage your connected devices in SUSI iOS, Android and Web clients. Here we will see initial setups for connecting SUSI Smart Speaker with iPhone/iPad (iOS Devices). You may aware that iOS does not allow connecting to wifi within the app. To connect to a particular Wi-Fi, you have to go to phone settings, from there you can connect to Wi-Fi. SUSI Smart Speaker create a temporary Hotspot for initial setups. Follow the instruction below to connect to SUSI Smart Speaker hotspot - Tap to Home button, and go to your iPhone Settings > Wi-Fi Connect to the Wi-Fi hotspot for the device that you are setting up. It will have name “susi.ai”, like in the image below Come back to the SUSI app to proceed with setup. These instruction is also available within the app when you are not connected to SUSI Smart Speaker hotspot and click `Setup a Device` or plus icon on Device Activity screen navigation bar. Devices Activity and getting current Wi-Fi SSID: Devices section in Settings screen shows the currently connected device. In Devices Activity screen, the user can manage the connected device. Only a logged-in user can access Devices Activity. When the user clicks on Device Accessories in setting, if the user is not logged-in, an alert is prompted with Login option. By clicking Login option, user directed to Login screen where the user can log in and come back to device section to proceed further. If the user is already logged-in, Device Activity screen is presented. We use following method to scan if iPhone/iPad is connected to SUSI Smart Speaker: func fetchSSIDInfo() -> String? { var ssid: String? if let interfaces = CNCopySupportedInterfaces() as? [String] { for interface in interfaces { if let interfaceInfo = CNCopyCurrentNetworkInfo(interface as CFString) as NSDictionary? { ssid = interfaceInfo[kCNNetworkInfoKeySSID as String] as? String break } } } return ssid } Apple’s SystemConfiguration API is used to get current Wi-Fi SSID. SystemConfiguration Allow applications to access a device’s network configuration settings. Determine the reachability of the device, such as whether Wi-Fi or cell connectivity is active. import SystemConfiguration.CaptiveNetwork The method above return the SSID of your device current Wi-Fi. SSID is simply the technical term for a network name. When you set up a wireless home network, you give it a name to distinguish it from other networks in your neighborhood. You'll see this name when you connect your device to your wireless network. If current Wi-Fi match with SUSI Smart Speaker hotspot, we display device in TableView, if not we…

Continue ReadingInitial Setups for Connecting SUSI Smart Speaker with iPhone/iPad