Skill Development using SUSI Skill CMS

There are a lot of personal assistants around like Google Assistant, Apple’s Siri, Windows’ Cortana, Amazon’s Alexa, etc. What is then special about SUSI.AI which makes it stand apart from all the different assistants in the world? SUSI is different as it gives users the ability to create their own skills in a Wiki-like system. You don’t need to be a developer to be able to enhance SUSI. And, SUSI is an Open Source personal assistant which can do a lot of incredible stuff for you, made by you. So, let’s say you want to create your own Skill and add it to the existing SUSI Skills. So, these are the steps you need to follow regarding the same - The current SUSI Skill Development Environment is based on an Etherpad. An Etherpad is a web-based collaborative real-time editor. https://dream.susi.ai/ is one such Etherpad. Open https://dream.susi.ai/ and name your dream (in lowercase letters). Define your skill in the Etherpad. The general skill format is ::name <Skill_name> ::author <author_name> ::author_url <author_url> ::description <description> ::dynamic_content <Yes/No> ::developer_privacy_policy <link> ::image <image_url> ::term_of_use <link> #Intent User query1|query2|query3.... Answer answer1|answer2|answer3...   Patterns in query can be learned easily via this tutorial. Open any SUSI Client and then write dream <your dream name> so that dreaming is enabled for SUSI. Once dreaming is enabled, you can now test any skills which you’ve made in your Etherpad. Once you’ve tested your skill, write ‘stop dreaming’ to disable dreaming for SUSI. If the testing was successful and you want your skill to be added to SUSI Skills, send a Pull Request to susi_skill_data repository providing your dream name. How do you modify an existing skill? SUSI Skill CMS is a web interface where you can modify the skills you’ve made. All the skills of SUSI are directly in sync with the susi_skill_data. To edit any skill, you need to follow these steps - Login to SUSI Skill CMS website using your email and password (or Sign Up to the website if you haven’t already). Click on the skill which you want to edit and then click on the “edit” icon. You can edit all aspects of the skill in the next state. Below is a preview: Make the changes and then click on “SAVE” button to save the skill. What’s happening Behind The Scenes in the EDIT process? SkillEditor.js is the file which is responsible for keeping a check over various validations in the Skill Editing process. There are certain validations that need to be made in the process. Those are as follows - Check whether User has logged in or not if (!cookies.get('loggedIn')) { notification.open({ message: 'Not logged In', description: 'Please login and then try to create/edit a skill', icon: <Icon type='close-circle' style={{ color: '#f44336' }} />, }); this.setState({ loading: false }); return 0; }   Check whether Commit Message has been entered by User or not if (this.state.commitMessage === null) { notification.open({ message: 'Please add a commit message', icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,…

Continue ReadingSkill Development using SUSI Skill CMS

Making a SUSI Skill to get details about bank from IFSC

We are going to make a SUSI skill that fetches information about a bank when the IFSC (Indian Financial System Code) is known. Here is a detailed explanation of how we going about doing this. Getting started with the skill creation API endpoint that returns the bank details Before going to the skill development, we need to find an API that would return the bank details from the IFSC, On browsing through various open source projects. I found an apt endpoint by Razorpay. Razorpay is a payment gateway for India which allows businesses to accept, process and disburse payments with ease. The Github link  to the repository is https://github.com/razorpay/ifsc. API endpoint -  https://ifsc.razorpay.com/<:ifsc> Request type -  GET Response type -  JSON Now, head over to the SUSI Etherpad, which is the current SUSI Skill Development Environment and create a new Pad.  Here, we need to define the skill in the Etherpad. We will now write rules/intents for the skill. An intent represents an action that fulfills a user's spoken request. Intents consist of 2 parts - User query - It contains different patterns of query that user can ask. Answer - It contains the possible answer to the user query. The main intent that our skill focuses on is, returning the bank name and address from the IFSC code. Here is how it looks - Name of bank with IFSC code * | Bank's name with IFSC code * !example:Name of bank with IFSC code SBIN0007245 !expect: The name of bank is State Bank of India !console:The name of bank with IFSC code $1$ is $object$ { "url":"https://ifsc.razorpay.com/$1$", "path":"$.BANK" } eol Part-wise explanation of the intent The first line contains the query pattern that the user can use while querying. You can see that a wildcard character (*) is used in the pattern. It contains the IFSC of the bank that we wish to know, and will later on use to fetch the details via the API. The second line contains an example query, followed by third line that contains the expected answer. Last part of the rule contains the answer that is fetched from an external API -  https://ifsc.razorpay.com/<:ifsc>  ,via the console service  provided by SUSI Skills. Here, <:ifsc> refers to the IFSC that the user wants to know about. We get it from the user query itself, and can access it by the variable name $1$ as it matches with the 1st wildcard present in the query. If there would be 2 wildcards, we could have accessed them by $1$ and $2$ respectively. The console service provides us with an option to enter the url of the API that we want to hit and path of the key we want to use. The sample response of the endpoint looks like this : { "BANK": "Karnataka Bank", "IFSC": "KARB0000001", "BRANCH": "RTGS-HO", "ADDRESS": "REGD. & HEAD OFFICE, P.B.NO.599, MAHAVEER CIRCLE, KANKANADY, MANGALORE - 575002", "CONTACT": "2228222", "CITY": "DAKSHINA KANNADA", "RTGS": true, "DISTRICT": "MANGALORE", "STATE": "KARNATAKA" }   Since, we want…

Continue ReadingMaking a SUSI Skill to get details about bank from IFSC

Implementing Skill Detail Section in SUSI Android App

SUSI Skills are rules that are defined in SUSI Skill Data repo which are basically the responses SUSI gives to the user queries. When a user queries something from the SUSI Android app, a query to SUSI Server is made which further fetches response from SUSI Skill Data and gives the response to the app. Similarly, when we need to list all skills, an API call is made to server to list all skills. The server then checks the SUSI Skill Data repo for the skills and then return all the required information to the app. Then the app displays all the information about the skill to user. User then can view details of each skill and then interact on the chat interface to use that skill. This process is similar to what SUSI Skill CMS does. The CMS is a skill wiki like interface to view all skills and then edit them. Though the app can not be currently used to edit the skills but it can be used to view them and try them on the chat interface. API Information For listing SUSI Skill groups, we have to call on /cms/getGroups.json This will give you all groups in SUSI model in which skills are present. Current response: { "session": {"identity": { "type": "host", "name": "14.139.194.24", "anonymous": true }}, "accepted": true, "groups": [ "Small Talk", "Entertainment", "Problem Solving", "Knowledge", "Assistants", "Shopping" ], "message": "Success: Fetched group list" } So, the groups object gives all the groups in which SUSI Skills are located. Next comes, fetching of skills. For that the endpoint is /cms/getGroups.json?group=GROUP_NAME Since we want all skills to be fetched, we call this api for every group. So, for example we will be calling http://api.susi.ai/cms/getSkillList.json?group=Entertainment for getting all skills in group “Entertainment”. Similarly for other groups as well. Sample response of skill: { "accepted": true, "model": "general", "group": "Shopping", "language": "en", "skills": {"amazon_shopping": { "image": "images/amazon_shopping.png", "author_url": "https://github.com/meriki", "examples": ["Buy a dress"], "developer_privacy_policy": null, "author": "Y S Ramya", "skill_name": "Shop At Amazon", "dynamic_content": true, "terms_of_use": null, "descriptions": "Searches items on Amazon.com for shopping", "skill_rating": null }}, "message": "Success: Fetched skill list", "session": {"identity": { "type": "host", "name": "14.139.194.24", "anonymous": true }} } It gives all details about skills: image author_url examples developer_privacy_policy author skill_name dynamic_content terms_of_use descriptions skill_rating Implementation in SUSI Android App Skill Detail Section UI of Google Assistant Skill Detail Section UI of SUSI SKill CMS Skill Detail Section UI of SUSI Android App The UI of skill detail section in SUSI Android App is the mixture of UI of Skill detail section in Google Assistant ap and SUSI Skill CMS. It displays details of skills in a beautiful manner with horizontal recyclerview used to display the examples. So, we have to display following details about the skill in Skill Detail Section: Skill Name Author Name Skill Image Try it Button Description Examples Rating Content type (Dynamic/Static) Terms of Use Developer’s Privacy policy Let’s see the implementation. 1. Whenever a skill Card View is clicked, showSkillDetailFragment()…

Continue ReadingImplementing Skill Detail Section in SUSI Android App

Implementing Skill Listing in SUSI Android App using Nested RecyclerViews

SUSI Skills are rules that are defined in SUSI Skill Data repo which are basically the responses SUSI gives to the user queries. When a user queries something from the SUSI Android app, a query to SUSI Server is made which further fetches response from SUSI Skill Data and gives the response to the app. Similarly, when we need to list all skills, an API call is made to server to list all skills. The server then checks the SUSI Skill Data repo for the skills and then return all the required information to the app. Then the app displays all the information about the skill to user. User then can view details of each skill and then interact on the chat interface to use that skill. This process is similar to what SUSI Skill CMS does. The CMS is a skill wiki like interface to view all skills and then edit them. Though the app can not be currently used to edit the skills but it can be used to view them and try them on the chat interface. API Information For listing SUSI Skill groups, we have to call on  /cms/getGroups.json This will give you all groups in SUSI model in which skills are present. Current response: { "session": {"identity": { "type": "host", "name": "14.139.194.24", "anonymous": true }}, "accepted": true, "groups": [ "Small Talk", "Entertainment", "Problem Solving", "Knowledge", "Assistants", "Shopping" ], "message": "Success: Fetched group list" } So, the groups object gives all the groups in which SUSI Skills are located. Next comes, fetching of skills. For that the endpoint is /cms/getGroups.json?group=GROUP_NAME Since we want all skills to be fetched, we call this api for every group. So, for example we will be calling http://api.susi.ai/cms/getSkillList.json?group=Entertainment for getting all skills in group “Entertainment”. Similarly for other groups as well. Sample response of skill: { "accepted": true, "model": "general", "group": "Shopping", "language": "en", "skills": {"amazon_shopping": { "image": "images/amazon_shopping.png", "author_url": "https://github.com/meriki", "examples": ["Buy a dress"], "developer_privacy_policy": null, "author": "Y S Ramya", "skill_name": "Shop At Amazon", "dynamic_content": true, "terms_of_use": null, "descriptions": "Searches items on Amazon.com for shopping", "skill_rating": null }}, "message": "Success: Fetched skill list", "session": {"identity": { "type": "host", "name": "14.139.194.24", "anonymous": true }} } It gives all details about skills: image author_url examples developer_privacy_policy author skill_name dynamic_content terms_of_use descriptions skill_rating Implementation in SUSI Android App Skill Listing UI of Google Assistant Skill Listing UI of SUSI SKill CMS Skill Listing UI of SUSI Android App The UI of skill listing in SUSI Android App is the mixture of UI of Skill listing in Google Assistant ap and SUSI Skill CMS. It displays skills in a beautiful manner with horizontal recyclerview nested in vertical recyclerview. So, for implementing horizontal recyclerview inside vertical recyclerview, you need two viewholders and two adapters (one each for a recyclerview). Let’s see the implementation. 1. First task is to fetch the information of groups in which skills are located. This line calls method in SkillListModel which then makes an API call to fetch groups. skillListingModel.fetchGroups(this) 2.…

Continue ReadingImplementing Skill Listing in SUSI Android App using Nested RecyclerViews

Uploading Images to SUSI Server

SUSI Skill CMS is a web app to create and modify SUSI Skills. It needs API Endpoints to function and SUSI Server makes it possible. In this blogpost, we will see how to add a servlet to SUSI Server to upload images and files. The CreateSkillService.java file is the servlet which handles the process of creating new Skills. It requires different user roles to be implemented and hence it extends the AbstractAPIHandler. Image upload is only possible via a POST request so we will first override the doPost method in this servlet. @Override protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { resp.setHeader("Access-Control-Allow-Origin", "*"); // enable CORS resp.setHeader enables the CORS for the servlet. This is required as POST requests must have CORS enables from the server. This is an important security feature that is provided by the browser. Part file = req.getPart("image"); if (file == null) { json.put("accepted", false); json.put("message", "Image not given"); } Image upload to servers is usually a Multipart Request. So we get the part which is named as “image” in the form data. When we receive the image file, then we check if the image with the same name exists on the server or not. Path p = Paths.get(language + File.separator + "images/" + image_name); if (image_name == null || Files.exists(p)) { json.put("accepted", false); json.put("message", "The Image name not given or Image with same name is already present "); } If the same file is present on the server then we return an error to the user requesting to give a unique filename to upload. Image image = ImageIO.read(filecontent); BufferedImage bi = this.createResizedCopy(image, 512, 512, true); if(!Files.exists(Paths.get(language.getPath() + File.separator + "images"))){ new File(language.getPath() + File.separator + "images").mkdirs(); } ImageIO.write(bi, "jpg", new File(language.getPath() + File.separator + "images/" + image_name)); Then we read the content for the image in an Image object. Then we check if images directory exists or not. If there is no image directory in the skill path specified then create a folder named “images”. We usually prefer square images at the Skill CMS. So we create a resized copy of the image of 512x512 dimensions and save that copy to the directory we created above. BufferedImage createResizedCopy(Image originalImage, int scaledWidth, int scaledHeight, boolean preserveAlpha) { int imageType = preserveAlpha ? BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB; BufferedImage scaledBI = new BufferedImage(scaledWidth, scaledHeight, imageType); Graphics2D g = scaledBI.createGraphics(); if (preserveAlpha) { g.setComposite(AlphaComposite.Src); } g.drawImage(originalImage, 0, 0, scaledWidth, scaledHeight, null); g.dispose(); return scaledBI; } The function above is used to create a  resized copy of the image of specified dimensions. If the image was a PNG then it also preserves the transparency of the image while creating a copy. Since the SUSI server follows an API centric approach, all servlets respond in JSON. resp.setContentType("application/json"); resp.setCharacterEncoding("UTF-8"); resp.getWriter().write(json.toString());’ At last, we set the character encoding and the character set of the output. This helps the clients to parse the data easily. To see this endpoint in live send a POST request at http://api.susi.ai/cms/createSkill.json. Resources Apache Docs: https://commons.apache.org/proper/commons-fileupload/using.html Multipart POST Request Tutorial: http://www.codejava.net/java-se/networking/upload-files-by-sending-multipart-request-programmatically…

Continue ReadingUploading Images to SUSI Server