For making a system more reliable and robust, continuous evaluation is quite important. So is in case of SUSI AI. User feedback is important to improve SUSI skills and create new ones. Previously we had only thumbs up / thumbs down as a feedback method, from the susi chat client. But now a 5 star rating system has been added to the SUSI Skill CMS so that users can rate a skill there. Before the implementation of API let’s look how data is stored in SUSI AI Susi_server uses DAO in which skill rating is stored as JSONTray.
The server side implementation
A new java class has been created for the API, FiveStarRateSkillService.java.
The getMinimalBaseRole method tells the minimum User role required to access this servlet it can also be ADMIN, USER or ANONYMOUS. In our case it is USER. A user needs to be logged in to rate a skill on a scale of 1-5 stars. The API runs at “/cms/fiveStarRateSkill.json” endpoint.
Next, create serviceImpl method in the above class to handle the request from the client and respond to it.
- Fetch the required query parameters and store them in variables. They include skill model, group, language, skill name and starts that the user has given in the rating.
- Then check if the skill exists. If not them throw an exception. Otherwise, increment the count of the corresponding rating. The rating object has keys as one_star, two_star, three_star, four_star and five_star that has the count of that star rating.
- Re-calculate the total rating done on that skill and its average rating and update the object. If the skill has not been already rated then create a new rating object and initialize it with the 0 star counts.
The complete FiveStarRateSkillService.java is available here : –
Rating a skill
This gives 3 star rating to the “aboutsusi” skill.
Getting the stats of Skill Ratings
This fetches the current ratings of the “aboutsusi” skill.
So this 5 star rating system will help in improving the SUSI skills. Also, it will help in making better decisions when we have multiple similar skills and we have to choose one to respond to the user query.
- Using SUSI AI Server to Store User Feedback for a Skill https://blog.fossasia.org/using-susi-ai-server-to-store-user-feedback-for-a-skill/