SUSI Skill Data is the storage place for susi skills. All the skills are stored in these repository follows schema  Models -> Groups -> Languages -> Skill -> Intents. Using this, one can access any skill based on four tuples parameters model, group, language, skill. The SUSI skill CMS is an editor to write and edit skill easily. It follows an API-centric approach where the SUSI server acts as API server and a web front-end  act as the client for the API and provides the user interface. In this blog we will see how SUSI server updates the skill data repository.

For implementing versioning of SUSI Skills, server uses JGit. JGit is an EDL licensed, lightweight, pure Java library implementing the Git version control system.  To include a Gradle dependency to add JGit to the SUSI Server project.

compile 'org.eclipse.jgit:org.eclipse.jgit:4.6.1.201703071140-r'

Next we initialize the SUSI Skill Data repository in DAO.

 susi_skill_repo = new File(data_dir.getParentFile().getParentFile(), "susi_skill_data/.git");
repository = builder.setGitDir((DAO.susi_skill_repo))
           .readEnvironment() // scan environment GIT_* variables
           .findGitDir() // scan up the file system tree
           .build();

The code above opens our local git repository and creates an object “git”. Then we perform further operations on “git” object. Now we add our changes to “git”. This is similar to when we run any command using git. Next, to pull the latest changes we define a pull method catching the IOException,

public static void pull(Git git) throws IOException {
       try {
           PullResult pullResult = git.pull().call();
           MergeResult mergeResult = pullResult.getMergeResult();

           if (mergeResult!=null && mergeResult.getConflicts()!=null) {
               pullStatus =false;
               // we have conflicts send email to admin
               try {
                   EmailHandler.sendEmail(getConfig("skill_repo.admin_email",""), "SUSI Skill Data Conflicts", getConflictsMailContent(mergeResult));
               } catch (Throwable e) {
                e.printStackTrace();
               }

           } else {
               PushCommand push = git.push();
               push.setCredentialsProvider(new UsernamePasswordCredentialsProvider(getConfig("github.username", ""), getConfig("github.password","")));
               push.call();
           }
           
       } catch (GitAPIException e) {
           throw new IOException (e.getMessage());
       }

We check if there are any merge conflicts while merging the latest changes and local commits, if there are any we send the email to administrator to resolve them manually. Otherwise we push the commits to update the local git repository. The credentials currently are stored in config.properties file, and is used using the getConfig method.
To pull Susi_SKill_Data every minute we add a thread in DAO initialization method.

 log("Starting Skill Data pull thread");
       Thread pullThread = new Thread() {
           @Override
           public void run() {
               while (DAO.pullStatus) {
                   try {
                       Thread.sleep(getConfig("skill_repo.pull_delay", 60000));
                   } catch (InterruptedException e) {
                       break;
                   }
                   try {
                       DAO.pull(DAO.getGit());
                   } catch (Exception e) {
                       DAO.pullStatus =false;
                       Log.getLog().warn("SKILL PULL THREAD", e);
                   }
               }
           }
       };
       pullThread.start();

The thread sleeps for a default of 60 seconds, and updates the skill_data until any exception is faced. In case of exception the threads stops and administrator is informed about the git status.

This is how server pulls skill data every minute. To add the metadata  to the skill visit susi_skill_data, the storage place for susi skills. For more information and complete code take a look at Susi server and join gitter chat channel for discussions.

Resources

• JGit documentation : https://eclipse.org/jgit/documentation/
• The source code SUSI Server : https://github.com/fossasia/susi_server
• Read about servlet interface – https://www.javatpoint.com/Servlet-interface
• Read about DAO- https://en.wikipedia.org/wiki/Data_access_object

The SUSI skill CMS is an editor to write and edit skill easily. It follows an API-centric approach where the SUSI server acts as API server and a web front-end  act as the client for the API and provides the user interface. A skill is a set of intents. One text file represents one skill, it may contain several intents which all belong together. All the skills are stored in SUSI Skill Data repository and the schema is Models -> Groups -> Languages -> Skill -> Intents. Using this, one can access any skill based on four tuples parameters model, group, language, skill.

A skill format looks like following,

::name <Skill_name>
::author <author_name>
::author_url <author_url>
::description <description> 
::dynamic_content <Yes/No>
::developer_privacy_policy <link>
::image <image_url>
::terms_of_use <link>

#Intent
User query1 query2 quer3....
!example:<The question that should be shown in public skill displays>
!expect:<The answer expected for the above example>
Answer for the user query

Susi Skill class in Susi Server provides parser methods for sets of intents, given as text files.  It reads the skill line by line. The SusiMind class process this JSON and stores the metadata in a map of skill path and data, which provides the API endpoint for getting the skill the metadata .    http://api.susi.ai/cms/getSkillMetadata.json?model=general&group=Knowledge&language=en&skill=cars. It takes four parameters, model, group, language, and skill. The endpoint returns null value if the skill metadata field is not present in the skill. The following is the response for model=general, group=Knowledge, language =en and skill =cars.

"skill_metadata": {
    "model": "general",
    "group": "Knowledge",
    "language": "en",
    "developer_privacy_policy": null,
    "descriptions": "A skill to tell user about cars made by a car manufacturer",
    "image": "images/cars.png",
    "author": "Uday Theja",
    "author_url": "https://github.com/uday96",
    "skill_name": "Cars",
    "terms_of_use": null,
    "dynamic_content": true,
    "examples": [
      "Know any car models manufactured by ford",
      "Know any car models manufactured by ford in the year 2016",
      "Know any car models manufactured in 2016 by ford"
    ]
  },

Image is the path to image of the skill, stored in skill data repository and dynamic content represents if the skill’s response changes based on the user inputs.
To make the required API call we first need the value of 4 parameters. We will get these as props from browse skill route.

<Link key={el}
   to={{
        pathname: '/skillPage',
state: { url: url, element: el, name: el, modelValue: self.state.modelValue, groupValue:self.state.groupValue,        languageValue:self.state.languageValue}
                             }}>

Using these props we will append the base url with all the four parameters.

 let baseUrl = 'http://api.susi.ai/cms/getSkillMetadata.json';
 url = baseUrl + '?model=' + modelValue + '&group=' + groupValue + '&language=' + languageValue + '&skill=' + name;

Next we will make a ajax call and get the data updated.

 $.ajax({
                url: url,
                jsonpCallback: 'pc',
                dataType: 'jsonp',
                jsonp: 'callback',
                crossDomain: true,
                success: function (data) {
                    self.updateData(data.skill_metadata)
                }
            });

To display data on Skill CMS we will use Material UI components. Before rendering we will check the each state if it is null , we will display the default values for it.

{this.state.imgUrl === undefined?
                            <CircleImage name={this.state.skill_name.toUpperCase()} size="250"/>:
                            <img className="avatar-img" alt="Thumbnail" src={this.state.imgUrl}/>

If the image is not present in the skill, we display the placeholder showing the name of the skill.

Otherwise, the image stored in skill_data repository is shown.

Similarly, we will display the other components using material-ui. Next, we add an edit button for editing the skill.

To make edit button we use floating action button. We link the button to skillEditor route and pass the  URL and skillname as props.

  <Link to={{
           pathname: '/skillEditor',
           state: { url: urlCode, name:name, }
   }}>

SkillEditor components uses getSkill.json endpoint and using these props make the ajax call to the endpoint like: http://api.susi.ai/cms/getSkill.json?model=general&group=Entertainment&language=en&skill=<skill_name>.

This is how skill Page is implemented in Susi skill CMS, To add more skills or modify the existing ones visit skills.susi.ai. For more information and complete code take a look at Susi server and Susi_Skill_cms and join gitter chat channel for discussions.

Resources

• Read more about Servlets – https://www.tutorialspoint.com/servlets/
• The source code –https://github.com/fossasia/susi_skill_cms/tree/master/src/components/SkillPage
• Read about servlet interface – https://www.javatpoint.com/Servlet-interface
• Read merits of Material UI- https://thenextweb.com/dd/2015/11/10/what-are-the-real-merits-of-material-design/#.tnw_hnWuEF06