Versioning of SUSI Skills

This is a concept for the management of the skill repository aka The “SUSI Skill CMS.

With SUSI we are building a personal assistant where the users are able to write and edit skills in the easiest way that we can think of. To do that we have to develop a very simple skill language and a very simple skill editor

The skill editor should be done as a ‘wiki’-like content management system (cms). To create the wiki, we follow an API-centric approach. The SUSI server acts as an API server with a web front-end which acts as a client of the API and provides the user interface.

The skill editor will be ‘expert-centric’, an expert is a set of skills. That means if we edit one text file, that text file represents one expert, it may contain several skills which all belong together.

An ‘expert’ is stored within the following ontology:

model  >  group  >  language  >  expert  >  skill

To Implement the CMS wiki system we need versioning with a working AAA System. To implement versioning we used JGit. JGit is an EDL licensed, lightweight, pure Java library implementing the Git version control system.

So I included a Gradle dependency to add JGit to the SUSI Server project.

compile 'org.eclipse.jgit:org.eclipse.jgit:4.6.1.201703071140-r'

Now the task was to execute git commands when the authorised user makes changes in any of the expert. The possible changes in an expert can be

1. Creating an Expert
2. Modifying an existing Expert
3. Deleting an Expert

1. git add <filename>

2. git commit -m “commit message”

Real Example in SUSI Server

This is the code that every servlet shares. It defines the base user role set a URL endpoint to trigger the endpoint

public class ModifyExpertService extends AbstractAPIHandler implements APIHandler {
    @Override
    public String getAPIPath() {
        return "/cms/modifyExpert.json";
    }

This is the part where we do all the processing of the URL parameters and store their versions. This method takes the “Query call” and then extracts the “get” parameters from it.
For the functioning of this service, we need 5 things, “model”, “group”, “language”, “expert” and the “commit message”.

@Override
public ServiceResponse serviceImpl(Query call, HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) {

    String model_name = call.get("model", "general");
    File model = new File(DAO.model_watch_dir, model_name);
    String group_name = call.get("group", "knowledge");
    File group = new File(model, group_name);
    String language_name = call.get("language", "en");
    File language = new File(group, language_name);
    String expert_name = call.get("expert", null);
    File expert = new File(language, expert_name + ".txt");

Then we need to open your SUSI Skill DATA repository and commit the new file in it. Here we call the functions of JGit, which do the work of git add and git commit.

FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repository = null;
try {

    repository = builder.setGitDir((DAO.susi_skill_repo))
            .readEnvironment() // scan environment GIT_* variables
            .findGitDir() // scan up the file system tree
            .build();

    try (Git git = new Git(repository)) {

The code above opens our local git repository and creates an object “git”. Then we perform further operations on “git” object. Now we add our changes to “git”. This is similar to when we run “git add . ”

    git.add()
                .addFilepattern(expert_name)
                .call();

Finally, we commit the changes. This is similar to “git commit -m “message”.

    git.commit()
                .setMessage(commit_message)
                .call();

At last, we return the success object and set the “accepted” value as “true”.

    json.put("accepted", true);
        return new ServiceResponse(json);
    } catch (GitAPIException e) {
        e.printStackTrace();
    }

Resources

JGit documentation : https://eclipse.org/jgit/documentation/

SUSI Server : https://github.com/fossasia/susi_server

 

Continue ReadingVersioning of SUSI Skills

Servlets and Containers in SUSI Server

The core of SUSI Clients is the SUSI server that holds the “intelligence” and “personality” of SUSI AI. SUSI Server is in JAVA and it uses the concepts of Servlets heavily. While implementing server for SUSI, we have used JAVA as the backend language and hence are using servlets and Servlet containers heavily.

The problem that servlets are solving is regarding how to generate dynamic content on every request if the request comes with different parameters.  This is not possible by simply using HTML.

Servlets are widely used for dynamic content and have inbuilt support for HTTP (Hypertext Transfer Protocol).

In this blog post, along with basics of servlets and servlet containers, I am explaininghow to make classes for custom servlets. I am using an example of AbstractAPIHandler class that we have used in SUSI server for clear picture.

Web Server

Before we can understand a  servlet container, we need to understand what is a web server.

A web server uses HTTP protocol to transfer data. When a user types in a URL in a browser or any client, he first sees loading and then see the content of the page. So actually there is a server that is sending the content of the page to the user. The transfer of data is in HTTP protocol.

Servlet Container

From the example above, a user can request static pages from the server. Hence the pages will be very basic and we cannot do much with user inputs and dynamic data such as making a real time chat bot. So the idea of having Servlet container is using Java to dynamically generate the content of a web page from the server side.  

The flow of Control

The servlet container manages servlets through servlet life cycles. The container calls a servlet as soon as it receives an HTTP request. Then the servlet does the processing of the data which it has received from the request. the container sends the HTTP response back to the client and then the clients render it to show the final output.

The flowchart below explains how a request is processed from a browser to a servlet container and then to a servlet. It shows the usage of a generic Servlet.

Similarly in SUSI-Server we have a “AbstractAPIHandler” which is a generic servlet that every other servlet inherits.

Structure of a Servlet

public class ServletName extends HttpServlet {

public void init() throws ServletException {
// Servlet Initialization
}

protected void service(HttpServletRequest req,
HttpServletResponse resp)
throws ServletException, IOException {
// Code for the Service Method
}
/**
* Process a GET request
*/
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws IOException, ServletException {
// Code for the doGet() method
}
/**
* Process a POST request
*/
protected void doPost(HttpServletRequest request,
HttpServletResponse response)
throws IOException, ServletException {
// Code for the doPost() method

}
/**
* Process a PUT request
*/
protected void doPut(HttpServletRequest req,
HttpServletResponse resp)
throws ServletException, IOException {
//Code for the doPut() method

}
}

Abstract API Handler

public abstract class AbstractAPIHandler extends HttpServlet implements APIHandler {
}

The function below sets the Minimum Base Role of the SUSI user. We can set which servlet is accessible by which user.

@Override
public abstract BaseUserRole getMinimalBaseUserRole();

The function below gets the Default Permissions of a user Role in SUSI Server. The permissions for each SUSI user can be different.

@Override
public abstract JSONObject getDefaultPermissions(BaseUserRole baseUserRole);

The function below finally sets whatever output we want to show as a response when we query the SUSI Server.

   public abstract ServiceResponse serviceImpl(Query post,  HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) throws APIException;

Each Servlet in SUSI Server extends this class (AbstractAPIHandler.java). This also increases code reusability.

A Servlet is not a just a simple java class. You cannot run a servlet using javac or by running main() function. In order to run a servlet, you have to deploy this Servlet on a web server.

References

SUSI Server : https://github.com/fossasia/susi_server/

Docs: http://docs.oracle.com/javaee/6/tutorial/doc/bnafd.html

Continue ReadingServlets and Containers in SUSI Server

Map Responses from SUSI Server

Giving user responses in Maps is a very common reply of various assistants. Android and iOS clients of SUSI.AI can Render their own custom map views but there is a challenge in doing that in Bots and other clients which can only render Images.

The MapServlet in SUSI-Server is a servlet that takes the constraints of a map as input parameters and returns PNG Image of the rendered map.

This is a simple HttpServlet. Since it does not requires any auth or user rights we do not extend AbstactAPIHandler in this class.

 
public class MapServlet extends HttpServlet {

This is to method which runs whenever the Servlet receives a GET Query. After getting the query the function call another function “process(…)” which processes the input parameters.

 
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse          response) throws ServletException, IOException {
Query post = RemoteAccess.evaluate(request);

This line is the for DDOS Protection.  It checks if the user query rate is too high and rate limit his queries. If Server suspects that the post is a DDOS then it returns a 503 error.

if (post.isDoS_blackout()) {
  response.sendError(503, "your request frequency is too high"); return;
} 
process(request, response, post);

The process method get the zoom, latitude, longitude, width and the height of the map and checks for the errors in them.

if (map.getHeight() > height || map.getWidth() > width) {
            BufferedImage bi = map.getImage();

            int xoff = (map.getWidth() - width) / 2;
            int yoff = (map.getHeight() - height) / 2;

            bi = bi.getSubimage(xoff, yoff, width, height);
            map = new RasterPlotter(width, height, RasterPlotter.DrawMode.MODE_REPLACE, "FFFFFF");
            map.insertBitmap(bi, 0, 0);
        }

Then we compute our image using RasterPlotter.

map.setDrawMode(DrawMode.MODE_SUB);
map.setColor(0xffffff);
if (text.length() > 0) PrintTool.print(map, 6, 12, 0, uppercase ? text.toUpperCase() : text, -1, false, 100);
PrintTool.print(map, map.getWidth() - 6, map.getHeight() - 6, 0, "MADE WITH LOKLAK.ORG", 1, false, 50);

Here we calculate the height of the map. If a user passed a height and width in get parameters, we scale the map according to that but if not its 256×256.

    int mx = (int) (map.getWidth() * (lon - west_lon) / (east_lon - west_lon));
    int my = (int) (map.getHeight() * (lat - north_lat) / (south_lat - north_lat));

// the marker has a height of 40 pixel and a width of 25 pixel
    final BufferedImage logo = ImageIO.read(FileSystems.getDefault().getPath("html").resolve("artwork").resolve("marker-red.png").toFile());
    map.insertBitmap(logo, Math.min(map.getWidth() - 25, Math.max(0, mx - 12)), Math.min(map.getHeight() - 40, Math.max(0, my - 40)), FilterMode.FILTER_ANTIALIASING);

We use the code below to set some text on the map for example the name of the place or the city etc.

map.setDrawMode(DrawMode.MODE_SUB);
map.setColor(0xffffff);
if (text.length() > 0) PrintTool.print(map, 6, 12, 0, uppercase ? text.toUpperCase() : text, -1, false, 100);
PrintTool.print(map, map.getWidth() - 6, map.getHeight() - 6, 0, "MADE WITH LOKLAK.ORG", 1, false, 50);

Finally we draw a marker on map

    int mx = (int) (map.getWidth() * (lon - west_lon) / (east_lon - west_lon));
    int my = (int) (map.getHeight() * (lat - north_lat) / (south_lat - north_lat));
      
    final BufferedImage logo = ImageIO.read(FileSystems.getDefault().getPath("html").resolve("artwork").resolve("marker-red.png").toFile());
    map.insertBitmap(logo, Math.min(map.getWidth() - 25, Math.max(0, mx - 12)), Math.min(map.getHeight() - 40, Math.max(0, my - 40)), FilterMode.FILTER_ANTIALIASING);

  At last we set the copyright message of OPENSTREETMAP at the bottom left.

PrintTool.print(map, 6, map.getHeight() - 6, 0, "(C) OPENSTREETMAP CONTRIBUTORS", -1, false, 100);

At the end we write the image and set the headers for Cross Origin Access.

    response.addHeader("Access-Control-Allow-Origin", "*");
        RemoteAccess.writeImage(fileType, response, post, map);
        post.finalize();
    }
}

 The servlet can be tested locally at

http://localhost:4000/vis/map.png?text=Test&mlat=1.28373&mlon=103.84379&zoom=18

Or on the live SUSI Server
http://api.susi.ai/vis/map.png?text=Test&mlat=1.28373&mlon=103.84379&zoom=18

Output:

References:

https://www.openstreetmap.org/

http://wiki.openstreetmap.org/wiki/Java_Access_Example

https://github.com/fossasia/susi_server/

Continue ReadingMap Responses from SUSI Server

Tray Icon for SUSI Desktop Client

Susi is an AI which replies smartly to anything that you put. Popular Assistants like Siri and Cortana, have their own tray icons that run on top of all apps. So, I thought to add a tray icon to SUSI AI as well.

A tray is an icon in an OS’s notification area usually attached with a context menu.

Platform limitations:

  • On Linux, app indicator is used if it is supported else GtkStatusIcon is used
  • App indicator is only shown when it has a context menu.
  • When app indicator is used all click events are ignored.

Starting Up:

You should have Node and ElectronJS installed

To create a tray I made two methods. One is for creating a tray and other is for creating a window when a tray icon is clicked.

app.on('ready', () => {
 createTray()
 createWindow()
 })

app.on(‘ready’,() => {}) is called when the electron is ready to run. It’s just like the main function.

Then we create the Tray.

const createTray = () => {
 tray = new Tray(path.join(__dirname, 'static/icons/png/tray.png'))
 tray.on('right-click', toggleWindow)
 tray.on('double-click', toggleWindow)
 tray.on('click', function (event) {
 toggleWindow()
 

When some developer is using SUSI he might // Show devtools when command clicked if 

(window.isVisible() && process.defaultApp && event.metaKey) {
 window.openDevTools({mode: 'detach'}) }}) }
Tray = new Tray(ICON HERE) // this creates a new tray with the icon specified.

Since there are 2 themes for the notification bar in OSX, I used a BLUE icon that looks good on both the themes.

I Used the icon above as a tray icon for SUSI.

Then next lines are for opening the tray with right-click or double-click. We can also change these functionalities like on right-click we can open some settings or give user an option to close the app.

const createWindow = () => {
  window = new BrowserWindow({
    width: 300,
    height: 450,
    show: false,
    frame: false,
    fullscreenable: false,
    resizable: false,
    transparent: true,
    webPreferences: {
      // Prevents renderer process code from not running when window is
      // hidden
      backgroundThrottling: false
    }
  })
  window.loadURL(`file://${path.join(__dirname, '/static/index.html')}`)
  // show inspect element in the window
  // window.openDevTools();
  // set window to always be on the top of other windows
  window.setAlwaysOnTop(true);
  // Hide the window when it loses focus
  window.on('blur', () => {
    if (!window.webContents.isDevToolsOpened()) {
      window.hide()
    }
  })
}

 

When we call createWindow it creates a Window with the HTML File specified in loadURL

window.loadURL(`file://${path.join(__dirname, '/static/index.html')}`)

In this we created a SUSI Chat Client in HTML and named it as index.html and passed that HTML to be loaded as the application window.

Sometimes when we a running a full screen, Especially in macOS then all apps are launched behind the fullscreen app. To run our tray icon we used another function that set the window to be always on top of all other windows.

 window.setAlwaysOnTop(true);

So if someone is reading a news article and wants to find a meaning of a word he need not open SUSI again, because SUSI is running on top of all the applications.

toggleWindow is called when we click the Tray Icon. The work of this function is fairly simple. On click of the tray icon, it makes hides or shows the SUSI application window.

const toggleWindow = () => {
  if (window.isVisible()) {
    window.hide()
  } else {
    showWindow()
  }
}

showWindow is the function that shows the windows and brings it to focus.

This function brings the SUSI Window to focus and shows it to the user.

const showWindow = () => {
 const position = getWindowPosition()
 window.setPosition(position.x, position.y, false)
 window.show()
 window.focus()
}

Running

Navigate to the project directory and run:

electron . 

SUSI icon will be visible in your notification tray.

Resources:

SUSI Desktop App: https://github.com/fossasia/susi_desktop/ 

Electron Documentation: https://electron.atom.io/

Electron Packager: https://github.com/electron-userland/electron-packager

Continue ReadingTray Icon for SUSI Desktop Client

SUSI Desktop App in Electron

 

Electron allows you to get a cross-platform desktop application up and running very quickly. An electron app is built using the node.js framework and is brought to us by Github. Apps built with Electron are just web sites which are opened in an embedded Chromium web browser.

Setting Up.

First things first, we need NodeJS installed. So go to https://nodejs.org/en/download/ and follow the instructions as per your Operating System.

Now we need Electron. We can install it globally and use it as a CLI, or install it locally in our application’s path. I recommend installing it globally, so that every app does not need to install it every time.

Open terminal and write:

$ npm install -g electron-prebuilt
$ npm install -g bower


Bower is used for managing front end components like HTML, CSS, JS etc.

To test the installation, type electron -h and it should show the version of Electron CLI.

Getting the project ready.

The main idea behind making desktop apps with JS is that you build one codebase and package it for each operating system separately. This abstracts away the knowledge needed to make native desktop apps and makes maintenance a lot easier.

First clone the SUSI Desktop app repository on your computer.

$ git clone https://github.com/fossasia/susi_desktop.git

Next step is to change directory to susi_desktop.

$ cd susi_desktop

Now we need to install dependencies which are used by our app.

$ npm install
$ bower install

Now we have the code and the dependencies setup. To run the Application write

$ npm start

Or

$ electron .

Any of the two commands can be used to run a electron application.

Screenshot:

Folder Structure.

In the Project folder or the root of the app we will keep the package.json file, which have all the dependencies to run the application.Also the main.js file and any other application-wide files we need are kept int the root of the app..

The “app” folder will house our HTML files of various types within folders like css, js and img.

Judging by the file structure, you can never guess this is a desktop app and not just a simple website. This project follows the AirBNB Style guide to code.

Purely starting the main process doesn’t give the users of your application any UI windows.

Conclusion

Overall Electron is an exciting way to build desktop web applications using web technologies.

SUSI Desktop app is built on it. Single code can be used to make native apps for Windows / Linux and Mac OS X.

Continue ReadingSUSI Desktop App in Electron

Deploying SUSI-Server on Digital Ocean

Create and Setup a Droplet

To create your first Droplet first login to your account. If you are using Github student developer pack then you will get $50 as Digital Ocean Credits.

The create button will be right there on the first page, click on “Create Droplet”
Then a page will open asking you some configurations of a server.

You are free to choose any Distribution and choose a size. I chose Ubuntu and $20/mo plan.
Now add information about the Data Center you want to use.

Add your SSH Keys. These keys will be used to authenticate you on remote login to your server.

Click Create. In a few seconds your droplet should be up.

Navigate to Droplets page and see that your droplet is live there. You should see your droplet name and IP there.

SSH to your server and Deploy SUSI

In order to connect to a remote Linux server via SSH, you must have following:

  • User name: The remote user to log in as. The default admin user, or Superuser, on most Linux servers is root
  • Password and/or SSH Key: The password that is used to authenticate the user that you are logging in as. If you added a public SSH key to your droplet when you created it, you must have the private SSH key of the key pair (and passphrase, if it has one)
  • Server IP address: This is the address that uniquely identifies your server on the Internet.

Open Terminal and run

ssh [email protected]_IP_ADDRESS

You should be able to login to your server.

Now to run SUSI Server

(Step 1 to 10 to setup java and gradle, skip if you have done it before)

Visit SERVER_IP_ADDRESS:4000 and see that SUSI Server is running there.

To stop the server use:

bin/stop.sh

Continue ReadingDeploying SUSI-Server on Digital Ocean