servlet

Read more about the article Implementing Feedback Feature in SUSI Android App

Implementing Feedback Feature in SUSI Android App

Recently, on SUSI Server, a new servlet was added which is used to rate SUSI Skills either positive or negative. The server stores the rating of a particular skill in a JSON file. These ratings help in improving answers provided by SUSI. So, the server part is done and it was required to implement this in the SUSI Android App. In this blog, I will cover the topic of implementation of the Rating or Feedback feature in SUSI Android App. This will including all the cases when feedback should be sent, when it should not be sent, when to send positive feedback, when to send negative feedback, etc.

API Information

For rating a SUSI Skill, we have to call on  /cms/rateSkill.json providing 5 parameters which are:

  1. model: The model of SUSI Skill. (String)
  2. group: The Group under the model in which that particular skill resides. (String)
  3. language: The language of skill. (String)
  4. skill: This is skill name. (String)
  5. rating: This can be two strings, either “positive” or “negative”. String)

Basically, in the SUSI Skill Data repo (in which all the skills are stored), models, groups language etc are part of folder structure.

So, if a skill is located here

https://github.com/fossasia/susi_skill_data/blob/master/models/general/Knowledge/en/news.txt

This would mean

model = general

group = Knowledge

language = en

skill = news

rating = positive/negative

Implementation in SUSI Android App

    

So, when the like button on a particular skill is clicked, a positive call is made and when the dislike button is clicked, a negative call is made.

Let’s see example when the thumbs up button or like button is clicked.

There can be three cases possible:

  1. None of Like button or dislike button is clicked already: In this case, initially, both like and dislike button will be transparent/hollow. So, when like button is clicked, the like button will be colored blue and a call will be made with positive feedback.
  2. Like button is already clicked: In this case, like button is already clicked. So, it will already be blue. So, when user clicks again on positive button, it should get back to normal/hollow indicating rating which was sent is cancelled and a a call will be made with negative feedback thus cancelling or neutralizing the earlier, positive feedback.
  3. Dislike button is already clicked: In this case, the dislike button is already blue, indicating a negative call is already made. So, now when the like button is clicked, we need to cancel the earlier negative feedback call and sending another negative feedback call. Thus, sending two negative feedback calls. And after that coloring dislike button as blue.

Look at the code below. It is self explanatory. There are three if-else conditions covering all the above mentioned three cases.

thumbsUp.setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View view) {
       thumbsUp.setImageResource(R.drawable.thumbs_up_solid);
       if(!model.isPositiveRated() && !model.isNegativeRated()) {
           rateSusiSkill(Constant.POSITIVE, model.getSkillLocation(), context);
           setRating(true, true);
       } else if(!model.isPositiveRated() && model.isNegativeRated()) {
           setRating(false, false);
           thumbsDown.setImageResource(R.drawable.thumbs_down_outline);
           rateSusiSkill(Constant.POSITIVE, model.getSkillLocation(), context);
           sleep(500);
           rateSusiSkill(Constant.POSITIVE, model.getSkillLocation(), context);
           setRating(true, true);
       } else if (model.isPositiveRated() && !model.isNegativeRated()) {
           rateSusiSkill(Constant.NEGATIVE, model.getSkillLocation(), context);
           setRating(false, true);
           thumbsUp.setImageResource(R.drawable.thumbs_up_outline);
       }
   }
});

Similarly for when dislike button is clicked, the above three mentioned cases still hold resulting in this code snippet.

thumbsDown.setOnClickListener(new View.OnClickListener() {
   @Override
   public void onClick(View view) {
       thumbsDown.setImageResource(R.drawable.thumbs_down_solid);
       if(!model.isPositiveRated() && !model.isNegativeRated()) {
           rateSusiSkill(Constant.NEGATIVE, model.getSkillLocation(), context);
           setRating(true, false);
       } else if(model.isPositiveRated() && !model.isNegativeRated()) {
           setRating(false, true);
           thumbsUp.setImageResource(R.drawable.thumbs_up_outline);
           rateSusiSkill(Constant.NEGATIVE, model.getSkillLocation(), context);
           sleep(500);
           rateSusiSkill(Constant.NEGATIVE, model.getSkillLocation(), context);
           setRating(true, false);
       } else if (!model.isPositiveRated() && model.isNegativeRated()) {
           rateSusiSkill(Constant.POSITIVE, model.getSkillLocation(), context);
           setRating(false, false);
           thumbsDown.setImageResource(R.drawable.thumbs_down_outline);
       }
   }
});

Summary

So, this blog talked about how the Feedback feature in SUSI Android App is implemented. This included how a network call is made, logic for sending positive/negative feedback, logic to withdraw feedback etc. So, If you are looking forward to contribute to SUSI Android App, this can help you a little. But if not so, this may also help you in understanding and how rating mechanism in social media websites like Facebook, Twitter, Quora, Reddit, etc work.

References

  1. To know about servlets https://en.wikipedia.org/wiki/Java_servlet
  2. To see how to implement one https://www.javatpoint.com/servlet-tutorial
  3. To see how to make network calls in android using Retrofit https://guides.codepath.com/android/Consuming-APIs-with-Retrofit
  4. To see how to implement click listeners on button https://developer.android.com/reference/android/view/View.OnClickListener.html
Continue ReadingImplementing Feedback Feature in SUSI Android App
Read more about the article Download SUSI.AI Setting Files from Data Folder

Download SUSI.AI Setting Files from Data Folder

In this blog, I will discuss how the DownloadDataSettings servlet hosted on SUSI server functions. This post also covers a step by step demonstration on how to use this feature if you have hosted your own custom SUSI server and have admin rights to it. Given below is the endpoint where the request to download a particular file has to be made.

/data/settings

For systematic functionality and workflow, Users with admin login, are given a special access. This allows them to download the settings files and go through them easily when needed. There are various files which have email ids of registered users (accounting.json), user roles associated to them (authorization.json), groups they are a part of (groups.json) etc. To list all the files in the folder, use the given below end point:

/aaa/listSettings.json

How does the above servlet works? Prior to that, let us see how to to get admin rights on your custom SUSI.AI server.
For admin login, it is required that you have access to files and folders on server. Signup with an account and browse to

/data/settings/authorization.json

Find the email id with which you signed up for admin login and change userRole to “admin”. For example,

{
	"email:test@test.com": {
		"permissions": {},
		"userRole": "user"
	}
}

If you have signed up with an email id “test@test.com” and want to give admin access to it, modify the userRole to “admin”. See below.

{
	"email:test@test.com": {
		"permissions": {},
		"userRole": "admin"
	}
}

Till now, server did not have any email id with admin login or user role equal to admin. Hence, this exercise is required only for the first admin. Later admins can use changeUserRole application and give/change/modify user roles for any of the users registered. By now you must have admin login session. Let’s see now how the download and file listing servlets work.
First, the server creates a path by locally referencing settings folder with the help of DAO.data_dir.getPath(). This will give a string path to the data directory containing all the data-settings files. Now the server just has to make a JSONArray and has to pass a String array to JSONArray’s constructor, which will eventually be containing the name of all the data/settings files. If the process is not successfull ,then, “accepted” = false will be sent as an error to the user. The base user role to access the servlet is ADMIN as only admins are allowed to download data/setting files,
The file name which you have to download has to be sent in a HTTP request as a get parameter. For example, if an admin has to download accounting.json to get the list of all the registered users, the request is to be made in the following way:

BASE_URL+/data/settings?file=file_name

*BASE_URL is the URL where the server is hosted. For standard server, use BASE_URL = http://api.susi.ai.

In the initial steps, Server generates a path to data/settings folder and finds the file, name of which it receives in the request. If no filename is specified in the API call, by default, the server sends accounting.json file.

File settings = new File(DAO.data_dir.getPath()+"/settings");
String filePath = settings.getPath(); 
String fileName = post.get("file","accounting"); 
filePath += "/"+fileName+".json";

Next, the server will extract the file and using ServletOutputStream class, it will generate properties for it and set appropriate context for it. This context will, in turn, fetch the mime type for the file generated. If the mime type is returned as null, by default, mime type for the file will be set to application/octet-stream. For more information on mime type, please look at the following link. A complete list of mime types is compiled and documented here.

response.setContentType(mimetype);
response.setContentLength((int)file.length());

In the above code snippet, mime type and length of the file being downloaded is set. Next, we set the headers for the download response and use filename for that.

response.setHeader("Content-Disposition", "attachment; filename=" + fileName +".json");

All the manual work is done by now. The only thing left is to open a buffer stream, size of which has been defined as a class variable.
Here we use a byte array of size 4096 elements and write the file to client’s default download storage.

private static final int BUFSIZE = 4096;
byte[] byteBuffer = new byte[BUFSIZE];
             DataInputStream in = new DataInputStream(new FileInputStream(file));
            while ((in != null) && ((length = in.read(byteBuffer)) != -1))
            {
                outStream.write(byteBuffer,0,length);
            }

            in.close();
            outStream.close();

All the above-mentioned steps are enclosed in a try-catch block, which catches an exception if any ,and logs it in the log file. This message is also sent to the client for appropriate user information along with the success or failure indication through a boolean flag. Do not forget to close the input and output buffers as it may lead to memory leaks and someone with proper knowledge of network and buffer stream would be able to steal any essential or secured data.

Additional Resources

Continue ReadingDownload SUSI.AI Setting Files from Data Folder
Read more about the article Adding API endpoint to SUSI.AI for Skill Historization

Adding API endpoint to SUSI.AI for Skill Historization

SUSI Skill CMS is an editor to write and edit skill easily. It follows an API centric approach where the Susi server acts as API server. Using Skill CMS we can browse history of a skill, where we get commit ID, commit message  and name the author who made the changes to that skills. In this blogpost we will see how to fetch complete commit history of a skill in the susi skill repository. A skill is a set of intents. One text file represents one skill, it may contain several intents which all belong together. Susi skills are stored in susi_skill_data repository. We can access any skill based on four tuples parameters model, group, language, skill.  For managing version control in skill data repository, the following dependency is added to build.gradle . JGit is a library which implements the Git functionality in Java.

dependencies {
 compile 'org.eclipse.jgit:org.eclipse.jgit:4.6.1.201703071140-r'
}

To implement our servlet we need to extend our servlet to AbstractAPIHandler. In Susi Server, an abstract class AbstractAPIHandler extending HttpServelets and implementing API handler interface is provided. 

public class HistorySkillService extends AbstractAPIHandler implements APIHandler {}

The AbstractAPIHandler checks the permissions of the user using the userroles of and comparing it with the value minimum base role of each servlet. Thus to specify the user permission for a servlet we need Override the getMinimalBaseUserRole method.

 @Override
    public BaseUserRole getMinimalBaseUserRole() {
        return BaseUserRole.ANONYMOUS;
    }

UserRoles can be Admin, Privilege, User, Anonymous. In our case it is Anonymous. A User need not to log in to access this endpoint.

  @Override
    public String getAPIPath() {
        return "/cms/getSkillHistory.json";
    }

This methods sets the api endpoint path. One need to send requests at http://api.susi.ai/cms/getSkillHistory.json to get the modification history of skill. Next we will implement The ServiceImpl method where we will be processing the user request and giving back the service response.

@Override
    public ServiceResponse serviceImpl(Query call, HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) {

        String model_name = call.get("model", "general");
        File model = new File(DAO.model_watch_dir, model_name);
        String group_name = call.get("group", "knowledge");
        File group = new File(model, group_name);
        String language_name = call.get("language", "en");
        File language = new File(group, language_name);
        String skill_name = call.get("skill", "wikipedia");
        File skill = new File(language, skill_name + ".txt");
        JSONArray commitsArray;
        commitsArray = new JSONArray();
        String path = skill.getPath().replace(DAO.model_watch_dir.toString(), "models");
        //Add to git
        FileRepositoryBuilder builder = new FileRepositoryBuilder();
        Repository repository = null;
        try {
            repository = builder.setGitDir((DAO.susi_skill_repo))
                    .readEnvironment() // scan environment GIT_* variables
                    .findGitDir() // scan up the file system tree
                    .build();
            try (Git git = new Git(repository)) {
                Iterable<RevCommit> logs;
                logs = git.log().addPath(path).call();
                int i = 0;
                for (RevCommit rev : logs) {
                    commit = new JSONObject();
                    commit.put("commitRev", rev);
                    commit.put("commitName", rev.getName());
                    commit.put("commitID", rev.getId().getName());
                    commit.put("commit_message", rev.getShortMessage());
                    commit.put("author",rev.getAuthorIdent().getName());
                    commitsArray.put(i, commit);
                    i++;
                } success=true;
            } catch (GitAPIException e) {
                e.printStackTrace();
                success=false;
           } if(commitsArray.length()==0){
            success=false;
        }
        JSONObject result = new JSONObject();
        result.put("commits",commitsArray);
        result.put("success",success);
        return new ServiceResponse(result);
    }

To access any skill we need parameters model, group, language. We get this through call.get method where first parameter is the key for which we want to get the value and second parameter is the default value. Based on received model, group and language browse files in that folder we build the susi_skill_data repository path read the git variables and scan up the file system tree using FileRepositoryBuilder build() method. Next we fetch all the logs of the skill file and store them in json commits array and finally pass as a server response with success messages. In case of exceptions, pass service with success flags as false.

We have successfully implemented the servlet. Check the working of endpoint by sending request like http://api.susi.ai/cms/getSkillHistory.json?model=general&group=knowledge&language=en&skill=bitcoin and checking the response.

Susi skill cms uses this endpoint to fetch the skill history, try it out at http://skills.susi.ai/browseHistory

Resources
Continue ReadingAdding API endpoint to SUSI.AI for Skill Historization
Read more about the article Reset Password for SUSI Accounts Using Verification Link

Reset Password for SUSI Accounts Using Verification Link

In this blog, I will discuss how the SUSI server interprets the verification link sent to your email id to reset SUSI account password. The email one receives to reset password looks like this :  

http://api.susi.ai/apps/resetpass/index.html?token={30 character long token}

*Original link contains a token of length of 30 characters. The link has been tampered for security purpose.

Taking a close look at the reset link, one would find it easy to decode. It simply contains path to an application on current SUSI Accounts hosting. Name of the application is “resetpass” for Reset Password. But what about the token in the link?

As soon as a user clicks on the link, the app is called and token is passed as a GET parameter. The app in background makes a call to the server where the token is evaluated for whether the token is hashed against some user’s email id and has not expired yet. Below is code of first step the client does which is to make a simple ajax call on Ready state.

$(document).ready(function()
{
	var passerr = false, confirmerr = false, tokenerr = false;

	// get password parameters
	var regex;
	var urltoken = getParameter('token');

	$.ajax(	"/aaa/recoverpassword.json", {
		data: { getParameters: true, token: urltoken },
		dataType: 'json',
		success: function (response) {
			regex = response.regex;
			var regexTooltip = response.regexTooltip;
			$('#pass').tooltip({'trigger':'focus', 'placement': 'left', 'title': regexTooltip});
			$('#status-box').text(response.message);
			tokenerr = false;
		},
		error: function (xhr, ajaxOptions, thrownError) {
			$('#status-box').text(thrownError);
			$('#status-box').addClass("error");
			$('#pass').prop( "disabled", true );
			$('#confirmpass').prop( "disabled", true );
			$('#resetbut').prop( "disabled", true );
			tokenerr = true;
		},
	});

As you can see the call is made to /aaa/recoverypassword.json end point with token as the request parameter. Now, the client has to just evaluate the JSON response and render the message for user accordingly. If this request returns an error then the error message is shown and the entries are disabled to enter the password. Otherwise, user email id is shown with green text and user can now enter new password and confirm it.

In next step, client simply evaluates the password and sends a query to server to reset it. Let us now look at how server functions and processes such requests.

//check if a token is already present
if (call.get("getParameters", false)) {
			if (call.get("token", null) != null && !call.get("token", null).isEmpty()) {
				ClientCredential credentialcheck = new ClientCredential(ClientCredential.Type.resetpass_token,
						call.get("token", null));
				if (DAO.passwordreset.has(credentialcheck.toString())) {
					Authentication authentication = new Authentication(credentialcheck, DAO.passwordreset);
					if (authentication.checkExpireTime()) {
						String passwordPattern = DAO.getConfig("users.password.regex", "^(?=.*\\d).{6,64}$");
						String passwordPatternTooltip = DAO.getConfig("users.password.regex.tooltip",
								"Enter a combination of atleast six characters");
						result.put("message", "Email ID: " + authentication.getIdentity().getName());
						result.put("regex", passwordPattern);
						result.put("regexTooltip", passwordPatternTooltip);
						result.put("accepted", true);
						return new ServiceResponse(result);
					}
					authentication.delete();
					throw new APIException(422, "Expired token");
				}
				throw new APIException(422, "Invalid token");
			} else {
				throw new APIException(422, "No token specified");
			}
		}

In the above code snippet, server evaluates the received token on the basis of three parameters. First it checks whether the token is provided or not. If not, it throws APIException with error code 422 and a message “No token specified”. If it passes the check, next it checks if the token passed is valid or not. If the token is invalid, it throws different APIException with same error code but different message “Invalid token”. Finally it checks whether the token is expired or not {life of each token is 7 days. After that, server marks it as expired}.

If all checks pass, the server finds the valid email id against which the token was hashed and sends it to user in JSON format. Now let us see how the final reset  password call is handled at the server.

If the token is valid  and still has life, user will be asked to enter new password and confirm it. Client locally checks whether new password and confirm password are same or not. It will now make a call to the below given servlet.

/aaa/resetpassword.json

Till now, server has already made the client identity using the token. Next it will check if the password matches regular expression or not. If not, it sends  an error message “Invalid Password” with error code 400. 

if (DAO.hasAuthentication(emailcred)) {
			Authentication emailauth = DAO.getAuthentication(emailcred);
			String salt = createRandomString(20);
			emailauth.remove("salt");
			emailauth.remove("passwordHash");
			emailauth.put("salt", salt);
			emailauth.put("passwordHash", getHash(newpass, salt));
		}

Above code snippet shows what happens when new password matches the conditions of regular expression. The server will generate a random string of 20 characters and use it as the new salt to hash the password. It updates the salt and password hash for the particular user. Next time whenever user makes a login request, server will use the new salt-hash pair to authorise the user. Below given is a flowchart for better understanding.

Resources

 

Continue ReadingReset Password for SUSI Accounts Using Verification Link