Showing RSS and Table Type Replies in SUSI Messenger Bots

All the messengers have a “plain text” reply support. To show web search (RSS) or table type replies, either:

  1. We need a “list type” (as in Facebook messenger) or “table type” reply support built in the messenger itself.
                                                                  or
  2. We need to convert the RSS or table type reply by SUSI API to plain text, so that we can send it, due to the “plain text” reply support available in almost every messenger.

The second point is the most favorable approach, as that way, replying with RSS or table type results is dependent only on the text support feature in the messenger. This way RSS or table type replies can be shown in messengers like REST API Gitter (which do not provide any other reply type support except text).

In SUSI web app, the UI of the web search and table type results:

As the SUSI web app is made in React js, it provided the app with necessary features to show the results this way. The messengers may not be having these required features.

So the task is we need to convert the RSS or table type replies by SUSI API to plain text to send it to the messenger.

Let’s work it out.

Converting RSS results to text:

First get familiar with the SUSI API reply to query “why” by visiting this url – http://api.susi.ai/susi/chat.json?q=why.

The JSON object returned will be constituting an array of objects as the value of the data key like:

"data": [
        {
        "title": "Why is Oracle male?",
        "description": "Why is Oracle male?. http dba oracle com why is male htm. Oracle Why is masculine?. ",
        "link": "http://dba-oracle.com/t_why_is_oracle_male.htm"
      }
]

If we notice carefully each object has 3 main keys namely “title”, “description” and “link”. So extracting these 3 properties from each object and binding them together into 1 string is the task we need to do.

So we traverse each object (i.e. rss result) in the data array and we keep on appending the values of “title”, “description” and “link” key values to the ans variable. At the end we send this resultant string to the messenger bot as a reply.  

Suppose we have the returned JSON object, in the data variable.

// storing the number of rss results
var metaCnt = data.answers[0].metadata.count;
    for(var i=0;i<metaCnt;i++){
        ans += ('Title : ');
ans += data.answers[0].data[i].title+', ';
        ans += ('Link : ');
        ans += data.answers[0].data[i].link+', ';
        ans += '\n\n';
    }

// send the message in ans variable to the messenger

Converting table type replies to text:

First get familiar with the SUSI API reply to query “why” by visiting this url – http://api.susi.ai/susi/chat.json?q=universities+in+australia.

The JSON object returned will be constituting an array of objects representing universities as the value of the data key in this form:

{
    "alpha_two_code": "AU",
    "name": "Australian Correspondence Schools",
    "country": "Australia",
    "web_page": "http://www.acs.edu.au/"
}

Here too, each object has 3 main keys namely “name”, “country” and “web_page”. So extracting these 3 properties from each object and binding them together into 1 string can make the things work.

Again traverse each object (i.e. university object) in the data array and we keep on appending the values of “name”, “country” and “web_page” key values to the ans variable. At the end we send this resultant string to the messenger bot as a reply.

Suppose we have the returned JSON object in the data variable.

    // the 3 main columns which we need are stored in colNames variable
    var colNames = data.answers[0].actions[0].columns;
    
    // storing the number of table entries
    var metaCnt = data.answers[0].metadata.count;
    for(var i=0;i<metaCnt;i++){
        for(var cN in colNames){
            // The column name
            ans += (colNames[cN]+' : ');
            // value for that column name
            ans += data.answers[0].data[i][cN]+', ';
        }
        ans += '\n\n';
    }

    // send the message in ans variable to the messenger

Resources

  1. By Slobodan Stojanović from smashing magazineDevelop a chat bot with node js.
  2. By Mikhail Larionov from Facebook blogsList templates and check box plugin
Continue ReadingShowing RSS and Table Type Replies in SUSI Messenger Bots

Generating the Mozilla All Hands Open Event Android App

The main aim of FOSSASIA Open Event Android App is to give an event organiser the ability to generate the app through a single click by providing the necessary json and binary files. The app was tested on the Mozilla All Hands 2017 event which is an annual conference held for Mozilla employees to showcase their experience and also interact with each other.  The sample files for the event can be found here. (https://github.com/fossasia/open-event/tree/master/sample/MozillaAllHands17). The data for the event was taken from here. (https://sanfranciscoallhandsjune2017.sched.com/).

What was required for building the sample for Mozilla All Hands 2017 event?

  • images folder containing the necessary images of speaker having support for local images too, the logo of the event, the background image of the event.
  • event json file which has all the event specific information like the name of the event, the schedule of the event, the description of the event etc.
  • forms json file having session and speaker form data.
  • meta json file having the root url of the event.
  • microlocations json file having all the locations where the sessions are going to happen.
  • session_types json file consisting data of all the type of session which will occur in the event (the length of session, the name of type and the id).
  • sessions json file consisting session specific data like the title of the session, start time and end time of session, which track that session belongs to etc.
  • speakers json file consisting of speaker specific data like the name of the speaker, image of the speaker, social links of the speaker etc.
  • sponsors json file consisting list of all sponsors of the event.
  • tracks json file consisting of tracks specific data.
  • config.json file which consists of the api url, app name.

After generating the required zip containing the above json files, the zip is uploaded to this site (http://droidgen.eventyay.com/) and we get an apk after filling the required information. This means all the event organiser is able to generate the sample app by providing the necessary information about the event.

The sample can be found here:

Folder Link: https://github.com/fossasia/open-event/tree/master/sample/MozillaAllHands17

Zip Link: https://github.com/fossasia/open-event/tree/master/sample/MozillaAllHands17.zip

How did the Mozilla All Hands 2017 sample app look like?

The screenshots of the sample apk can be seen below:

 

 

What were the issues found in the sample app?

As compared to the previous sample apk like Google IO Open Event Android App some issues like support for local speaker images, background issue of the logo were solved. However there were certain issues which we observed on testing the app with the Mozilla All Hands 2017 event:

  1. The theme of the app remains the same no matter which event it is. It is important to give the event organiser the ability to customise the theme of the app.
  2. Certain information in the app like the event information is hard-coded and needs to be taken from the assets folder instead of strings.xml.

Related Links

Continue ReadingGenerating the Mozilla All Hands Open Event Android App

Using Templates in SUSI FBbot

The SUSI AI Fbbot previously showed rss and table type replies as plain text to the user. To enhance the user experience, Facebook provides with templates which can be used in it’s messenger. Using those, we show rss and table type replies wrapped up in a better U.I. creating a better user experience.

The 4 basic template structures that can be used for this task are:

  1. Button template
  2. Generic template
  3. List template
  4. Receipt template

List template is used in SUSI A.I. Fbbot because rss reply and table type reply both are lists of data.
The basic syntax for list template with reference to Fb official docs is:

"message": {
    "attachment": {
        "type": "template",
        "payload": {
            "template_type": "list",
            "top_element_style": "compact",
            "elements": [
                {
                    "title": "Classic White T-Shirt",
                    "subtitle": "100% Cotton, 200% Comfortable",
                    "default_action": {
                        "type": "web_url",
                        "url": "https://peterssendreceiveapp.ngrok.io/view?item=100"
                    },
                    "buttons": [
                        {
                            "title": "Buy",
                            "type": "web_url",
                            "url": "https://peterssendreceiveapp.ngrok.io/shop?item=100"                     
                        }
                    ]                
                }
            ]
        }
    }
}

This code shows a reply to the user like this:

If we want to show a “View more” button at the end of the list, we can add a “buttons” key and an array as its value which will have information regarding all the buttons we want to show.

The code below will show a “View more” button at the end of the list:

"elements": [
            {
                // all the elements, like shirt in the above case               
            }
       ],
       "buttons": [
            {
                "title": "View More",
                "type": "postback",
                "payload": "payload"                        
               }
       ]

Let’s learn how to incorporate these features to rss results in SUSI Fbbot:

When sending a reply using list template, “elements” key must have an array type value which will be constituted of list items. Therefore, we need to push all the rss results into that array and set it as the value of the “elements” key.

Let’s develop the code part:

Fetch the JSON object from SUSI API url with query as “why” i.e. http://api.susi.ai/susi/chat.json?q=why. Let body be a variable which stores the returned JSON object.

  • The below code fills up an array (namely elementsVal) with all the rss results:
var elementsVal = [];
var metaCnt = body.answers[0].metadata.count;
for(var i=0;i<((metaCnt>4)?4:metaCnt);i++){
    elementsVal.push(
        {
            "title": body.answers[0].data[i].title,
            "subtitle": body.answers[0].data[i].link
        }
    );
}
  • Setting the elementsVal array as the value of the “elements” key:
var message = {
    "type": "template",
    "payload": 
    {
        "template_type": "list",
        "top_element_style": "compact",
        "elements": elementsVal
    }
};
  • Sending this message as a reply to the user:
sendTextMessage(sender, message);

Same procedure can be used to render table type replies in the bot using the list template.

Generic template provided by Facebook can also used to render the web and table type replies. This template helps in showing the results in square boxes, which can be changed using left or right arrows.

Resources:

  1. By Mikhail Larionov from Facebook blogsList templates and check box plugin
  2. By Slobodan Stojanović from smashing magazineDevelop a chat bot with node js.
Continue ReadingUsing Templates in SUSI FBbot

Using Jackson Library in Open Event Android App for JSON Parsing

Jackson library is a popular library to map Java objects to JSON and vice-versa and has a better parsing speed as compared to other popular parsing libraries like GSON, JSONP etc. This blog post gives a basic explanation on how we utilised the Jackson library in the Open Event Android App.

To use the Jackson library we need to add the dependency for it in the app/build.gradle file.

compile 'com.squareup.retrofit2:converter-jackson:2.2.0'

After updating the app/build.gradle file with the code above we were able to use Jackson library in our project.

Example of Jackson Library JSON Parsing

To explain how the mapping is done let us take an example. The file given below is the sponsors.json file from the android app.

[
    “description”: “”,
    “id”: 1,
    “level”: 3,
    “name”:  KI Group,
    “type”: Gold,
    “url: “”,
    “logo-url”: “” 
]

The sponsors.json consists of mainly 7 attributes namely description, id, level, name, type, url and logo url for describing one sponsor of the event. The Sponsors.java file for converting the json response to Java POJO objects was done as follows utilizing the Jackson library:

public class Sponsors extends RealmObject {
    @JsonProperty(“id”)
    private int id;

    @JsonProperty(“type”)
    private String type;

    @JsonProperty(“description”)
    private String description;

    @JsonProperty(“level”)
    private String level;

    @JsonProperty(“name”)
    private String name;

    @JsonProperty(“url”)
    private String url;

    @JsonProperty(“logo-url”)
    private String logoUrl;
}

As we can see from the above code snippet, the JSON response is converted to Java POJO objects simply by using the annotation “@JsonProperty(“”)” which does the work for us.

Another example which makes this library amazing are the setter and getter annotations which help us use a single variable for two different json attributes if need be. We faced this situation when we were moving from the old api to the new json api. In that case we wanted support for both, old and new json attributes. In that case we simply used the following code snippet which made our transition to new api easier.

public class Sponsors extends RealmObject {
    private int id;
    private String type;
    private String description;
    private String level;
    private String name;
    private String url;
    private String logoUrl;

    @JsonSetter(“logo”)
    public void setLogo(String logoUrl) {
        this.logoUrl = logoUrl;
    }

    @JsonSetter(“logo-url”)
    public void setLogo(String logoUrl) {
        this.logoUrl = logoUrl;
    }
}

As we can see the setter annotations allow easy naming of variable to multiple attributes if need be thus making the code easily adaptable with less overload.

Related Links:

Continue ReadingUsing Jackson Library in Open Event Android App for JSON Parsing

Using Flask SocketIO Library in the Apk Generator of the Open Event Android App

Recently Flask SocketIO library was used in the apk generator of the Open Event Android App as it gave access to the low latency bi-directional communications between the client and the server side. The client side of the apk generator was written in Javascript which helped us to use a SocketIO official client library to establish a permanent connection to the server.

The main purpose of using the library was to display logs to the user when the app generation process goes on. This gives the user an additional help to check what is the mistake in the file uploaded by them in case an error occurs. Here the library established a connection between the server and the client so that during the app generation process the server would send real time logs to the client so that they can be viewed by the user displayed in the frontend.

To use this library we first need to download it using pip command:

pip install flask-socketio

This library depends on the asynchronous services which can be selected amongst the following listed below:

  1. Eventlet
  2. Gevent
  3. Flask development server based on Werkzeug

Amongst the three listed, eventlet is the best performant option as it has support for long polling and WebSocket transports.

The next thing was importing this library in the flask application i.e the apk generator of the app as follows:

from flask_socketio import SocketIO

current_app = create_app()
socketio = SocketIO(current_app)

if __name__ == '__main__':
    socketio.run(current_app)

The main use of the above statement (socket.run(current_app)) is that with this the startup of the web server is encapsulated. When the application is run in debug mode it is preferred to use the Werkzeug development server. However in production mode the use of eventlet web server or gevent web server is recommended.

We wanted to show the status messages currently shown to the user in the form of logs. For this firstly the generator.py file was looked upon which had the status messages and these were sent to the client side of the generator by establishing a connection between them using this library. The code written on the server side to send messages to the client side was as follows:

def handle_message(message):
    if message not None:
        namespace = “/” + self.identifier;
        send(message, namespace = namespace)

As we can see from the above code the message had to be sent to that particular namespace. The main idea of using namespace was that if there were more than one users using the generator at the same time, it would mean the send() method would send the messages to all the clients connected which would lead to the mixing up of status messages and hence it was important to use namespace so that a message was sent to that particular client.

Once the server sent the messages to the client we needed to add functionality to the our client side to receive the messages from them and also update the logs area with suitable content received. For that first the socket connection was established once the generate button was clicked which generated the identifier for us which had to be used as the namespace for that process.

socket = io.connect(location.protocol + "//" + location.host + "/" + identifier, {reconnection: false});

This piece of code established the connection of the socket and helps the client side to receive and send messages from and to the server end.

Next thing was receiving messages from the server. For this the following code snippet was added:

socket.on('message', function(message) {
    $buildLog.append(message);
});

This piece of code receives the messages from the server for unnamed events. Once the connection is established and the messages are received, the logs are updated with each message being appended so as to show the user the real time information about their build status of their app.

This was how the idea of logs being shown in the apk generator was incorporated by making the required changes in the server and client side code using the Flask SocketIO library.

Related Links:

Continue ReadingUsing Flask SocketIO Library in the Apk Generator of the Open Event Android App

How to Implement Memory like Servlets in SUSI.AI

In this blog, I’ll be discussing about how a server gets previous messages from the Log files in SUSI server. SUSI AI clients, Android, iOS and web chat, follow a very simple rule. Whenever a user logs in to the app, the app makes a http GET call to the server in the background and in response, server returns the chat history.

Link to the API endpoint -> http://api.susi.ai/susi/memory.json

But parsing a lot of data might depend on the connection speed. If the connection is poor or lacking speed, the history would cost user’s time. To prevent this, server by default returns last 10 pair of messages. It is up to the client that how many messages they want to render. So for example, if the client requests last 5 messages, then the client has to make a GET request and pass the cognitions parameter. Hence the modified end point will be :

http://api.susi.ai/susi/memory.json?cognitions=2

But how does the server process it? Let us see.
Browse to susi_server/src/ai/susi/server/api/susi/UserService.java file. This is the main working servlet. If you are new and wondering how servlets for susi are implemented, Please go through this first how-to-add-a-new-servletapi-to-susi-server
This is how serviceImpl() method looks like :

@Override
    public ServiceResponse serviceImpl(Query post, HttpServletResponse response, Authorization user, final JsonObjectWithDefault permissions) throws APIException {

        int cognitionsCount = Math.min(10, post.get("cognitions", 10));
        String client = user.getIdentity().getClient();
        List<SusiCognition> cognitions = DAO.susi.getMemories().getCognitions(client);
        JSONArray coga = new JSONArray();
        for (SusiCognition cognition: cognitions) {
            coga.put(cognition.getJSON());
            if (--cognitionsCount <= 0) break;
        }
        JSONObject json = new JSONObject(true);
        json.put("cognitions", coga);
        return new ServiceResponse(json);
    }

In the first step, we find the minimum of default value (i.e. 10) and the value of cognitions received as GET parameter. Messages equivalent to minimum variable are encoded in JSONArray and sent to the client.

Whenever the server receives a valid signup request, It makes a directory with the name “email_emailid”. In this directory, a log.txt file is maintained which stores all the queries along with the other details associated with it. For example if user has signed up with the email id example@example.com, Then the path of this directory will be /data/susi/email_example@example.com. If a user queries “http://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=flip+a+coin”,  then

{
	"query": "flip a coin",
	"count": 1,
	"client_id": "",
	"query_date": "2017-06-30T12:22:05.918Z",
	"answers": [{
		"data": [{
			"0": "flip a coin",
			"token_original": "coin",
			"token_canonical": "coin",
			"token_categorized": "coin",
			"timezoneOffset": "-330",
			"answer": "tails",

			"skill": "/susi_skill_data/models/general/entertainment/en/flip_coin.txt",
			"_etherpad_dream": "cricket"
		},
		"metadata": {
			"count": 1
		},
		"actions": [{
			"type": "answer",
			"expression": "tails"
		}],
		"skills": ["/susi_skill_data/models/general/entertainment/en/flip_coin.txt"]
	}],
	"answer_date": "2017-06-30T12:22:05.928Z",
	"answer_time": 10,
	"language": "en"
}

The server has user’s identity. It will use this identity and store (will be appended) in the respective log file.

The next steps in retrieving the message are pretty easy and includes getting the identity of the current user session. Use this identity to populate the JSONArray named coga. This is finally encoded in a JSONObject along with other basic details and returned to the clients where they render the data received, and show the messages in an appropriate way.

Resources

Continue ReadingHow to Implement Memory like Servlets in SUSI.AI

Writing Tests for ISO8601Date.java class of the Open Event Android App

ISO8601Date.java class of Open Event Android App was a util class written to perform the date manipulation functions and ensure the code base got more simpler and deterministic. However it was equally important to test the result from this util class so as to ensure the result returned by it was what we wanted. A test class named “DateTest.java” was written to ensure all the edge cases of conversion of the dates string from one timezone to another timezone were handled properly.

For writing unit tests, first we needed to add these libraries as dependencies in the app’s top level build.gradle file as shown below:

dependencies {
  testCompile 'junit:junit:4.12'
}

Then a JUnit 4 Test class, which was a Java class containing the required test methods was created. The structure of the class looked like this:

public class DateTest {
   @Test
   public void methodName() {
   }
}

Next step was including all the required methods which ensured the util class returned the correct results according to our needs. Various edge cases were taken into account by including functions like converting of date string from local time zone to specified timezone, from international timezone to local timezone, from local timezone to international timezone and many more. Some of the methods which were added in the class are shown below:

  • Test of conversion from local timezone to specified timezone:

This function aimed at ensuring the util class worked well with date conversion from local time zone to a specified timezone. An example as shown below was taken where the conversion of the date string was tested from UTC timezone to Singapore timezone.

@Test
public void shouldConvertLocalTimeZoneDateStringToSpecifiedTimeZoneDateString() {
    ISO8601Date.setTimeZone("UTC");
    ISO8601Date.setEventTimeZone("Asia/Singapore");

    String dateString1 = "2017-06-02T07:59:10Z";
    String actualString = ISO8601Date.getTimeZoneDateStringFromString(dateString1);
    String expectedString = "Thu, 01 Jun 2017, 23:59, UTC";
    Assert.assertEquals(expectedString, actualString);
}
  • Test of conversion from local timezone to international timezone:

This function aimed at ensuring the util class worked well with the date conversion from local timezone to international timezone. An example as shown below was taken where the conversion of the date string was tested from Amsterdam timezone to Singapore timezone.

@Test
public void shouldConvertLocalTimeZoneDateStringToInternationalTimeZoneDateString() {
    ISO8601Date.setTimeZone("Europe/Amsterdam");
    ISO8601Date.setEventTimeZone("Asia/Singapore");

    String dateString = "2017-06-02T02:29:10Z";
    String actualString = ISO8601Date.getTimeZoneDateStringFromString(dateString);
    String expectedString = "Thu, 01 Jun 2017, 20:29, GMT+02:00";
    Assert.assertEquals(expectedString, actualString);
}


Above were some functions added to ensure the conversion of a date string from one timezone to another was correct and thus ensured the util class was working properly and returned the results as required.

The last thing left was running the test to check the results the util class returned. For this we had to do two things:

  1. Sync the project with Gradle.
  2. Run the test by right clicking on the class and selecting “Run” option.

Through this we were able to run the test and check the output of the util class on different cases through the results which could be seen on the Android Monitor in the Android Studio.

Related Links:

  1. This link is about building effective unit tests in android. (https://developer.android.com/training/testing/unit-testing/index.html)
  2. This link is about the unit testing on date processing. (https://stackoverflow.com/questions/565289/unit-testing-code-that-does-date-processing-based-on-todays-date)
Continue ReadingWriting Tests for ISO8601Date.java class of the Open Event Android App

Expandable ListView In PSLab Android App

In the PSLab Android App, we show a list of experiments for the user to perform or refer to while performing an experiment, using PSLab hardware device. A long list of experiments need to be subdivided into topics like Electronics, Electrical, School Level, Physics, etc. In turn, each category like Electronics, Electrical, etc can have a sub-list of experiments like:

  • Electronics
    • Diode I-V characteristics
    • Zener I-V characteristics
    • Transistor related experiments
  • Electrical
    • Transients RLC
    • Bode Plots
    • Ohm’s Law

This list can continue in similar fashion for other categories as well. We had to  display  this experiment list to the users with a good UX, and ExpandableListView seemed the most appropriate option.

ExpandableListView is a two-level listView. In the Group view an individual item can be expanded to show it’s children. The Items associated with ExpandableListView come from ExpandableListAdapter.

 

 

 

 

 

 

 

 

Implementation of Experiments List Using ExpandableListView

First, the ExpandableListView was declared in the xml layout file inside some container like LinearLayout/RelativeLayout.

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:orientation="vertical">
   <ExpandableListView
       android:id="@+id/saved_experiments_elv"
       android:layout_width="match_parent"
       android:layout_height="wrap_content"
       android:divider="@color/colorPrimaryDark"
       android:dividerHeight="2dp" />
</LinearLayout>

Then we populated the data onto the ExpandableListView, by making an adapter for ExpandableListView by extending BaseExpandableListAdapter and implementing its methods. We then passed a Context, List<String> and Map<String,List<String>> to the Adapter constructor.

Context: for inflating the layout

List<String>: contains titles of unexpanded list

Map<String,List<String>>: contains sub-list mapped with title string

public SavedExperimentAdapter(Context context,
                                 List<String> experimentGroupHeader,
                                 HashMap<String, List<String>> experimentList) {
       this.context = context;
       this.experimentHeader = experimentGroupHeader;
       this.experimentList = experimentList;
   }

In getGroupView() method, we inflate, set title and return group view i.e the main list that we see on clicking and the  sub-list is expanded. You can define your own layout in xml and inflate it. For PSLab Android, we used the default one provided by Android

 android.R.layout.simple_expandable_list_item_2
@Override
public View getGroupView(int groupPosition, boolean isExpanded, View convertView, ViewGroup parent) {
   String headerTitle = (String) getGroup(groupPosition);
   if (convertView == null) {
       LayoutInflater inflater = (LayoutInflater) this.context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
       convertView = inflater.inflate(android.R.layout.simple_expandable_list_item_2, null);
   }
   TextView tvExperimentListHeader = (TextView) convertView.findViewById(android.R.id.text1);
   tvExperimentListHeader.setTypeface(null, Typeface.BOLD);
   tvExperimentListHeader.setText(headerTitle);
   TextView tvTemp = (TextView) convertView.findViewById(android.R.id.text2);
   tvTemp.setText(experimentDescription.get(groupPosition));
   return convertView;
}

Similarly, in getChildView() method, we inflate, set data and return child view. We wanted simple TextView as sub-list item thus inflated the layout containing only TextView and setText by taking reference of textView from the inflated view.

@Override
public View getChildView(int groupPosition, int childPosition, boolean isLastChild, View convertView, ViewGroup parent) {
   String experimentName = (String) getChild(groupPosition, childPosition);
   if (convertView == null) {
       LayoutInflater inflater = (LayoutInflater) this.context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
       convertView = inflater.inflate(R.layout.experiment_list_item, null);
   }
   TextView tvExperimentTitle = (TextView) convertView.findViewById(R.id.exp_list_item);
   tvExperimentTitle.setText(experimentName);
   return convertView;
}

The complete code for the Adapter can be seen here.

After creating the adapter we proceeded similarly to the normal ListView. Take the reference for ExpandableListView by findViewById() or BindView if you are using ButterKnife and set the adapter as an instance of adapter created above.

@BindView(R.id.saved_experiments_elv)
ExpandableListView experimentExpandableList;
experimentAdapter = new SavedExperimentAdapter(context, headerList, map);
experimentExpandableList.setAdapter(experimentAdapter);
Source: PSLab Android

Roadmap

We are planning to divide the experiment sub-list into categories like

  • Electronics
    • Diode
      • Diode I-V
      • Zener I-V
      • Diode Clamping
      • Diode Clipping
    • BJT and FET
      • Transistor CB (Common Base)
      • Transistor CE (Common Emitter)
      • Transistor Amplifier
      • N-FET output characteristic
    • Op-Amps
  • Electrical

This is a bit more complex than it looks, I tried using an ExpandableListView as a child for a group item but ran into some errors. I will write a post as soon as this view hierarchy has been achieved.

Resources

Continue ReadingExpandable ListView In PSLab Android App

Integration of SUSI AI to Facebook

It’s easy to create your own SUSI AI Facebook messenger bot. You can read the official documentation by Facebook to get started.

Messenger bots use a web server to send and receive messages. You also need to have the bot be authenticated to speak with the web server and be approved by Facebook before getting public.

If any problems faced, visit the susi_fbbot repository which hosts the code for SUSI Facebook Messenger bot.

We will be using Node js technology to develop the FB bot. First, let’s see on how to request an answer from the SUSI API.

To call Susi API and fetch an answer from it for a query (‘hi’ in this case). Let’s first visit http://api.asksusi.com/susi/chat.json?q=hi from the browser. We will get a JSON object as follows:

The answer can be found as the value of the key named expression. In this case, it is “Hallo!”.

To fetch the answer through coding, we can use this code snippet in Node js:

// including request module
var request = require(‘request’);

// setting options to make a successful call to Susi API.
var options = {
            method: 'GET',
            url: 'http://api.asksusi.com/susi/chat.json',
            qs: 
            {
                timezoneOffset: '-330',
                q: 'hi'
            }
        };

// A request to the Susi bot
request(options, function (error, response, body) {
    if (error)
        throw new Error(error);
    // answer fetched from susi
    ans = (JSON.parse(body)).answers[0].actions[0].expression;
}

The properties required for the call are set up through a JSON object (i.e. options). Pass the options object to our request function as its 1st parameter. The response from the API will be stored in ‘body’ variable. We need to parse this body, to be able to access the properties of that body object. Hence, fetching the answer from Susi API.

Now let’s dive into the code of receiving and messaging back to the user on Facebook:

  • A generic function to send a message to the user.
// the first argument is the sender id and the second is the text to send.
function sendTextMessage(sender, text) {
    var messageData = { text:text };
    
    // making a post request to facebook graph API to send message.
    request({
        url: 'https://graph.facebook.com/v2.6/me/messages',
        qs: {access_token:token},
        method: 'POST',
        json: {
            recipient: {id:sender},
            message: messageData,
        }
    }, function(error, response, body) {
        if (error) {
            console.log('Error sending messages: ', error);
        } else if (response.body.error) {
            console.log('Error: ', response.body.error);
        }
    });
}
  • According to step 9, in the below instructions we need to include this code snippet too:
// for facebook verification
app.get('/webhook/', function (req, res) {
    if (req.query['hub.verify_token'] === 'my_voice_is_my_password_verify_me') {
        res.send(req.query['hub.challenge']);
    }
    res.send('Error, wrong token');
});
  • When user messages to our bot, we need to extract the text of the message from the request body. Then we need to extract the reply from the SUSI API and send it back to the user.
// when a message from a user is received.
app.post('/webhook/', function (req, res) {
    var messaging_events = req.body.entry[0].messaging
    for (var i = 0; i < messaging_events.length; i++) {
        // fetching the current event
        var event = req.body.entry[0].messaging[i];

        // fetching the sender id
        var sender = event.sender.id;
        
        // if the event is a message event
        if (event.message && event.message.text) {
            var text = event.message.text;

            // Construct the query for susi
            var queryUrl = 'http://api.susi.ai/susi/chat.json?q='+encodeURI(text);
            var message = '';

            // Wait until done then reply
            request({
                url: queryUrl,
                json: true
            }, function (error, response, body) {
                if (!error && response.statusCode === 200) {
              // fetch the answer from the response body and save it in message variable.
                    // send the reply
                    sendTextMessage(sender, message);
                } 
                
            // if, due some reasons the answer couldn’t be fetched
            else {
                    message = 'Oops, Looks like Susi is taking a break, She will be back soon';
                    sendTextMessage(sender, message);
                }
            });
        }
        if (event.postback) {
            var text = JSON.stringify(event.postback);
            sendTextMessage(sender, "Postback received: "+text.substring(0, 200), token);
            continue;
        }
    }
    res.sendStatus(200)
})

Upload the code developed to a repository.

Let’s follow the below steps, to achieve a working fb messenger bot:

  1. Create a Facebook page here.

    Creating a FB Page
    New FB Page

    1. Create a new Heroku app here.

    New Heroku App

    1. Connect the Heroku app to the repository hosting our code.

    Connect to Github

    4. Deploy on the development branch. If you intend to contribute, it is recommended to Enable Automatic Deploys.

    Branch Deployment

    Successful Deployment

    5. Create or configure a Facebook App or Page here.

    New FB App

    6. Get started with Messenger tab in the created app.


    7.
    In the Page Access Token select the FB page that you created and generate the token and save it somewhere for future use.

    Token Generation

    8. Now, go to the Heroku app, select the settings tab and add the environment variable as shown, where the key is FB_PAGE_ACCESS_TOKEN and value is the token generated in the previous step.

    9. Create a web hook on the facebook app dashboard. The Callback URL should be https://<your_app_name>.herokuapp.com/webhook/ and rest should be as shown in the image below.

    App Complete

    10. Go to Terminal and type in this command to trigger the Facebook app to send messages. Remember to use the token you requested earlier.
    “`
    curl -X POST “https://graph.facebook.com/v2.6/me/subscribed_apps?access_token=<PAGE_ACCESS_TOKEN>”
    “`

    11. Go to the Facebook page created and locate ‘Message Now’ or go to https://m.me/PAGE_USERNAME

    12. Enjoy chatting with Susi.

    Resources

Continue ReadingIntegration of SUSI AI to Facebook

Implementing Stickers on an Image in Phimpme

One feature we implemented in the Phimpme photo editing application is to enable users to add stickers on top of the image. In this blog, I will explain how stickers are implemented in Phimpme.

Features of Stickers

  • Users can resize the stickers.
  • Users can place the stickers wherever in the canvas.

Step.1-Storing the Stickers in assets folder

In Phimpme I stored the stickers in the assets folder. To distribute the stickers in different categories I made different folders according to the categories namely type1, type2, type3, type4 and so on.  

Displaying stickers

We used onBindViewHolder to Display the stickers in different categories like:

  • Facial
  • Express
  • Objects
  • Comments
  • Wishes
  • Emojis
  • Hashtags

String path will get the position of the particular type of stickers collection. This type is then loaded to the ImageLoader with the specific icon associating with the types.   

@Override
public void onBindViewHolder(mRecyclerAdapter.mViewHolder holder, final int position) {

   String path = pathList.get(position);
       ImageLoader.getInstance().displayImage("assets://" + path,holder.icon, imageOption);
       holder.itemView.setTag(path);
       holder.title.setText("");

   int size = (int) getActivity().getResources().getDimension(R.dimen.icon_item_image_size_filter_preview);
   LinearLayout.LayoutParams layoutParams = new LinearLayout.LayoutParams(size,size);
   holder.icon.setLayoutParams(layoutParams);

   holder.itemView.setOnClickListener(new View.OnClickListener() {
       @Override
       public void onClick(View v) {
           String data = (String) v.getTag();
           selectedStickerItem(data);
       }
   });
}

Step.2- Applying a sticker on the image

When a particular sticker is selected selectedStickerItem() function is called.This function calls StickerView class to add the Bitmap on the image. It sends the path of the sticker as a parameter.  

public void selectedStickerItem(String path) {
   mStickerView.addBitImage(getImageFromAssetsFile(path));
}

In StickerView class the image of the sticker is then converted into a Bitmap. It creates an object(item) of StickerItem class. This object calls the init function, which handles the size of the sticker and the placement of the sticker on the image.

public void addBitImage(final Bitmap addBit) {
   StickerItem item = new StickerItem(this.getContext());
   item.init(addBit, this);
   if (currentItem != null) {
       currentItem.isDrawHelpTool = false;
   }
   bank.put(++imageCount, item);
   this.invalidate();
}

Step.3-Resizing the Sticker in the canvas

A bitmap or any image has two axes namely x and y. We can resize the image using matrix calculation.

float c_x = dstRect.centerX();
float c_y = dstRect.centerY();

float x = this.detectRotateRect.centerX();
float y = this.detectRotateRect.centerY();

We then calculate the source length and the current length:

float srcLen = (float) Math.sqrt(xa * xa + ya * ya);
float curLen = (float) Math.sqrt(xb * xb + yb * yb);

Then we calculate the scale. This is required to calculate the zoom ratio.

float scale = curLen / srcLen;

We need to rescale the bitmap. That is if the user rotates the sticker or zoom in or zoom out the sticker. A helpbox surrounds the stickers showing the actual size of the sticker. This helpbox which is rectangular shape helps in resizing the sticker.

RectUtil.scaleRect(this.dstRect, scale);// Zoom destination rectangle

// Recalculate the Toolbox coordinates
helpBox.set(dstRect);
updateHelpBoxRect();// Recalculate
rotateRect.offsetTo(helpBox.right - BUTTON_WIDTH, helpBox.bottom
       - BUTTON_WIDTH);
deleteRect.offsetTo(helpBox.left - BUTTON_WIDTH, helpBox.top
       - BUTTON_WIDTH);

detectRotateRect.offsetTo(helpBox.right - BUTTON_WIDTH, helpBox.bottom
       - BUTTON_WIDTH);
detectDeleteRect.offsetTo(helpBox.left - BUTTON_WIDTH, helpBox.top
       - BUTTON_WIDTH);

Conclusion

In Phimpme a user can now place the sticker on top of the image. Resize the sticker, that is Zoom in the image or zoom out of the image. Move the image around the canvas. This will give users the flexibility to add multiple stickers on the image.

Github

Resources

Continue ReadingImplementing Stickers on an Image in Phimpme