FOSSASIA

Read more about the article Add Autocomplete SearchView in Open Event Android App

Add Autocomplete SearchView in Open Event Android App

The Open Event Android App has a map for showing all locations of sessions. All the locations have a marker in the map. It is difficult to find a particular location on the map because to know the name of location user has to click on the marker. Adding autocomplete SearchView will improve user experience by providing an ability to search the location by name and by suggesting name according to the search query. In this post I explain how to add autocomplete SearchView in the fragment or activity.

Add search icon in actionbar

The first step to do is to create a menu xml file and add a search menu item in it. Then inflate this menu xml file in Fragment in onCreateOptionsMenu() method.

1. Create menu.xml file

In this file add search menu element. Inside menu element add search menu item. Define id, title, and icon of search menu item. Add android.support.v7.widget.SearchView” as actionViewClass which will be used as action view when the user clicks on the icon.

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools">
    
   <item
        android:id="@+id/action_search"
        android:icon="@drawable/ic_search_white_24dp"
        android:title="@string/search"
        app:actionViewClass="android.support.v7.widget.SearchView"
        app:showAsAction="ifRoom | collapseActionView"/>
</menu>

2. Inflate menu.xml file in Fragment

In the fragment’s onCreateOptionsMenu() method inflate menu.xml file using MenuInflater’s inflate() method. Then find search menu item using menu’s findItem() method by passing id of search menu item as parameter.

public void onCreateOptionsMenu(Menu menu, MenuInflater inflater) {
        super.onCreateOptionsMenu(menu, inflater);
        inflater.inflate(R.menu.menu_map, menu);
        MenuItem item = menu.findItem(R.id.action_search);
}

Add and initialize SearchView  

Now after adding search icon we need to add SearchView and SearchAutoComplete fields in the fragment.

private SearchView searchView;
private SearchView.SearchAutoComplete   mSearchAutoComplete;

Initialize SearchView in onCreateOptionMenu() method by passing search menu item in the getActionView() method of MenuItemCompat.

Here SearchAutoComplete is a child object of SearchView so initialize it using findViewById method of SearchView by passing the id as parameter.

searchView = (SearchView) MenuItemCompat.getActionView(item);
mSearchAutoComplete = (SearchView.SearchAutoComplete) searchView.findViewById(android.support.v7.appcompat.R.id.search_src_text);

Define properties of SearchAutoCompleteView

By default background of drop down menu in SearchAutoComplete is black. You can change background using setDropDownBackgroundResource() method. Here i’m making it white by providing white drawable resource.

mSearchAutoComplete.setDropDownBackgroundResource(R.drawable.background_white);
mSearchAutoComplete.setDropDownAnchor(R.id.action_search);
mSearchAutoComplete.setThreshold(0)

The setDropDownAnchor() method sets the view to which the auto-complete drop down list should anchor. The setThreshold() method specifies the minimum number of characters the user has to type in the edit box before the drop down list is shown.

Create array adapter

Now it’s time to make the ArrayAdapter object which will provide the data set (strings) which will be used to run search queries.

ArrayAdapter<String> adapter = new ArrayAdapter<>(getActivity(), android.R.layout.simple_list_item_1, searchItems);

Here searchItems is List of strings. Now set this adapter to the mSearchAutoComplete object using setAdapter() method.

mSearchAutoComplete.setAdapter(adapter);

Now we are all set to run the app on device or emulator. Here’s demo how it will look

Conclusion

The SearchView with an ability to give suggestions serves the great user experience in the application.

Additional resources:

Continue ReadingAdd Autocomplete SearchView in Open Event Android App
Read more about the article Simplifying Scrapers using BaseScraper

Simplifying Scrapers using BaseScraper

Loklak Server‘s main function is to scrape data from websites and other sources and output in different formats like JSON, xml and rss. There are many scrapers in the project that scrape data and output them, but are implemented with different design and libraries which makes them different from each other and a difficult to fix changes.

Due to variation in scrapers’ design, it is difficult to modify them and fix the same issue (any issue, if it appears) in each of them. This issue signals fault in design. To solve this problem, Inheritance can be brought into application. Thus, I created BaseScraper abstract class so that scrapers are more concentrated on fetching data from HTML and all supportive tasks like creating connection with the help of url are defined in BaseScraper.

The concept is pretty easy to implement, but for a perfect implementation, there is a need to go through the complete list of tasks a scraper does.

These are the following tasks with descriptions and how they are implemented using BaseScraper:

  1. Endpoint that triggers the scraper

Every search scraper inherits class AbstractAPIHandler. This is used to fetch get parameters from the endpoint according to which data is scraped from the scraper. The arguments from serviceImpl method is used to generate output and is returned to it as JSONObject.

For this task, the method serviceImpl has been defined in BaseScraper and method getData is implemented to return the output. This method is the driver method of the scraper.

public JSONObject serviceImpl(Query call, HttpServletResponse response, Authorization rights, JSONObjectWithDefault permissions) throws APIException {
    this.setExtra(call);
    return this.getData().toJSON(false, "metadata", "posts");
}

 

  1. Constructor

The constructor of Scraper defines the base URL of the website to be scraped, name of the scraper and data structure to fetch all get parameters input to the scraper. For get parameters, the Map data structure is used to fetch them from Query object.

Since every scraper has it’s own different base URL, scraper name and get parameters used, so it is implemented in respective Scrapers. QuoraProfileScraper is an example which has these variables defined.

  1. Get all input variables

To get all input variables, there are setters and getters defined for fetching them as Map from Query object in BaseScraper. There is also an abstract method getParam(). It is defined in respective scrapers to fetch the useful parameters for scraper and set them to the scraper’s class variables.

// Setter for get parameters from call object
protected void setExtra(Query call) {
    this.extra = call.getMap();
    this.query = call.get("query", "");
    this.setParam();
}

// Getter for get parameter wrt to its key
public String getExtraValue(String key) {
    String value = "";
    if(this.extra.get(key) != null) {
        value = this.extra.get(key).trim();
    }
    return value;
}

// Defination in QuoraProfileScraper
protected void setParam() {
    if(!"".equals(this.getExtraValue("type"))) {
        this.typeList = Arrays.asList(this.getExtraValue("type").trim().split("\\s*,\\s*"));
    } else {
        this.typeList = new ArrayList<String>();
        this.typeList.add("all");
        this.setExtraValue("type", String.join(",", this.typeList));
    }
}

 

  1.  URL creation for web scraper

The URL creation shall be implemented in a separate method as in TwitterScraper. The following is the rough implementation adapted from one of my pull request:

protected String prepareSearchUrl(String type) {
    URIBuilder url = null;
    String midUrl = "search/";

    try {
        switch(type) {
            case "question":
                url = new URIBuilder(this.baseUrl + midUrl);
                url.addParameter("q", this.query);
                url.addParameter("type", "question");
        .
        .
    }
    .
    .
    return url.toString();
}

 

  1. Get BufferedReader object from InputStream

getDataFromConnection method fetches the BufferedReader object from ClientConnection. This object reads the web page line by line by the scrape method to fetch data. See here.

ClientConnection connection = new ClientConnection(url);
BufferedReader br = getHtml(connection);
.
.
.
public BufferedReader getHtml(ClientConnection connection) {

    if (connection.inputStream == null) {
        return null;
    }

    BufferedReader br = new BufferedReader(new InputStreamReader(connection.inputStream, StandardCharsets.UTF_8));
    return br;
}

 

  1. Scraping of data from HTML

The Scraper method for scraping data is declared abstract in BaseScraper and defined in the scraper. This can be a perfect example of implementation for BaseScraper (See code the here) and scraper (here).

  1. Output of data

The output of scrape method is fetched in Post data objects that are implemented for the respective scraper. These Post objects are added to Timeline iterator and which outputs data as JSONArray. Later the objects are output in enclosed Post object wrapper.

This data can be directly output as Post object, but adding it to iterator makes the Post Objects capable to be sorted in an order and be indexed to ElasticSearch.

 

Resources

Continue ReadingSimplifying Scrapers using BaseScraper
Read more about the article Iterating the Loklak Server data
Iterating the Loklak Server data

Iterating the Loklak Server data

Loklak Server is amazing for what it does, but it is more impressive how it does the tasks. Iterators are used for and how to use them, but this project has a customized iterator that iterates Twitter data objects. This iterator is Timeline.java .

Timeline implements an interface iterable (isn’t it iterator?). This interface helps in using Timeline as an iterator and add methods to modify, use or create the data objects. At present, it only iterates Twitter data objects. I am working on it to modify it to iterate data objects from all web scrapers.

The following is a simple example of how an iterator is used.

// Initializing arraylist
List<String> stringsList = Arrays.asList("foo", "bar", "baz");

// Using iterator to display contents of stringsList
System.out.print("Contents of stringsList: ");

Iterator iter = al.iterator();
while(iter.hasNext()) {
    System.out.print(iter.next() + " ");
}

 

This iterator can only iterate data the way array does. (Then why do we need it?) It does the task of iterating objects perfectly, but we can add more functionality to the iterator.

 

Timeline iterator iterates the MessageEntry objects i.e. superclass of TwitterTweet objects. According to Javadocs, “Timeline is a structure which holds tweet for the purpose of presentation, There is no tweet retrieval method here, just an iterator which returns the tweets in reverse appearing order.”

Following are some of the tasks it does:

  1. As an iterator:

This basic use of Timeline is to iterate the MessageEntry objects. It not only iterates the data objects, but also fetches them (See here).

// Declare Timeline object according to order the data object has been created
Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

// Adding data objects to the timeline
tline.add(me1);
tline.add(me2);
.
.
.
// Outputing all data objects as array of JSON objects
for (MessageEntry me: tline) {
    JSONArray postArray = new JSONArray();
    for (MessageEntry post : this) {
        postArray.put(post.toJSON());
    }
}

 

  1. The order of iterating the data objects

Timeline can arrange and iterate the data objects according to the date of creation of the twitter post, number of retweets or number of favourite counts. For this there is an Enum declaration of Order in the Timeline class which is initialized during creation of Timeline object. [link]

    Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

 

  1. Pagination of data objects

There is an object cursor, some methods, including getter and setters to support pagination of the data objects. It is only internally implemented, but can also be used to return a section of the result.

  1. writeToIndex method

This method can be used to write all data fetched by Timeline iterator to ElasticSearch for indexing and to dump that can be used for testing. Thus, indexing of data can concurrently be done while it is iterated. It is implemented here.

  1. Other methods

It also has methods to output all data as JSON and customized method to add data to Timeline keeping user object and Data separate, etc. There are a bit more things in this iterable class which shall be explored instead.

 

Resources:

Continue ReadingIterating the Loklak Server data
Read more about the article Multithreading implementation in Loklak Server

Multithreading implementation in Loklak Server

Loklak Server is a near-realtime system. It performs a large number of tasks and are very costly in terms of resources. Its basic function is to scrape all data from websites and output it at the endpoint. In addition to scraping data, there is also a need to perform other tasks like refining and cleaning of data. That is why, multiple threads are instantiated. They perform other tasks like:

  1. Refining of data and extract more data

The data fetched needs to be cleaned and refined before outputting it. Some of the examples are:

a) Removal of html tags from tweet text:

After extracting text from html data and feeding to TwitterTweet object, it concurrently runs threads to remove all html from text.

b) Unshortening of url links:

The url links embedded in the tweet text may track the users with the help of shortened urls. To prevent this issue, a thread is instantiated to unshorten the url links concurrently while cleaning of tweet text.

  1. Indexing all JSON output data to ElasticSearch

While extracting JSON data as output, there is a method here in Timeline.java that indexes data to ElasticSearch.

Managing multithreading

To manage multithreading, Loklak Server applies following objects:

1. ExecutorService

To deal with large numbers of threads ExecutorService object is used to handle threads as it helps JVM to prevent any resource overflow. Thread’s lifecycle can be controlled and its creation cost can be optimized. This is the best example of ExecutorService application is here:

.
.
public class TwitterScraper {
    // Creation of at max 40 threads. This sets max number of threads to 40 at a time
    public static final ExecutorService executor = Executors.newFixedThreadPool(40);
    .
    .
    .
    .
    // Feeding of TwitterTweet object with data
    TwitterTweet tweet = new TwitterTweet(
        user.getScreenName(),
        Long.parseLong(tweettimems.value),
        props.get("tweettimename").value,
        props.get("tweetstatusurl").value,
        props.get("tweettext").value,
        Long.parseLong(tweetretweetcount.value),
        Long.parseLong(tweetfavouritecount.value),
        imgs, vids, place_name, place_id,
        user, writeToIndex,  writeToBackend
    );
    // Starting thread to refine TwitterTweet data
    if (tweet.willBeTimeConsuming()) {
       executor.execute(tweet);
    }    .
    .
    .

 

2. basic Thread class

Thread class can also be used instead of ExecutorService in cases where there is no resource crunch. But it is always suggested to use ExecutorService due to its benefits. Thread implementation can be used as an anonymous class like here.

3. Runnable interface

Runnable interface can be used to create an anonymous class or classes which does more task than just a task concurrently. In Loklak Server, TwitterScraper concurrently indexes the data to ElasticSearch, unshortens link and cleans data. Have a look at implementation here.

Resources:

Continue ReadingMultithreading implementation in Loklak Server
Read more about the article Dynamic Segments in Open Event Frontend

Dynamic Segments in Open Event Frontend

In the Open Event Frontend project we have a page where we show all types of events. We have classified events into six classes like live, draft, past etc. Now each event type should have it’s own page describing it. What should we do now? Make six different routes corresponding to each class of event? Isn’t that too cumbersome. Is there any other method to do it?

Dynamic segment is the answer to the above question. Dynamic segment is that segment of the path for a route that will change based on content of  the page. Dynamic segments are frequently used in Open Event Frontend.

One such use is in /admin/events. Here we have button for different categories of events. We do not make separate routes for each of them, instead we use dynamic segments. We’ll have a single route and we will change the data in the route corresponding to the tab chosen.

Lets now add dynamic segments to /admin/events. First of all we’ll add the following code snippet in router.js.

this.route(‘events’, function() {
      this.route(‘list’, { path: ‘/:events_status’ });
      this.route(‘import’);
    });

Here : signifies the presence of dynamic segment and it is followed by an identifier. It’s the identifier by which the route matches the corresponding model to show.

Now as our route would show data depending upon the tab selected, we must change the title of page depending upon the same. For this we add the following code snippet in admin/events/list.js. Here we set the title of the page using titleToken() function and access the dynamic portion of url using params.events_status

export default Route.extend({
  titleToken() {
    switch (this.get(‘params.events_status’)) {
      case ‘live’:
        return this.l10n.t(‘Live’);
      case ‘draft’:
        return this.l10n.t(‘Draft’);
      case ‘past’:
        return this.l10n.t(‘Past’);
      case ‘deleted’:
        return this.l10n.t(‘Deleted’);
    }
  },
  model(params) {
    this.set(‘params’, params);
    return [{
    // Events data
    }];
  }
});

We’ll now link the tabs template/admin/events to the corresponding models using Link-to the helper.Here we have linked ‘Live’, ‘Draft’, ‘Past’, ‘Deleted’ buttons to dynamic segments. Let’s understand how it works.Let’s take example of Live button.Live button is linked to admin/events/list and this list is replaced by ‘live’. So our final route becomes admin/events/live.

<div class=“ui grid stackable”>
  <div class=“row”>
    <div class=“sixteen wide column”>
      {{#tabbed-navigation isNonPointing=true}}
        {{#link-to ‘admin.events.index’ class=’item’}}
          {{t ‘All Events’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘live’ class=’item’}}
          {{t ‘All Live’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘draft’ class=’item’}}
          {{t ‘All Draft’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘past’ class=’item’}}
          {{t ‘All Past’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘deleted’ class=’item’}}
          {{t ‘All Deleted’}}
        {{/link-to}}
        {{#link-to ‘admin.events.import’ class=’item’}}
          {{t ‘Import’}}
        {{/link-to}}
      {{/tabbed-navigation}}
    </div>
  </div>
  <div class=“row”>
    {{outlet}}
  </div>
</div>

Additional Resources

Continue ReadingDynamic Segments in Open Event Frontend
Read more about the article Unifying Data from Different Scrapers of loklak server using Post

Unifying Data from Different Scrapers of loklak server using Post

Loklak Server project is a software that scrapes data from different websites through different endpoints. It is difficult to create a single endpoint. For a single endpoint, there is a need of a decent design for using multiple scrapers. For such a task, multiple changes are needed. That is why one of the changes I introduced was Post class that acts as both wrapper and an interface for data objects of search scrapers (though implementation in scrapers is in progress).

Post is a subclass of JSONObject that helps in working with JSON data in Java. In other words, Post is a JSONObject with an identity (we call it postId) and and a timestamp of the data scraped. It is used to capture data fetched by the web-scrapers. Benefit of JSONObject as superclass is that it provides methods to capture and access data efficiently.

Why Post?

At present there is a Class MessageEntry which is the superclass of TwitterTweet (data object of TwitterScraper). It has numerous methods that can be used by data objects to clean and analyse data. But it has a disadvantage, it is a specialized for social websites like Twitter, but will become redundant for different types websites like Quora, Github, etc.

Whereas Post object is a small but powerful and flexible object with its ability to deal with data like JSONObject. It contains getter and setter methods, identity members used to provide each Post object a unique identity. It doesn’t have any methods for analysis and cleaning of data, but MessageEntry class’ methods can be used for this purpose.

Uses of Post Object

When I started working on Post Object, it could be used as marker interface for data objects. Following are the advantages I came up with it:

1) Accessing the data object of any scraper using its variable. And yes, this is the primary reason it is an interface.

2) But in addition to accessing the data objects, one can also directly use it to fetch, modify or use data without knowing the scraper it belongs. This feature is useful in Timeline iterator.

This is an example how Post interface is used to append two lists of Posts (maybe carrying different type of data) into one.

public void mergePost(PostTimeline list) {
    for (Post post: list) {
        this.add(post);
    }
}

 

Post as a wrapper object

While working on Post object, I converted it into a class to also use it as a wrapper. But why a wrapper? Wrapper can be used to wrap a list of Post objects into one object. It doesn’t have any identity or timestamp. It is just a utility to dump a pack of data objects with homogeneous attributes.

This is an example implementation of Post object as wrapper. typeArray is a wrapper which is used to store 2 arrays of data objects in it. These data object arrays are timeline objects that are saved as JSONArray objects in the Post wrapper.

    Post typeArray = new Post(true);
    switch(type) {
        case "users":
            typeArray.put("users", scrapeProfile(br, url).toArray());
            break;
        case "question":
            typeArray.put("question", scrapeQues(br, url).toArray());
            break;
        default:
            break;
    }

 

Resources:

 

Continue ReadingUnifying Data from Different Scrapers of loklak server using Post
Read more about the article Using Vector Images in SUSI Android

Using Vector Images in SUSI Android

SUSI is an artificial intelligence for interactive chat bots. For making it more user friendly and interactive we add a lot of images in the form of drawable resources in the SUSI Android App (https://github.com/fossasia/susi_android). Most of these drawables are in the form of PNGs. There are certain problems associated with the use of PNG images.

  1. PNGs cannot be scaled without losing quality. Due to which for the same PNG image we have to include separate images of varied quality. Otherwise the image will become blur.
  2. PNGs tends to take large disk space which can be easily reduced with the use of vector images.
  3. PNGs have fixed color and dimensions which cannot be changed.

Due to the above shortcomings of PNG images we decided to use vector drawable images instead of them.

Advantages associated with Vector images

  1. They can be scaled to any size without the loss in quality. Thus we need to include only a single image in the app and not of varied qualities.
  2. They are very small in size as compared to PNGs.
  3. They can be easily modified programmatically in XML file unlike PNGs.

Using Vector Images in Android Studio

Android Studio provide tools by which we can directly import vector drawables in the project. To import Vector images go to File>New>Vector Assets in studio.

From here we can choose the icon we want to include in our project and click OK. The icon will appear in the drawables directory and can be used anywhere in the projects.

Implementation in SUSI Android

In Susi Android we have used various vector images such as arrows, pointer and even the logo of the app. Here below is the logo of SUSI.

This is actually a vector image below we will see the code required to get this logo as the output.

<vector android:height="50dp" android:viewportHeight="279.37604"

  android:viewportWidth="1365.2" android:width="220dp" xmlns:android="http://schemas.android.com/apk/res/android">

<path android:fillColor="#ffffff"

      android:pathData="M127.5,7.7c-26.8,3.3 -54.2,16.8 -75.9,37.4 -11.8,11.1 -20.4,22.9 -28.1,38.4 -8.9,17.8 -12.8,32.1 -13.7,51l-0.3,6 39,0 39,0 0.3,-4c0.7,-12.1 6.8,-24.1 17.2,-34.5 8.5,-8.4 16.2,-13.4 25.9,-16.7l6.6,-2.2 81.3,-0.1 81.2,0 0,-38 0,-38 -84.7,0.1c-46.7,0.1 -86.1,0.4 -87.8,0.6z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M319.2,11.3l-4.3,4.3 0.3,103c0.4,113.2 0,105.9 6.4,118.6 10.8,21.3 35.1,41.9 56.2,47.3 8.5,2.3 99.1,2.2 107.7,0 18.7,-4.9 39.2,-20.7 51.5,-39.7 3.4,-5.1 7.1,-12.2 8.3,-15.8l2.2,-6.5 0.5,-103.3 0.5,-103.3 -4.5,-4.4 -4.6,-4.5 -31.5,0 -31.5,0 -4.7,4.8 -4.7,4.8 0,93 0,93 -3.3,3.2 -3.3,3.2 -29,0 -29,0 -2.6,-2.7 -2.7,-2.8 -0.7,-94.2 -0.7,-94.2 -4.3,-4 -4.2,-4.1 -31.9,0 -31.9,0 -4.2,4.3z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M680,7.6c-31.6,4.8 -56.1,17.3 -79,40.3 -23.2,23.3 -36.3,50.5 -38.9,80.9 -0.5,5.9 -0.7,11 -0.4,11.4 0.2,0.5 17.7,0.8 38.8,0.8l38.4,0 0.6,-4.8c3.2,-23.2 21.3,-44.1 44.7,-51.3 5.6,-1.8 10.6,-1.9 86.6,-1.9l80.7,0 -0.3,-38 -0.2,-38 -84.3,0.1c-46.3,0.1 -85.3,0.3 -86.7,0.5z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M869.1,13.4l-4.1,6.4 0,126.4 0,126.3 4.8,6.7 4.7,6.8 31.6,0 31.6,0 4.7,-7 4.6,-7 0,-125.7 0,-125.8 -4.7,-6.7 -4.8,-6.8 -32.1,0 -32.1,0 -4.2,6.4z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M222.5,152.2c-0.2,0.7 -0.9,4.2 -1.5,7.7 -3.4,19.5 -19.4,38 -40,46.4l-5.5,2.2 -83,0.5 -83,0.5 -0.3,37.8 -0.2,37.8 89.2,-0.3 89.3,-0.3 9.6,-2.7c57.7,-16.3 100.1,-67.4 102.1,-123.3l0.3,-7 -38.3,-0.3c-30.1,-0.2 -38.3,0 -38.7,1z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M774.5,152.2c-0.2,0.7 -0.9,4.1 -1.5,7.5 -3.3,19.2 -18.8,37.3 -39.4,46.2l-6.1,2.6 -83,0.5 -83,0.5 -0.3,37.7 -0.2,37.8 85.9,0c93.7,0 91.4,0.1 110.1,-5.9 26.4,-8.5 53.3,-28.4 69.8,-51.7 15.2,-21.3 25.1,-50.1 24,-69.9l-0.3,-6 -37.8,-0.3c-29.7,-0.2 -37.8,0 -38.2,1z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff" android:pathData="m1146.99,0 l-1.38,1.19c-0.76,0.66 -1.85,1.61 -2.43,2.13 -0.58,0.51 -1.75,1.54 -2.61,2.28 -1.52,1.31 -1.58,1.41 -2.4,3.53 -0.46,1.2 -0.92,2.37 -1.01,2.59 -30.55,82.93 -61.62,165.72 -96.03,259.63 0,0.08 1.61,1.88 3.57,3.98l3.57,3.84 33.47,-0.04 33.47,-0.04c12.28,-35.6 25.13,-72.47 37.4,-107.27 0.06,-0.25 0.28,-0.64 0.5,-0.88 0.37,-0.41 0.61,-0.43 4.2,-0.43 3.63,0 3.83,0.02"/>

  <path android:fillColor="#ffffff" android:pathData="m967.09,279.18c-2.48,-3.74 -4.97,-7.04 -8.09,-11.76l0.09,-43.92c3.34,-5.26 5.31,-6.73 8.42,-11.51 17.91,0.02 34.3,0.26 50.88,0.26 3.21,4.88 4.09,6.72 7.81,12.66 -0.05,13.98 0.1,27.96 -0.12,41.94 -2.9,4.2 -4.27,7.42 -7.78,12.18 -18.81,-0.04 -35.43,0.2 -51.21,0.15z"/>

  <path android:fillColor="#ffffff"

      android:pathData="m1287.3,6.59 l-4.1,6.4 0,126.4 0,126.3 4.8,6.7 4.7,6.8 31.6,0 31.6,0 4.7,-7 4.6,-7 0,-125.7 0,-125.8 -4.7,-6.7 -4.8,-6.8 -32.1,0 -32.1,0 -4.2,6.4z" android:strokeColor="#00000000"/>


</vector>

In this code we can easily change the color and minor details for the logo which could have been not possible if the logo was in PNG format. Also we don’t need multiple logo images of varied qualities as it can be scaled without decreasing quality.

Resources

Continue ReadingUsing Vector Images in SUSI Android
Read more about the article Query Model Structure of Loklak Search

Query Model Structure of Loklak Search

Need to restructure

The earlier versions of loklak search applications had the issues of breaking changes whenever any new feature was added in the application. The main reason for these unintended bugs was identified to be the existing query structure. The query structure which was used in the earlier versions of the application only comprised of the single entity a string named as queryString.

export interface Query {
 queryString: string;
}

This simple query string property was good enough for simple string based searches which were the goals of the application in the initial phases, but as the application progressed we realized this simple string based implementation is not going to be feasible for the long term. As there are only a limited things we can do with strings. It becomes extremely difficult to set and reset the portions of the string according to the requirements. This was the main vision for the alternate architecture design scheme was to the ease of enabling and disabling the features on the fly.

Application Structure

Therefore, to overcome the difficulties faced with the simple string based structure we introduced the concept of an attribute based structure for queries. The attribute based structure is simpler to understand and thus easier to maintain the state of the query in the application.

export interface Query {
 displayString: string;
 queryString: string;
 routerString: string;
 filter: FilterList;
 location: string;
 timeBound: TimeBound;
 from: boolean;
}

The reason this is called an attribute based structure is that here each property of an interface is independent of one another. Thus each one can be thought of as a separate little key placed on the query, but each of these keys are different and are mutually exclusive. What this means is, if I want to write an attribute query, then it does not matter to me which other attributes are already present on the query. The query will eventually be processed and sent to the server and the corresponding valid results if exists will be shown to the user.

Now the question arises how do we modify the values of these attributes? Now before answering this I would like to mention that this interface is actually instantiated in the the Redux state, so now our question automatically gets answered, the modification to redux state corresponding to the query structure will be done by specific reducers meant for modification of each attribute. These reducers are again triggered by corresponding actions.

export const ActionTypes = {
 VALUE_CHANGE: '[Query] Value Change',
 FILTER_CHANGE: '[Query] Filter Change',
 LOCATION_CHANGE: '[Query] Location Change',
 TIME_BOUND_CHANGE: '[Query] Time Bound Change',
};

This ActionTypes object contains the the corresponding actions which are used to trigger the reducers. These actions can be dispatched in response to any user interaction by any of the components, thus modifying a particular state attribute via the reducer.

Converting from object to string

Now for our API endpoint to understand our query we need to send the proper string in API accepted format. For this there is need to convert dynamically from query state to query string, for this we need a simple function which take in query state as an input return the query string as output.

export function parseQueryToQueryString(query: Query): string {
 let qs: string;
 qs = query.displayString;
 if (query.location) {
qs += ` near:${query.location);

 if (query.timeBound.since) {
   qs += ` since:${parseDateToApiAcceptedFormat(query.timeBound.since)}`;

 if (query.timeBound.until) {
   qs += ` until:${parseDateToApiAcceptedFormat(query.timeBound.until)}`;

 return qs;
}

In this function we are just checking and updating the query string according to the various attributes set in the structure, and then returning the query string. So if eventually we have to convert to the string, then what is the advantage of this approach? The main advantage of this approach is that we know the query structure beforehand and we use the structure to build the string not just randomly selecting and removing pieces of information from a string. Whenever we update any of the attribute of the query state, the query is generated fresh, and not modifying the old string.

Conclusion

This approach makes the application to be able to modify the search queries sent to server in a streamlined and logical way, just by using simple data structure. This query model has provided us with a lot of advantages which are visible in the aspect of application stability and performance. This model has cuts out dirty regex matching, of typed queries and thus again help us to make simpler queries.

Resources and Links

Continue ReadingQuery Model Structure of Loklak Search
Read more about the article Adding tip to drop downs in Susper using CSS in Angular

Adding tip to drop downs in Susper using CSS in Angular

To create simple drop downs using twitter bootstrap, it is fairly easy for developers. The issue faced in Susper, however, was to add a tip on the top over such dropdowns similar to Google:

This is how it looks finally, in Susper, with a tip over the standard rectangular drop-down:

This is how it was done:

  1. First, make sure you have designed your drop-down according to your requirements, added the desired height, width and padding. These were the specifications used in Susper’s drop-down.

.dropdown-menu{
height: 500px;
width: 327px;
padding: 28px;
}
  1. Next add the following code to your drop-down class css:

.dropdown-menu:before {
position: absolute;
top: -7px;
right: 19px;
display: inline–block;
border-right: 7px solid transparent;
border-bottom: 7px solid #ccc;
border-left: 7px solid transparent;
border-bottom-color: rgba(0, 0, 0, 0.2);
content: ”;
}
.dropdown-menu:after {
position: absolute;
top: -5px;
right: 20px;
display: inline–block;
border-right: 6px solid transparent;
border-bottom: 6px solid #ffffff;
border-left: 6px solid transparent;
content: ”;
}

In css, :before inserts the style before any other html, whereas :after inserts the style after the html is loaded. Some of the parameters are explained here:

  • Top: can be used to change the position of the menu tip vertically, according to the position of your button and menu.
  • Right: can be used to change the position of the menu tip horizontally, so that it can be positioned used below the menu icon.
  • Position : absolute is used to make sure all our values are absolute and not relative to the higher div hierarchically
  • Border: All border attributes are used to specify border thickness, color and transparency before and after, which collectively gives the effect of a tip for the drop down.
  • Content : This value is set to a blank string ‘’, because otherwise none of our changes will be visible, since the divs will have no space allocated to them.

Resources

Continue ReadingAdding tip to drop downs in Susper using CSS in Angular
Read more about the article Using SUSI AI Server to Store User Feedback for a Skill

Using SUSI AI Server to Store User Feedback for a Skill

User feedback is valuable information that plays an important  role in improving the quality of service. In SUSI AI server we are planning to make a feedback mechanism to see if the user liked the answer or not. The result of that user input (which can be given using a vote button) will be then learned to enhance the future use of the rule. So as a first step for implementation of  skill rating system with guided learning, we need to store the user rating of a skill . In this blogpost we will learn how to make an endpoint for getting skill rating from user. This API endpoint will be used  by its web and mobile clients.
Before the implementation of API  let’s look how data is stored in SUSI AI Susi_server uses DAO in which skill rating is stored as JSONTray. 

 public JsonTray(File file_persistent, File file_volatile, int cachesize) throws IOException {
        this.per = new JsonFile(file_persistent);
        this.vol = new CacheMap<String, JSONObject>(cachesize);
        this.file_volatile = file_volatile;
        if (file_volatile != null && file_volatile.exists()) try {
            JSONObject j = JsonFile.readJson(file_volatile);
            for (String key: j.keySet()) this.vol.put(key, j.getJSONObject(key));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

JsonTray takes three parameters the persistent file, volatile file and cache size to store them as cache map in String and JsonObject pairs. The HttpServlet class which provides methods, such as doGet and doPost, for handling HTTP-specific services.In Susi Server an abstract class AbstractAPIHandler extending HttpServelets and implementing API handler interface is provided. Next we will inherit our RateSkillService class from AbstractAPIHandler and implement APIhandler interface.

public class RateSkillService extends AbstractAPIHandler implements APIHandler {
    private static final long serialVersionUID =7947060716231250102L;
    @Override
    public BaseUserRole getMinimalBaseUserRole() {
        return BaseUserRole.ANONYMOUS;
    }

    @Override
    public JSONObject getDefaultPermissions(BaseUserRole baseUserRole) {
        return null;
    }

    @Override
    public String getAPIPath() {
        return "/cms/rateSkill.json";
    }
…
}

The getMinimalBaseRole method tells the minimum Userrole required to access this servlet it can also be ADMIN, USER. In our case it is Anonymous. A User need not to log in to access this endpoint. The getAPIPath() methods sets the API endpoint path, it gets appended to base path which is 127.0.0.1:4000/cms/rateSkill.json for local host .

Next we will implement serviceImpl method

  @Override
    public ServiceResponse serviceImpl(Query call, HttpServletResponse response, Authorization rights, final JsonObjectWithDefault permissions) {

        String model_name = call.get("model", "general");
        File model = new File(DAO.model_watch_dir, model_name);
        String group_name = call.get("group", "knowledge");
        File group = new File(model, group_name);
        String language_name = call.get("language", "en");
        File language = new File(group, language_name);
        String skill_name = call.get("skill", null);
        File skill = new File(language, skill_name + ".txt");
        String skill_rate = call.get("rating", null);

        JSONObject result = new JSONObject();
        result.put("accepted", false);
        if (!skill.exists()) {
            result.put("message", "skill does not exist");
            return new ServiceResponse(result);

        }
        JsonTray skillRating = DAO.skillRating;
        JSONObject modelName = new JSONObject();
        JSONObject groupName = new JSONObject();
        JSONObject languageName = new JSONObject();
        if (skillRating.has(model_name)) {
            modelName = skillRating.getJSONObject(model_name);
            if (modelName.has(group_name)) {
                groupName = modelName.getJSONObject(group_name);
                if (groupName.has(language_name)) {
                    languageName = groupName.getJSONObject(language_name);
                    if (languageName.has(skill_name)) {
                        JSONObject skillName = languageName.getJSONObject(skill_name);
                        skillName.put(skill_rate, skillName.getInt(skill_rate) + 1 + "");
                        languageName.put(skill_name, skillName);
                        groupName.put(language_name, languageName);
                        modelName.put(group_name, groupName);
                        skillRating.put(model_name, modelName, true);
                        result.put("accepted", true);
                        result.put("message", "Skill ratings updated");
                        return new ServiceResponse(result);
                    }
                }
            }
        }
        languageName.put(skill_name, createRatingObject(skill_rate));
        groupName.put(language_name, languageName);
        modelName.put(group_name, groupName);
        skillRating.put(model_name, modelName, true);
        result.put("accepted", true);
        result.put("message", "Skill ratings added");
        return new ServiceResponse(result);

    }

    /* Utility function*/
    public JSONObject createRatingObject(String skill_rate) {
        JSONObject skillName = new JSONObject();
        skillName.put("positive", "0");
        skillName.put("negative", "0");
        skillName.put(skill_rate, skillName.getInt(skill_rate) + 1 + "");
        return skillName;
    }

 

One can access any skill based on four tuples parameters model, group, language, skill. Before rating a skill we must ensure whether it exists or not. We can get the required parameters through call.get() method where first parameter is the key for which we want to get the value and second parameter is the default value. If skill.exists() method return false we generate error message stating “No such skill exists”. Otherwise check if the skill exist in our skillRating.json file if so, update the current ratings otherwise create a new json object and add it to rating file based on model, group and language. After successful implementation go ahead and test your endpoint on http://localhost:4000/cms/rateSkill.json?model=general&group=knowledge&skill=who&rating=positive

You can also check for the updated json file in  susi_server/data/skill_rating/skillRating.json 

{"general": {
 "assistants": {"en": {
   "language_translation": {
     "negative": "1",
     "positive": "0"
  }}},
 "smalltalk": {"en": {
   "aboutsusi": {
   "negative": "0",
   "positive": "1"
 }}},
 "knowledge": {"en": {
   "who": {
     "negative": "2",
     "positive": "4"
   }}}
}}

And if the skill is not present if will generate error message

We have successfully implemented the API endpoint for storing the user skill’s feedback. For more information take a look at Susi server and join gitter chat channel for discussions.

Resources 

Continue ReadingUsing SUSI AI Server to Store User Feedback for a Skill