FOSSASIA

Read more about the article Add Autocomplete SearchView in Open Event Android App

Add Autocomplete SearchView in Open Event Android App

The Open Event Android App has a map for showing all locations of sessions. All the locations have a marker in the map. It is difficult to find a particular location on the map because to know the name of location user has to click on the marker. Adding autocomplete SearchView will improve user experience by providing an ability to search the location by name and by suggesting name according to the search query. In this post I explain how to add autocomplete SearchView in the fragment or activity.

Add search icon in actionbar

The first step to do is to create a menu xml file and add a search menu item in it. Then inflate this menu xml file in Fragment in onCreateOptionsMenu() method.

1. Create menu.xml file

In this file add search menu element. Inside menu element add search menu item. Define id, title, and icon of search menu item. Add android.support.v7.widget.SearchView” as actionViewClass which will be used as action view when the user clicks on the icon.

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools">
    
   <item
        android:id="@+id/action_search"
        android:icon="@drawable/ic_search_white_24dp"
        android:title="@string/search"
        app:actionViewClass="android.support.v7.widget.SearchView"
        app:showAsAction="ifRoom | collapseActionView"/>
</menu>

2. Inflate menu.xml file in Fragment

In the fragment’s onCreateOptionsMenu() method inflate menu.xml file using MenuInflater’s inflate() method. Then find search menu item using menu’s findItem() method by passing id of search menu item as parameter.

public void onCreateOptionsMenu(Menu menu, MenuInflater inflater) {
        super.onCreateOptionsMenu(menu, inflater);
        inflater.inflate(R.menu.menu_map, menu);
        MenuItem item = menu.findItem(R.id.action_search);
}

Add and initialize SearchView  

Now after adding search icon we need to add SearchView and SearchAutoComplete fields in the fragment.

private SearchView searchView;
private SearchView.SearchAutoComplete   mSearchAutoComplete;

Initialize SearchView in onCreateOptionMenu() method by passing search menu item in the getActionView() method of MenuItemCompat.

Here SearchAutoComplete is a child object of SearchView so initialize it using findViewById method of SearchView by passing the id as parameter.

searchView = (SearchView) MenuItemCompat.getActionView(item);
mSearchAutoComplete = (SearchView.SearchAutoComplete) searchView.findViewById(android.support.v7.appcompat.R.id.search_src_text);

Define properties of SearchAutoCompleteView

By default background of drop down menu in SearchAutoComplete is black. You can change background using setDropDownBackgroundResource() method. Here i’m making it white by providing white drawable resource.

mSearchAutoComplete.setDropDownBackgroundResource(R.drawable.background_white);
mSearchAutoComplete.setDropDownAnchor(R.id.action_search);
mSearchAutoComplete.setThreshold(0)

The setDropDownAnchor() method sets the view to which the auto-complete drop down list should anchor. The setThreshold() method specifies the minimum number of characters the user has to type in the edit box before the drop down list is shown.

Create array adapter

Now it’s time to make the ArrayAdapter object which will provide the data set (strings) which will be used to run search queries.

ArrayAdapter<String> adapter = new ArrayAdapter<>(getActivity(), android.R.layout.simple_list_item_1, searchItems);

Here searchItems is List of strings. Now set this adapter to the mSearchAutoComplete object using setAdapter() method.

mSearchAutoComplete.setAdapter(adapter);

Now we are all set to run the app on device or emulator. Here’s demo how it will look

Conclusion

The SearchView with an ability to give suggestions serves the great user experience in the application.

Additional resources:

Continue ReadingAdd Autocomplete SearchView in Open Event Android App
Read more about the article Using CoreLocation in SUSI iOS

Using CoreLocation in SUSI iOS

The SUSI Server responds with intelligent answers to the user’s queries. To make these answers better, the server makes use of the user’s location which is sent as a parameter to the query request each time. To implement this feature in the SUSI iOS client, we use the CoreLocation framework provided by Apple which helps us to get the user’s location coordinates and add them as a parameter to each request made.

In order to start with using the CoreLocation framework, we first import it inside the view controller.

import CoreLocation

Now, we create a variable of type CLLocationManager which will help us to use the actual functionality.

// Location Manager
var locationManager = CLLocationManager()

The location manager has some delegate methods which give an option to get the maximum accuracy for a user’s location.  To set that, we need the controller to conform to the CLLocationManagerDelegate, so we create an extension of the view controller conforming to this.

extension MainViewController: CLLocationManagerDelegate {

   // use functionality

}

Next, we set the manager delegate.

locationManager.delegate = self

And create a method to ask for using the user’s location and set the delegate properties.

func configureLocationManager() {
       locationManager.delegate = self
       if CLLocationManager.authorizationStatus() == .notDetermined || CLLocationManager.authorizationStatus() == .denied {
           self.locationManager.requestWhenInUseAuthorization()
       }

       locationManager.distanceFilter = kCLDistanceFilterNone
       locationManager.desiredAccuracy = kCLLocationAccuracyBest
}

Here, we ask for the user location if it was previously denied or is not yet determined and following that, we set the `distanceFilter` as kCLDistanceFilterNone  and `desiredAccuray` as kCLLocationAccuracyBest.. Finally, we are left with starting to update the location which we do by:

locationManager.startUpdatingLocation()

We call this method inside viewDidLoad to start updation of the location when the view first loads. The complete extension looks like below:

extension MainViewController: CLLocationManagerDelegate {

   // Configures Location Manager
   func configureLocationManager() {
       locationManager.delegate = self
       if CLLocationManager.authorizationStatus() == .notDetermined || CLLocationManager.authorizationStatus() == .denied {
           self.locationManager.requestWhenInUseAuthorization()
       }

       locationManager.distanceFilter = kCLDistanceFilterNone
       locationManager.desiredAccuracy = kCLLocationAccuracyBest
       locationManager.startUpdatingLocation()
   }

}

Now, it’s very easy to use the location manager and get the coordinates and add it to the params for each request.

if let location = locationManager.location {
   params[Client.ChatKeys.Latitude] = location.coordinate.latitude as AnyObject
   params[Client.ChatKeys.Longitude] = location.coordinate.longitude as AnyObject
}

Now the params which is a dictionary object is added to each request made so that the user get’s the most accurate results for each query he makes.

References:

Continue ReadingUsing CoreLocation in SUSI iOS
Read more about the article Simplifying Scrapers using BaseScraper

Simplifying Scrapers using BaseScraper

Loklak Server‘s main function is to scrape data from websites and other sources and output in different formats like JSON, xml and rss. There are many scrapers in the project that scrape data and output them, but are implemented with different design and libraries which makes them different from each other and a difficult to fix changes.

Due to variation in scrapers’ design, it is difficult to modify them and fix the same issue (any issue, if it appears) in each of them. This issue signals fault in design. To solve this problem, Inheritance can be brought into application. Thus, I created BaseScraper abstract class so that scrapers are more concentrated on fetching data from HTML and all supportive tasks like creating connection with the help of url are defined in BaseScraper.

The concept is pretty easy to implement, but for a perfect implementation, there is a need to go through the complete list of tasks a scraper does.

These are the following tasks with descriptions and how they are implemented using BaseScraper:

  1. Endpoint that triggers the scraper

Every search scraper inherits class AbstractAPIHandler. This is used to fetch get parameters from the endpoint according to which data is scraped from the scraper. The arguments from serviceImpl method is used to generate output and is returned to it as JSONObject.

For this task, the method serviceImpl has been defined in BaseScraper and method getData is implemented to return the output. This method is the driver method of the scraper.

public JSONObject serviceImpl(Query call, HttpServletResponse response, Authorization rights, JSONObjectWithDefault permissions) throws APIException {
    this.setExtra(call);
    return this.getData().toJSON(false, "metadata", "posts");
}

 

  1. Constructor

The constructor of Scraper defines the base URL of the website to be scraped, name of the scraper and data structure to fetch all get parameters input to the scraper. For get parameters, the Map data structure is used to fetch them from Query object.

Since every scraper has it’s own different base URL, scraper name and get parameters used, so it is implemented in respective Scrapers. QuoraProfileScraper is an example which has these variables defined.

  1. Get all input variables

To get all input variables, there are setters and getters defined for fetching them as Map from Query object in BaseScraper. There is also an abstract method getParam(). It is defined in respective scrapers to fetch the useful parameters for scraper and set them to the scraper’s class variables.

// Setter for get parameters from call object
protected void setExtra(Query call) {
    this.extra = call.getMap();
    this.query = call.get("query", "");
    this.setParam();
}

// Getter for get parameter wrt to its key
public String getExtraValue(String key) {
    String value = "";
    if(this.extra.get(key) != null) {
        value = this.extra.get(key).trim();
    }
    return value;
}

// Defination in QuoraProfileScraper
protected void setParam() {
    if(!"".equals(this.getExtraValue("type"))) {
        this.typeList = Arrays.asList(this.getExtraValue("type").trim().split("\\s*,\\s*"));
    } else {
        this.typeList = new ArrayList<String>();
        this.typeList.add("all");
        this.setExtraValue("type", String.join(",", this.typeList));
    }
}

 

  1.  URL creation for web scraper

The URL creation shall be implemented in a separate method as in TwitterScraper. The following is the rough implementation adapted from one of my pull request:

protected String prepareSearchUrl(String type) {
    URIBuilder url = null;
    String midUrl = "search/";

    try {
        switch(type) {
            case "question":
                url = new URIBuilder(this.baseUrl + midUrl);
                url.addParameter("q", this.query);
                url.addParameter("type", "question");
        .
        .
    }
    .
    .
    return url.toString();
}

 

  1. Get BufferedReader object from InputStream

getDataFromConnection method fetches the BufferedReader object from ClientConnection. This object reads the web page line by line by the scrape method to fetch data. See here.

ClientConnection connection = new ClientConnection(url);
BufferedReader br = getHtml(connection);
.
.
.
public BufferedReader getHtml(ClientConnection connection) {

    if (connection.inputStream == null) {
        return null;
    }

    BufferedReader br = new BufferedReader(new InputStreamReader(connection.inputStream, StandardCharsets.UTF_8));
    return br;
}

 

  1. Scraping of data from HTML

The Scraper method for scraping data is declared abstract in BaseScraper and defined in the scraper. This can be a perfect example of implementation for BaseScraper (See code the here) and scraper (here).

  1. Output of data

The output of scrape method is fetched in Post data objects that are implemented for the respective scraper. These Post objects are added to Timeline iterator and which outputs data as JSONArray. Later the objects are output in enclosed Post object wrapper.

This data can be directly output as Post object, but adding it to iterator makes the Post Objects capable to be sorted in an order and be indexed to ElasticSearch.

 

Resources

Continue ReadingSimplifying Scrapers using BaseScraper
Read more about the article Package Manager Translation for Meilix

Package Manager Translation for Meilix

There are many Linux distros and all of them use variety of different package managers. So a particular user of that specific Linux distro is familiar with that distro package manager commands only. Due to which when that user is out at a event or someplace else and require to install or remove or update package using the commands he is familiar with, he may get errors in doing so if that distro doesn’t have a package manager that he is familiar with.

To overcome this problem we can have a solution of adding package manager command translating functionality to Meilix. To translate the commands of package manager like pacman, apt, yum, zypper we have build translation modules for each. To install these modules we first check the Linux distro and map it to the package manager it is using. For this we write the following script.

 declare -A osInfo;
  osInfo[/etc/redhat-release]=yum
  osInfo[/etc/arch-release]=pacman
  osInfo[/etc/gentoo-release]=emerge
  osInfo[/etc/SuSE-release]=zypp
  osInfo[/etc/debian_version]=apt-get

 

Then after checking the native package manager it copy the modules required for that packet manger to the bin and makes them executable.These modules can be called by the names of the packet manager not available on

These modules can be called by the names of the packet manager not available on system. The module reads the arguments and convert command according to it. Like for pacman to apt module, a simple pacman command to install a app is

Now, the pacman is a module called from bin using two arguments and these two arguments use a switch statement are converted.

Example of commands in ubuntu / debian based system using apt but the user was familiar with pacman

Installing package:

pacman -S PACKAGE

Gets translated to:

apt install PACKAGE

Remove package:

apt install PACKAGE

Gets translated to:

apt remove PACKAGE

Update software database :

pacman -Syy

Gets translated to:

apt update

Show updatable packages:

pacman -Qu

Gets translated to:

apt list --upgradable

Update all:

pacman -Syu

Gets translated to:

apt upgrade

 

Mew ensures the cross distro package manager command compatibility by providing translations which is a helpful tool especially at events where users may find it difficult to operate system if he cannot install or add the specific package he requires at that time. Mew helps in making the user experience better as the user don’t have to struggle with the package manager commands he is not familiar with.

contribute to the project by forking: https://github.com/fossasia/mew

Continue ReadingPackage Manager Translation for Meilix
Read more about the article Iterating the Loklak Server data
Iterating the Loklak Server data

Iterating the Loklak Server data

Loklak Server is amazing for what it does, but it is more impressive how it does the tasks. Iterators are used for and how to use them, but this project has a customized iterator that iterates Twitter data objects. This iterator is Timeline.java .

Timeline implements an interface iterable (isn’t it iterator?). This interface helps in using Timeline as an iterator and add methods to modify, use or create the data objects. At present, it only iterates Twitter data objects. I am working on it to modify it to iterate data objects from all web scrapers.

The following is a simple example of how an iterator is used.

// Initializing arraylist
List<String> stringsList = Arrays.asList("foo", "bar", "baz");

// Using iterator to display contents of stringsList
System.out.print("Contents of stringsList: ");

Iterator iter = al.iterator();
while(iter.hasNext()) {
    System.out.print(iter.next() + " ");
}

 

This iterator can only iterate data the way array does. (Then why do we need it?) It does the task of iterating objects perfectly, but we can add more functionality to the iterator.

 

Timeline iterator iterates the MessageEntry objects i.e. superclass of TwitterTweet objects. According to Javadocs, “Timeline is a structure which holds tweet for the purpose of presentation, There is no tweet retrieval method here, just an iterator which returns the tweets in reverse appearing order.”

Following are some of the tasks it does:

  1. As an iterator:

This basic use of Timeline is to iterate the MessageEntry objects. It not only iterates the data objects, but also fetches them (See here).

// Declare Timeline object according to order the data object has been created
Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

// Adding data objects to the timeline
tline.add(me1);
tline.add(me2);
.
.
.
// Outputing all data objects as array of JSON objects
for (MessageEntry me: tline) {
    JSONArray postArray = new JSONArray();
    for (MessageEntry post : this) {
        postArray.put(post.toJSON());
    }
}

 

  1. The order of iterating the data objects

Timeline can arrange and iterate the data objects according to the date of creation of the twitter post, number of retweets or number of favourite counts. For this there is an Enum declaration of Order in the Timeline class which is initialized during creation of Timeline object. [link]

    Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

 

  1. Pagination of data objects

There is an object cursor, some methods, including getter and setters to support pagination of the data objects. It is only internally implemented, but can also be used to return a section of the result.

  1. writeToIndex method

This method can be used to write all data fetched by Timeline iterator to ElasticSearch for indexing and to dump that can be used for testing. Thus, indexing of data can concurrently be done while it is iterated. It is implemented here.

  1. Other methods

It also has methods to output all data as JSON and customized method to add data to Timeline keeping user object and Data separate, etc. There are a bit more things in this iterable class which shall be explored instead.

 

Resources:

Continue ReadingIterating the Loklak Server data
Read more about the article Multithreading implementation in Loklak Server

Multithreading implementation in Loklak Server

Loklak Server is a near-realtime system. It performs a large number of tasks and are very costly in terms of resources. Its basic function is to scrape all data from websites and output it at the endpoint. In addition to scraping data, there is also a need to perform other tasks like refining and cleaning of data. That is why, multiple threads are instantiated. They perform other tasks like:

  1. Refining of data and extract more data

The data fetched needs to be cleaned and refined before outputting it. Some of the examples are:

a) Removal of html tags from tweet text:

After extracting text from html data and feeding to TwitterTweet object, it concurrently runs threads to remove all html from text.

b) Unshortening of url links:

The url links embedded in the tweet text may track the users with the help of shortened urls. To prevent this issue, a thread is instantiated to unshorten the url links concurrently while cleaning of tweet text.

  1. Indexing all JSON output data to ElasticSearch

While extracting JSON data as output, there is a method here in Timeline.java that indexes data to ElasticSearch.

Managing multithreading

To manage multithreading, Loklak Server applies following objects:

1. ExecutorService

To deal with large numbers of threads ExecutorService object is used to handle threads as it helps JVM to prevent any resource overflow. Thread’s lifecycle can be controlled and its creation cost can be optimized. This is the best example of ExecutorService application is here:

.
.
public class TwitterScraper {
    // Creation of at max 40 threads. This sets max number of threads to 40 at a time
    public static final ExecutorService executor = Executors.newFixedThreadPool(40);
    .
    .
    .
    .
    // Feeding of TwitterTweet object with data
    TwitterTweet tweet = new TwitterTweet(
        user.getScreenName(),
        Long.parseLong(tweettimems.value),
        props.get("tweettimename").value,
        props.get("tweetstatusurl").value,
        props.get("tweettext").value,
        Long.parseLong(tweetretweetcount.value),
        Long.parseLong(tweetfavouritecount.value),
        imgs, vids, place_name, place_id,
        user, writeToIndex,  writeToBackend
    );
    // Starting thread to refine TwitterTweet data
    if (tweet.willBeTimeConsuming()) {
       executor.execute(tweet);
    }    .
    .
    .

 

2. basic Thread class

Thread class can also be used instead of ExecutorService in cases where there is no resource crunch. But it is always suggested to use ExecutorService due to its benefits. Thread implementation can be used as an anonymous class like here.

3. Runnable interface

Runnable interface can be used to create an anonymous class or classes which does more task than just a task concurrently. In Loklak Server, TwitterScraper concurrently indexes the data to ElasticSearch, unshortens link and cleans data. Have a look at implementation here.

Resources:

Continue ReadingMultithreading implementation in Loklak Server
Read more about the article Dynamic Segments in Open Event Frontend

Dynamic Segments in Open Event Frontend

In the Open Event Frontend project we have a page where we show all types of events. We have classified events into six classes like live, draft, past etc. Now each event type should have it’s own page describing it. What should we do now? Make six different routes corresponding to each class of event? Isn’t that too cumbersome. Is there any other method to do it?

Dynamic segment is the answer to the above question. Dynamic segment is that segment of the path for a route that will change based on content of  the page. Dynamic segments are frequently used in Open Event Frontend.

One such use is in /admin/events. Here we have button for different categories of events. We do not make separate routes for each of them, instead we use dynamic segments. We’ll have a single route and we will change the data in the route corresponding to the tab chosen.

Lets now add dynamic segments to /admin/events. First of all we’ll add the following code snippet in router.js.

this.route(‘events’, function() {
      this.route(‘list’, { path: ‘/:events_status’ });
      this.route(‘import’);
    });

Here : signifies the presence of dynamic segment and it is followed by an identifier. It’s the identifier by which the route matches the corresponding model to show.

Now as our route would show data depending upon the tab selected, we must change the title of page depending upon the same. For this we add the following code snippet in admin/events/list.js. Here we set the title of the page using titleToken() function and access the dynamic portion of url using params.events_status

export default Route.extend({
  titleToken() {
    switch (this.get(‘params.events_status’)) {
      case ‘live’:
        return this.l10n.t(‘Live’);
      case ‘draft’:
        return this.l10n.t(‘Draft’);
      case ‘past’:
        return this.l10n.t(‘Past’);
      case ‘deleted’:
        return this.l10n.t(‘Deleted’);
    }
  },
  model(params) {
    this.set(‘params’, params);
    return [{
    // Events data
    }];
  }
});

We’ll now link the tabs template/admin/events to the corresponding models using Link-to the helper.Here we have linked ‘Live’, ‘Draft’, ‘Past’, ‘Deleted’ buttons to dynamic segments. Let’s understand how it works.Let’s take example of Live button.Live button is linked to admin/events/list and this list is replaced by ‘live’. So our final route becomes admin/events/live.

<div class=“ui grid stackable”>
  <div class=“row”>
    <div class=“sixteen wide column”>
      {{#tabbed-navigation isNonPointing=true}}
        {{#link-to ‘admin.events.index’ class=’item’}}
          {{t ‘All Events’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘live’ class=’item’}}
          {{t ‘All Live’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘draft’ class=’item’}}
          {{t ‘All Draft’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘past’ class=’item’}}
          {{t ‘All Past’}}
        {{/link-to}}
        {{#link-to ‘admin.events.list’ ‘deleted’ class=’item’}}
          {{t ‘All Deleted’}}
        {{/link-to}}
        {{#link-to ‘admin.events.import’ class=’item’}}
          {{t ‘Import’}}
        {{/link-to}}
      {{/tabbed-navigation}}
    </div>
  </div>
  <div class=“row”>
    {{outlet}}
  </div>
</div>

Additional Resources

Continue ReadingDynamic Segments in Open Event Frontend
Read more about the article Create an AutocompleteTextView dropdown for the email input in the Open Event Orga Android App

Create an AutocompleteTextView dropdown for the email input in the Open Event Orga Android App

In the first version of the Open Event Organizer App, the event organizer was required to enter his full email each time he logged out of his account and therefore it was hindering the user experience. AutoCompleteTextView with shared preferences is a solution to this problem. This feature provides an editable text view that shows completion suggestions automatically while the user is typing. The list of suggestions is displayed in a drop down menu. The user can choose an item to replace the content of the edit box with. It is extremely useful in enhancing user experience.

The solution we implemented was to create an autocomplete textview for the email input, store the email address of the user on a successful login in the shared preference in a set of strings to prevent duplicacy and display it in the dropdown on subsequent login attempts.

Implementation

Change your TextInputLayout structure to accommodate the autocompletetextview. Remember to create a separate autocompletetextview object with the specific id of the view.

<android.support.v7.widget.AppCompatAutoCompleteTextView
       android:id="@+id/email_dropdown"
       android:layout_width="match_parent"
       android:layout_height="wrap_content"
       android:hint="@string/email"
       android:inputType="textEmailAddress" />

 

Create Utility methods to get/store the emails in the shared preferences. The set data structure has been used here so that there is no duplicacy while storing the emails in the shared preferences.

public Set<String> getStringSet(String key, Set<String> defaultValue) {
   return sharedPreferences.getStringSet(key, defaultValue);
}

public void saveStringSet(String key, Set<String> value) {
   SharedPreferences.Editor editor = sharedPreferences.edit();
   editor.putStringSet(key, value);
   editor.apply();
}

public void addStringSetElement(String key, String value) {
   Set<String> set = getStringSet(key, new HashSet<>());
   set.add(value);
   saveStringSet(key, set);
}

 

Create helper methods to add an email and retrieve the list of emails from the shared preferences to provide it to the views.

private void saveEmail(String email) {
   utilModel.addStringSetElement(Constants.SHARED_PREFS_SAVED_EMAIL, email);
}

private Set<String> getEmailList() {
   return utilModel.getStringSet(Constants.SHARED_PREFS_SAVED_EMAIL, null);
}

 

Create an autocompleteTextView object in your activity with the help of the R id from the layout and set the adapter with the set of strings retrieved from the shared preferences. You could create a custom adapter for this case too, but as far as the Open Event Orga App was concerned, using the array adapter made sense.

autoCompleteEmail.setAdapter(new ArrayAdapter<>(this, android.R.layout.simple_list_item_1,
   new ArrayList<String>(emails)));

 

Conclusion

It is important that the user is served with the best possible experience of the application and the autocomplete text view for the email serves just that.

Resources

Continue ReadingCreate an AutocompleteTextView dropdown for the email input in the Open Event Orga Android App
Read more about the article Unifying Data from Different Scrapers of loklak server using Post

Unifying Data from Different Scrapers of loklak server using Post

Loklak Server project is a software that scrapes data from different websites through different endpoints. It is difficult to create a single endpoint. For a single endpoint, there is a need of a decent design for using multiple scrapers. For such a task, multiple changes are needed. That is why one of the changes I introduced was Post class that acts as both wrapper and an interface for data objects of search scrapers (though implementation in scrapers is in progress).

Post is a subclass of JSONObject that helps in working with JSON data in Java. In other words, Post is a JSONObject with an identity (we call it postId) and and a timestamp of the data scraped. It is used to capture data fetched by the web-scrapers. Benefit of JSONObject as superclass is that it provides methods to capture and access data efficiently.

Why Post?

At present there is a Class MessageEntry which is the superclass of TwitterTweet (data object of TwitterScraper). It has numerous methods that can be used by data objects to clean and analyse data. But it has a disadvantage, it is a specialized for social websites like Twitter, but will become redundant for different types websites like Quora, Github, etc.

Whereas Post object is a small but powerful and flexible object with its ability to deal with data like JSONObject. It contains getter and setter methods, identity members used to provide each Post object a unique identity. It doesn’t have any methods for analysis and cleaning of data, but MessageEntry class’ methods can be used for this purpose.

Uses of Post Object

When I started working on Post Object, it could be used as marker interface for data objects. Following are the advantages I came up with it:

1) Accessing the data object of any scraper using its variable. And yes, this is the primary reason it is an interface.

2) But in addition to accessing the data objects, one can also directly use it to fetch, modify or use data without knowing the scraper it belongs. This feature is useful in Timeline iterator.

This is an example how Post interface is used to append two lists of Posts (maybe carrying different type of data) into one.

public void mergePost(PostTimeline list) {
    for (Post post: list) {
        this.add(post);
    }
}

 

Post as a wrapper object

While working on Post object, I converted it into a class to also use it as a wrapper. But why a wrapper? Wrapper can be used to wrap a list of Post objects into one object. It doesn’t have any identity or timestamp. It is just a utility to dump a pack of data objects with homogeneous attributes.

This is an example implementation of Post object as wrapper. typeArray is a wrapper which is used to store 2 arrays of data objects in it. These data object arrays are timeline objects that are saved as JSONArray objects in the Post wrapper.

    Post typeArray = new Post(true);
    switch(type) {
        case "users":
            typeArray.put("users", scrapeProfile(br, url).toArray());
            break;
        case "question":
            typeArray.put("question", scrapeQues(br, url).toArray());
            break;
        default:
            break;
    }

 

Resources:

 

Continue ReadingUnifying Data from Different Scrapers of loklak server using Post
Read more about the article Using Vector Images in SUSI Android

Using Vector Images in SUSI Android

SUSI is an artificial intelligence for interactive chat bots. For making it more user friendly and interactive we add a lot of images in the form of drawable resources in the SUSI Android App (https://github.com/fossasia/susi_android). Most of these drawables are in the form of PNGs. There are certain problems associated with the use of PNG images.

  1. PNGs cannot be scaled without losing quality. Due to which for the same PNG image we have to include separate images of varied quality. Otherwise the image will become blur.
  2. PNGs tends to take large disk space which can be easily reduced with the use of vector images.
  3. PNGs have fixed color and dimensions which cannot be changed.

Due to the above shortcomings of PNG images we decided to use vector drawable images instead of them.

Advantages associated with Vector images

  1. They can be scaled to any size without the loss in quality. Thus we need to include only a single image in the app and not of varied qualities.
  2. They are very small in size as compared to PNGs.
  3. They can be easily modified programmatically in XML file unlike PNGs.

Using Vector Images in Android Studio

Android Studio provide tools by which we can directly import vector drawables in the project. To import Vector images go to File>New>Vector Assets in studio.

From here we can choose the icon we want to include in our project and click OK. The icon will appear in the drawables directory and can be used anywhere in the projects.

Implementation in SUSI Android

In Susi Android we have used various vector images such as arrows, pointer and even the logo of the app. Here below is the logo of SUSI.

This is actually a vector image below we will see the code required to get this logo as the output.

<vector android:height="50dp" android:viewportHeight="279.37604"

  android:viewportWidth="1365.2" android:width="220dp" xmlns:android="http://schemas.android.com/apk/res/android">

<path android:fillColor="#ffffff"

      android:pathData="M127.5,7.7c-26.8,3.3 -54.2,16.8 -75.9,37.4 -11.8,11.1 -20.4,22.9 -28.1,38.4 -8.9,17.8 -12.8,32.1 -13.7,51l-0.3,6 39,0 39,0 0.3,-4c0.7,-12.1 6.8,-24.1 17.2,-34.5 8.5,-8.4 16.2,-13.4 25.9,-16.7l6.6,-2.2 81.3,-0.1 81.2,0 0,-38 0,-38 -84.7,0.1c-46.7,0.1 -86.1,0.4 -87.8,0.6z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M319.2,11.3l-4.3,4.3 0.3,103c0.4,113.2 0,105.9 6.4,118.6 10.8,21.3 35.1,41.9 56.2,47.3 8.5,2.3 99.1,2.2 107.7,0 18.7,-4.9 39.2,-20.7 51.5,-39.7 3.4,-5.1 7.1,-12.2 8.3,-15.8l2.2,-6.5 0.5,-103.3 0.5,-103.3 -4.5,-4.4 -4.6,-4.5 -31.5,0 -31.5,0 -4.7,4.8 -4.7,4.8 0,93 0,93 -3.3,3.2 -3.3,3.2 -29,0 -29,0 -2.6,-2.7 -2.7,-2.8 -0.7,-94.2 -0.7,-94.2 -4.3,-4 -4.2,-4.1 -31.9,0 -31.9,0 -4.2,4.3z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M680,7.6c-31.6,4.8 -56.1,17.3 -79,40.3 -23.2,23.3 -36.3,50.5 -38.9,80.9 -0.5,5.9 -0.7,11 -0.4,11.4 0.2,0.5 17.7,0.8 38.8,0.8l38.4,0 0.6,-4.8c3.2,-23.2 21.3,-44.1 44.7,-51.3 5.6,-1.8 10.6,-1.9 86.6,-1.9l80.7,0 -0.3,-38 -0.2,-38 -84.3,0.1c-46.3,0.1 -85.3,0.3 -86.7,0.5z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M869.1,13.4l-4.1,6.4 0,126.4 0,126.3 4.8,6.7 4.7,6.8 31.6,0 31.6,0 4.7,-7 4.6,-7 0,-125.7 0,-125.8 -4.7,-6.7 -4.8,-6.8 -32.1,0 -32.1,0 -4.2,6.4z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M222.5,152.2c-0.2,0.7 -0.9,4.2 -1.5,7.7 -3.4,19.5 -19.4,38 -40,46.4l-5.5,2.2 -83,0.5 -83,0.5 -0.3,37.8 -0.2,37.8 89.2,-0.3 89.3,-0.3 9.6,-2.7c57.7,-16.3 100.1,-67.4 102.1,-123.3l0.3,-7 -38.3,-0.3c-30.1,-0.2 -38.3,0 -38.7,1z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff"

      android:pathData="M774.5,152.2c-0.2,0.7 -0.9,4.1 -1.5,7.5 -3.3,19.2 -18.8,37.3 -39.4,46.2l-6.1,2.6 -83,0.5 -83,0.5 -0.3,37.7 -0.2,37.8 85.9,0c93.7,0 91.4,0.1 110.1,-5.9 26.4,-8.5 53.3,-28.4 69.8,-51.7 15.2,-21.3 25.1,-50.1 24,-69.9l-0.3,-6 -37.8,-0.3c-29.7,-0.2 -37.8,0 -38.2,1z" android:strokeColor="#00000000"/>

  <path android:fillColor="#ffffff" android:pathData="m1146.99,0 l-1.38,1.19c-0.76,0.66 -1.85,1.61 -2.43,2.13 -0.58,0.51 -1.75,1.54 -2.61,2.28 -1.52,1.31 -1.58,1.41 -2.4,3.53 -0.46,1.2 -0.92,2.37 -1.01,2.59 -30.55,82.93 -61.62,165.72 -96.03,259.63 0,0.08 1.61,1.88 3.57,3.98l3.57,3.84 33.47,-0.04 33.47,-0.04c12.28,-35.6 25.13,-72.47 37.4,-107.27 0.06,-0.25 0.28,-0.64 0.5,-0.88 0.37,-0.41 0.61,-0.43 4.2,-0.43 3.63,0 3.83,0.02"/>

  <path android:fillColor="#ffffff" android:pathData="m967.09,279.18c-2.48,-3.74 -4.97,-7.04 -8.09,-11.76l0.09,-43.92c3.34,-5.26 5.31,-6.73 8.42,-11.51 17.91,0.02 34.3,0.26 50.88,0.26 3.21,4.88 4.09,6.72 7.81,12.66 -0.05,13.98 0.1,27.96 -0.12,41.94 -2.9,4.2 -4.27,7.42 -7.78,12.18 -18.81,-0.04 -35.43,0.2 -51.21,0.15z"/>

  <path android:fillColor="#ffffff"

      android:pathData="m1287.3,6.59 l-4.1,6.4 0,126.4 0,126.3 4.8,6.7 4.7,6.8 31.6,0 31.6,0 4.7,-7 4.6,-7 0,-125.7 0,-125.8 -4.7,-6.7 -4.8,-6.8 -32.1,0 -32.1,0 -4.2,6.4z" android:strokeColor="#00000000"/>


</vector>

In this code we can easily change the color and minor details for the logo which could have been not possible if the logo was in PNG format. Also we don’t need multiple logo images of varied qualities as it can be scaled without decreasing quality.

Resources

Continue ReadingUsing Vector Images in SUSI Android