Using Open Layers 3 to Render a loklak Emoji Heatmap

In Emoji Heatmapper App I am implementing a heatmap with the help of Open Layers 3. Open Layers 3 contains a really handy class, ol.layer.Heatmap. In this blog post am going to tell you how the heatmap actually works in the backend.

A heatmap is an impressive way to visualize data. For a given matrix of data in which each value is represented by a color. The heatmap implementation is usually expensive in computation terms: For each grid’s pixel we need to compute its color from a set of known values. It is not a feasible method to be implemented on the client side because map rendering would take so much of time.

Open Layers 3 contains an easy-to-use class called ol.layer.Heatmap, which allows to render vector data as a heatmap. So how is this implemented?

The ol.layer.Heatmap layer uses a smart estimation to the design which produces relatively good results and which is also fast. The steps can be outlined as:

  • A gradient of colors is created as an image.
  • Each value is rendered in a canvas as a blurred point using the default radius and gradient. This produces a canvas where the blurred points may overlap each other and create more fuzzy zones.
  • Finally, an image is obtained from the canvas. The color is obtained from the previous image and the obtained color value may vary from 0 to 255.

Example usage of ol.layer.Heatmap class:

var heatMap = new ol.layer.Heatmap({
  source: vector,
  blur: parseInt(15, 10),
  radius: parseInt(5, 10),
  opacity: 0.9,
  gradient: ['#0000ff', '#f00', '#f00', '#ff0', '#f00']
});

 

The colored image is then rendered in the map canvas, obtaining a nice effect suited to be used for density maps. The ol.layer.Heatmap offers some properties we can use to visualize the map in a better way. The properties include blur, radius, gradient, shadow and weight. This can be configured as per feature, according to the level of importance to each feature determining in more or less measure of the final color we want.

Fig: Default colors of gradient property

Fig: Used gradient property for different colors

Resources

Continue ReadingUsing Open Layers 3 to Render a loklak Emoji Heatmap

How to add Markers in Map Fragment of Open Event Android App

The Open Event Android project helps event organizers to generate Apps (apk format) for their events/conferences by providing API endpoint or zip generated using Open Event server. In the  Open Event Android App, we have a map for showing all locations of sessions. In this map users should be able to see different locations with multiple markers. In this post I explain how to add multiple markers in the Google map fragment and set the bound in the map so that all markers are visible on one screen with specified padding.

Create Map Fragment

The first step to do is to create a simple xml file and add a fragment element in it. Then create MapsFragment.java file and find fragment element added in the xml file using findFragmentById() method.

SupportMapFragment supportMapFragment = ((SupportMapFragment)
           getChildFragmentManager().findFragmentById(R.id.map));

1. Create fragment_map.xml file

In this file add FrameLayout as a top level element and add fragment element inside FrameLayout with the name “com.google.android.gms.maps.SupportMapFragment”.

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:orientation="vertical">

       <fragment
           android:id="@+id/map"
           android:name="com.google.android.gms.maps.SupportMapFragment"
           android:layout_width="match_parent"
           android:layout_height="match_parent" />

</FrameLayout>

2. Create MapsFragment.java

MapsFragment.java extends SupportMapFragement and implements LocationListener, OnMapReadyCallback. Make instance of GoogleMap object. In onViewCreated method inflate fragment_map.xml file using inflater. Now find the fragment element added in the xml file using findFragmentById() method and assign it to SupportMapFragement instance.

public class MapsFragment extends SupportMapFragment implements LocationListener, OnMapReadyCallback {

    private GoogleMap mMap;

    @Override
    public void onViewCreated(View view, @Nullable Bundle     savedInstanceState) {
        View view = inflater.inflate(R.layout.fragment_map, container, false);
        SupportMapFragment supportMapFragment = ((SupportMapFragment)
        getChildFragmentManager().findFragmentById(R.id.map));
supportMapFragment.getMapAsync(this);

        return view;
    }

    @Override
    public void onMapReady(GoogleMap map) {  }
    ...
}

Create list of locations

Location object has three variables name, latitude & longitude. Create a list of locations for which we want to add markers to the map.

public class Location {
    private String name;
    private float latitude;
    private float longitude;
}

List<Microlocation> mLocations= new ArrayList<>();

 

Add location objects in mLocations in onViewCreated() method

mLocations.add(location);

 

You can add multiple locations using for loop or fetching from the database.

Add markers

Add following code in onMapReady(GoogleMap map) method. onMapReady(GoogleMap map) is called when map is ready to be used. setMapToolbarEnabled(true) used to show toolbar for marker. If the toolbar is enabled users will see a bar with various context-dependent actions, including ‘open this map in the Google Maps app’ and ‘find directions to the highlighted marker in the Google Maps app’.

if(map != null){
    mMap = map;
    mMap.getUiSettings().setMapToolbarEnabled(true);
}
showEventLocationOnMap();

 

Create showEventLocationOnMap() method and add following code

private void showLocationsOnMap(){

   float latitude;
   float longitude;
   Marker marker;

   //Add markers for all locations
   for (Location location : mLocations) {

       latitude = location.getLatitude();
       longitude = location.getLongitude();
       latlang = new LatLng(latitude, longitude);

       marker = mMap.addMarker(new MarkerOptions()
                .position(latlang)
                .title(location.getName()));
   }
}

 

So what we are doing here?

For each location object in mLocations list, we are creating LatLng(latitude, longitude)  object and adding it to the map using

marker = mMap.addMarker(new MarkerOptions().position(latlang).title(location.getName()));

Setting bound

We want to set the zoom level so that all markers are visible. To do this, we can use LatLngBounds.Builder. Add the position of the marker in the builder object using include method while adding the marker in the map.

LatLngBounds.Builder builder = new LatLngBounds.Builder();
builder.include(marker.getPosition());

 

Then make the LatLngBounds object from builder.

LatLngBounds bounds = builder.build();
CameraUpdate cameraUpdate = CameraUpdateFactory.newLatLngBounds(bounds, dpToPx(40));
mMap.moveCamera(cameraUpdate);

private int dpToPx(int dp) {
    return (int) (dp * Resources.getSystem().getDisplayMetrics().density);
}

 

dpTopx converts dp to px so that it set the same padding in all devices.

Conclusion

Markers in map give very nice user experience for an event or a conference because they show the relative position of places and we now offer this feature in Open Event Android.

Additional resources:

Continue ReadingHow to add Markers in Map Fragment of Open Event Android App

Improving Custom PyPI Theme Support In Yaydoc

Yaydoc has been supporting custom themes from nearly it’s inception. Themes, which it could not find locally, it would automatically try to install it via pip and set up appropriate metadata about the themes in the generated conf.py.  It was one of the first major enhancement we provided as compared to when using bare sphinx to generate documentation. Since then, a large number of features have been added to ease the process of documentation generation but the core theming aspects have remained unchanged.

To use a theme, sphinx needs the exact name of the theme and the absolute path to it. To obtain these metadata, the existing implementation accessed the __file__ attribute of the imported package to get the absolute path to the __init__.py file, a necessary element of all python packages. From there we searched for a file named theme.conf, and thus the directory containing that file was our required theme.

There were a few mistakes in our earlier implementation. For starters, we assumed that the distribution name of the theme in PyPI and the package name which should be imported would be same. This is generally true but is not necessary. One such theme from PyPI is Flask-Sphinx-Themes. While you need to install it using

pip install Flask-Sphinx-Themes

yet to import it in a module one needs to

import flask_sphinx_themes

This lead to build errors when specific themes like this was used. To solve this, we used the pkg_resources package. It allows us to get various metadata about a package in an abstract way without needing to specifically handle if the package is zipped or not.

try:
    dist = pkg_resources.get_distribution('{{ html_theme }}')
    top_level = list(dist._get_metadata('top_level.txt'))[0]
    dist_path = os.path.join(dist.location, top_level)
except (pkg_resources.DistributionNotFound, IndexError):
    print("\nError with distribution {0}".format('{{ html_theme }}'))
    html_theme = 'fossasia_theme'
    html_theme_path = ['_themes']

The idea here is that instead of searching for __init__.py, we read the name of the top_level directory using the first entry of the top_level.txt, a file created by setuptools when installing the package. We build the path by joining the location attribute of the Distribution object and the name of the top_level directory. The advantage with this approach is that we don’t need to import anything and thus no longer need to know the exact package name.

With this update, Support for custom themes has been greatly increased.

Resources

Continue ReadingImproving Custom PyPI Theme Support In Yaydoc

GlobalSearchAdapter Setup in Open Event Android App

In this blog post I describe how the GlobalSearchAdapter in Open Event Android was made which enabled users to search quickly within the app. This post also outlines how to create Recycler Views with heterogenous layouts and explains how to write ViewHolders.

Adapter Logic

A custom adapter was built for the population of views in the Recycler View in the SearchActivity.

private List<Object> filteredResultList = new ArrayList<>();
//ViewType Constants
private final int TRACK = 0;
private final int SPEAKER = 2;
private final int LOCATION = 3;
private final int DIVIDER = 4;

The DIVIDER constant was assigned to the Result Type Header View.

In a gist all the item types such as Speaker, Track, Location, Divider etc have been designated some constants.

Getting the ItemViewType

@Override
public int getItemViewType(int position) {

   if(filteredResultList.get(position) instanceof Track){
       return TRACK;
   }
   else if(filteredResultList.get(position) instanceof String){
       return DIVIDER;
   }
   ...Similarly for other ItemTypes such as Session or Location
   else{
       return 1;
   }
}

As the filteredResultList is of type Object we can insert objects of any type into the list as Object is a superclass of all classes. We would want a view which represents a TRACK if we have an object of type Track in the filteredResultList. And similarly for the other result types we could insert objects of type LOCATION, SPEAKER types in this list. getItemViewType() basically determines the type of the item that is visible to us. If the list consists of an item of type SPEAKER, in the RecyclerView.

Speaker Item Type
Track Item Type
Divider Item Type
Location Item Type

Code for onCreateViewHolder in GlobalSearchAdapter for the Recycler View

@Override
public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {

   RecyclerView.ViewHolder resultHolder = null;
   LayoutInflater inflater = LayoutInflater.from(parent.getContext());

   switch(viewType) {
       case TRACK:
           View track = inflater.inflate(R.layout.item_track, parent,   false);
           resultHolder = new TrackViewHolder(track,context);
           break;
       case SPEAKER:
           View speaker = inflater.inflate(R.layout.search_item_speaker, parent, false);
           resultHolder = new SpeakerViewHolder(speaker,context);
           break;
       //Similarly for other types
       default:
           break;
   }
   return resultHolder;
}

Depending upon the the viewType returned the desired layout is inflated and the desired ViewHolder is returned.

Code for onBindViewHolder in GlobalSearchAdapter for the Recycler View

@Override
 public void onBindViewHolder(RecyclerView.ViewHolder holder, int position) {
 
    switch (holder.getItemViewType()){
        case TRACK:
            TrackViewHolder trackSearchHolder = (TrackViewHolder)holder;
            final Track currentTrack = (Track)getItem(position);
            trackSearchHolder.setTrack(currentTrack);
            trackSearchHolder.bindHolder();
            break;
         //Similarly for all the other View Types
        default:
            break;
    }
 }

These functions are being used to bind the data to the layouts that have been inflated already in the earlier snippet of code of onCreateViewHolder.

The bindHolder functions of each ViewHolder type are being used to do the view binding i.e converting the information in the Object Track into what we see in the TrackViewHolder as seen in TrackViewFormat.

All ViewHolders have been defined as separate classes in order to enable re usability of these classes.

ViewHolder Implementation

There are 4 main ViewHolders that were made to enable such a search. I’ll be talking about the TrackViewHolder in detail.

public class TrackViewHolder extends RecyclerView.ViewHolder {
    
    @BindView(R.id.imageView)
    ImageView trackImageIcon;
    @BindView(R.id.track_title)
    TextView trackTitle;
    @BindView(R.id.track_description)
    TextView trackDescription;
 
    private Track currentTrack;
    private Context context;
    private TextDrawable.IBuilder drawableBuilder = TextDrawable.builder().round();
 
    public void setTrack(Track track) {
        this.currentTrack = track;
    }
 
    public TrackViewHolder(View itemView,Context context) {
        super(itemView);
        ButterKnife.bind(this, itemView);
        this.context = context;
    }
    public void bindHolder(){
 
        //Set all Views to their correct configurations
        itemView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                Intent intent = new Intent(context,   TrackSessionsActivity.class);
                intent.putExtra(ConstantStrings.TRACK,   currentTrack.getName());
 
                // Send Track ID to Activity to leverage color cache
                intent.putExtra(ConstantStrings.TRACK_ID,   currentTrack.getId());
                context.startActivity(intent);
            }
        });
} }

Those @BindView annotations that we can see are the result of a library called as Butterknife which is used to reduce standard boilerplate findViewById lines.

@BindView(R.id.imageView) ImageView trackImageIcon;
IS THE SAME AS THIS  
ImageView trackImageIcon = (ImageView)findViewById(R.id.imageView);

The advantage of such a ViewHolder is that it knows what kind of data it stores as compared to traditional ViewHolders which do not know the kind of data it stores.

By making ViewHolders separate from the RecyclerViewAdapter we are essentially decoupling classes and are enabling reusability of code. Also we make the ViewHolder a bit more intelligent by storing the object it binds in the ViewHolder itself. In the above example we are storing an object of Track which is bind to the ViewHolder. We also see that we do the view binding inside the viewholder itself. All this helps us to reduce code inside the adapter class.

A recent addition to the app was custom colors for all TRACKS in the app that improved the visual feel of the app. So basically, for example if a SESSION has been associated with the track of Blockchain it would be given a color such as purple. onClickListeners are also being set with some extras which are self-descriptive in nature. Similarly the other ViewHolders have been implemented.

Resources

Continue ReadingGlobalSearchAdapter Setup in Open Event Android App

Global Search in Open Event Android

In the Open Event Android app we only had a single data source for searching in each page that was the content on the page itself. But it turned out that users want to search data across an event and therefore across different screens in the app. Global search solves this problem. We have recently implemented  global search in Open Event Android that enables the user to search data from the different pages i.e Tracks, Speakers, Locations etc all in a single page. This helps the user in obtaining his desired result in less time. In this blog I am describing how we implemented the feature in the app using JAVA and XML.

Implementing the Search

The first step of the work is to to add the search icon on the homescreen. We have done this with an id R.id.action_search_home.

@Override
public void onCreateOptionsMenu(Menu menu, MenuInflater inflater) {
   super.onCreateOptionsMenu(menu, inflater);
   inflater.inflate(R.menu.menu_home, menu);
   // Get the SearchView and set the searchable configuration
   SearchManager searchManager = (SearchManager)getContext().    getSystemService(Context.SEARCH_SERVICE);
   searchView = (SearchView) menu.findItem(R.id.action_search_home).getActionView();
  // Assumes current activity is the searchable activity
 searchView.setSearchableInfo(searchManager.getSearchableInfo(
getActivity().getComponentName()));
searchView.setIconifiedByDefault(true); 
}

What is being done here is that the search icon on the top right of the home screen  is being designated a searchable component which is responsible for the setup of the search widget on the Toolbar of the app.

@Override
 public boolean onCreateOptionsMenu(Menu menu) {
    MenuInflater inflater = getMenuInflater();
    inflater.inflate(R.menu.menu_home, menu);
    SearchManager searchManager =
            (SearchManager) getSystemService(Context.SEARCH_SERVICE);
    searchView = (SearchView) menu.findItem(R.id.action_search_home).getActionView();
    searchView.setSearchableInfo(
            searchManager.getSearchableInfo(getComponentName()));
 
    searchView.setOnQueryTextListener(this);
    if (searchText != null) {
        searchView.setQuery(searchText, true);
    }
    return true; }

We can see that a queryTextListener has been setup in this function which is responsible to trigger a function whenever a query in the SearchView changes.

Example of a Searchable Component

<?xml version="1.0" encoding="utf-8"?>
 <searchable xmlns:android="http://schemas.android.com/apk/res/android"
    android:hint="@string/global_search_hint"
    android:label="@string/app_name" />

For More Info : https://developer.android.com/guide/topics/search/searchable-config.html

If this searchable component is inserted into the manifest in the required destination activity’s body the destination activity is set and intent filter must be set in this activity to tell whether or not the activity is searchable.

Manifest Code for SearchActivity

<activity
        android:name=".activities.SearchActivity"
        android:launchMode="singleTop"
        android:label="Search App"
        android:parentActivityName=".activities.MainActivity">
    <intent-filter>
        <action android:name="android.intent.action.SEARCH" />
    </intent-filter>
    <meta-data
        android:name="android.app.searchable"
        android:resource="@xml/searchable" />
 </activity>

And the attribute  android:launchMode=”singleTop is very important as if we want to search multiple times in the SearchActivity all the instances of our SearchActivity would get stored on the call stack which is not needed and would also eat up a lot of memory.

Handling the Intent to the SearchActivity

We basically need to do a standard if check in order to check if the intent is of type ACTION_SEARCH.

if (Intent.ACTION_SEARCH.equals(getIntent().getAction())) {
    handleIntent(getIntent());
 }
@Override
 protected void onNewIntent(Intent intent) {
    super.onNewIntent(intent);
    handleIntent(intent);
 }
 public void handleIntent(Intent intent) {
    final String query = intent.getStringExtra(SearchManager.QUERY);
    searchQuery(query);
 }

The function searchQuery is called within handleIntent in order to search for the text that we received from the Homescreen.

SearchView Trigger Functions

Next we need to add two main functions in order to get the search working:

  • onQueryTextChange
  • onQueryTextSubmit

The function names are self-explanatory. Now we will move on to the code implementation of the given functions.

@Override
 public boolean onQueryTextChange(String query) {
    if(query!=null) {
        searchQuery(query);
        searchText = query;
    }
    else{
        results.clear();
        handleVisibility();
    }
   return true;
 }
 
 @Override
 public boolean onQueryTextSubmit(String query) {
    searchView.clearFocus();
    return true;
 }

The role of the searchView.clearFocus() inside the above code snippet is to remove the keyboard popup from the screen to enable the user to have a clear view of the search result.

Here the main search logic is being handled by the function called searchQuery which I’ll talking about now.

Search Logic

private void searchQuery(String constraint) {
 
    results.clear();
 
    if (!TextUtils.isEmpty(constraint)) {
        String query = constraint.toLowerCase(Locale.getDefault());
        addResultsFromTracks(query);
        addResultsFromSpeakers(query);
        addResultsFromLocations(query);
    }
 }
//THESE ARE SOME VARIABLES FOR REFERENCE
//This is the custom recycler view adapter that has been defined for the search
private GlobalSearchAdapter globalSearchAdapter;
 //This stores the results in an Object Array
 private List<Object> result

We are assuming that we have POJO’s(Plain Old Java Objects) for Tracks , Speakers , and Locations and for the Result Type Header.

The code posted below performs the function of getting the required results from the list of tracks. All the results are being fetched asynchronously from Realm and here we have also attached a header for the result type to denote whether the result is of type Track , Speaker or Location. We also see that we have added a changeListener to notify us if any changes have occurred in the set of results.

Similarly this is being done for all the result types that we need i.e Tracks, Locations and Speakers.

public void addResultsFromTracks(String queryString) {

   RealmResults<Track> filteredTracks = realm.where(Track.class)
                                        .like("name", queryString,                     Case.INSENSITIVE).findAllSortedAsync("name");
      filteredTracks.addChangeListener(tracks -> {

       if(tracks.size()>0){
            results.add("Tracks");
        }
       results.addAll(tracks);
       globalSearchAdapter.notifyDataSetChanged();
       Timber.d("Filtering done total results %d", tracks.size());
       handleVisibility();
});}

We now have a “Global Search” feature in the Open Event Android app. Users had asked for this feature and a learning for us is, that it would have been even better to do more tests with users when we developed the first versions. So, we could have included this feedback and implemented Global Search earlier on.

Resources

Continue ReadingGlobal Search in Open Event Android

Open Event Server: Working with Migration Files

FOSSASIA‘s Open Event Server uses alembic migration files to handle all database operations and updations.  From creating tables to updating tables and database, all works with help of the migration files.
However, many a times we tend to miss out that automatically generated migration files mainly drops and adds columns rather than just changing them. One example of this would be:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.add_column('session', sa.Column('submission_date', sa.DateTime(), nullable=True))
    op.drop_column('session', 'date_of_submission')

Here, the idea was to change the has_session_speakers(string) to is_session_speakers_enabled (boolean), which resulted in the whole dropping of the column and creation of a new boolean column. We realize that, on doing so we have the whole data under  has_session_speakers lost.

How to solve that? Here are two ways we can follow up:

  • op.alter_column:
    ———————————-

When update is as simple as changing the column names, then we can use this. As discussed above, usually if we migrate directly after changing a column in our model, then the automatic migration created would drop the old column and create a new column with the changes. But on doing this in the production will cause huge loss of data which we don’t want. Suppose we want to just change the name of the column of start_time to starts_at. We don’t want the entire column to be dropped. So an alternative to this is using op.alter_column. The two main necessary parameters of the op.alter_column is the table name and the column which you are willing to alter. The other parameters include the new changes. Some of the commonly used parameters are:

  1. nullable Optional: specify True or False to alter the column’s nullability.
  2. new_column_name – Optional; specify a string name here to indicate the new name within a column rename operation.
  3. type_Optional: a TypeEngine type object to specify a change to the column’s type. For SQLAlchemy types that also indicate a constraint (i.e. Boolean, Enum), the constraint is also generated.
  4. autoincrement –  Optional: set the AUTO_INCREMENT flag of the column; currently understood by the MySQL dialect.
  5. existing_typeOptional: a TypeEngine type object to specify the previous type. This is required for all column alter operations that don’t otherwise specify a new type, as well as for when nullability is being changed on a column.

    So, for example, if you want to change a column name from “start_time” to “starts_at” in events table you would write:
    op.alter_column(‘events’, ‘start_time’, new_column_name=’starts_at’)
def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.alter_column('sessions_version', 'end_time', new_column_name='ends_at')
    op.alter_column('sessions_version', 'start_time', new_column_name='starts_at')
    op.alter_column('events_version', 'end_time', new_column_name='ends_at')
    op.alter_column('events_version', 'start_time', new_column_name='starts_at')


Here,
session_version and events_version are the tables name altering columns start_time to starts_at and end_time to ends_at with the op_alter_column parameter new_column_name.

  • op.execute:
    ——————–

Now with alter_column, most of the alteration in the column name or constraints or types is achievable. But there can be a separate scenario for changing the column properties. Suppose I change a table with column “aspect_ratio” which was a string column and had values “on” and “off” and want to convert the type to Boolean True/False. Just changing the column type using alte_column() function won’t work since we need to also modify the whole data. So, sometimes we need to execute raw SQL commands. To do that, we can use the op.execute() function.
The way it is done:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.execute("ALTER TABLE image_sizes ALTER full_aspect TYPE boolean USING CASE 
            full_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER icon_aspect TYPE boolean USING CASE 
            icon_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER thumbnail_aspect TYPE boolean USING CASE 
            thumbnail_aspect WHEN 'on' THEN TRUE ELSE FALSE END"execution_options=None)

For a little more advanced use of op.execute() command will be:

op.alter_column('events', 'type', new_column_name='event_type_id')
    op.alter_column('events_version', 'type', new_column_name='event_type_id')
    op.execute('INSERT INTO event_types(name, slug) SELECT DISTINCT event_type_id, 
                lower(replace(regexp_replace(event_type_id, \'& |,\', \'\', \'g\'),
                \' \', \'-\')) FROM events where not exists (SELECT 1 FROM event_types 
                where event_types.name=events.event_type_id) and event_type_id is not
                null;')
    op.execute('UPDATE events SET event_type_id = (SELECT id FROM event_types WHERE 
                event_types.name=events.event_type_id)')
    op.execute('ALTER TABLE events ALTER COLUMN event_type_id TYPE integer USING 
                event_type_id::integer')

In this example:

  • op.alter_column() renames the column type to event_type_id of events table
  • op.execute() does the following:
  • Inserts into column name of event_types table the value of event_type_idN (which previously contained the name of the event_type) from events table, and
  • Inserts into slug column of event_types table the value of event_type_id where all letters are changed to lowercase; “& ” and “,” to “”; and spaces to “-”.
    1. Checks whether a type with that name already exists so as to disallow any duplicate entries in the event_types table.
    2. Checks whether the event_type_id is null because name of event_types table cannot be null.

You can learn more on Alembic migrations here: http://alembic.zzzcomputing.com/en/latest/ops.html

Continue ReadingOpen Event Server: Working with Migration Files

Creating and Maintaining User Sessions Using Universal-Cookies in SUSI Web Chat

If you login to SUSI Web Chat, and come back again after some days, you find that you didn’t have to login and all your previous sent messages are in there in the message pane. To achieve this, SUSI Web Chat uses cookies stored in your browser which is featured in this blog.  

In ReactJS, it’s highly misleading and extensive to use the conventional Javascript methodology of saving tokens and deleting them. However, universal-cookie, a node package allows you to store and get cookies with the least possible confusion. In the following examples, I have made use of the get, set and remove functions of the universal-cookie package which have documentations easily available at this link. The basic problem one finds while setting cookies and maintaining sessions is the time through which it should be valid and to secure the routes. We shall look at the steps below to figure out how was it implemented in SUSI Web Chat.

1. The first step is to install the packages by using the following command in your project’s root-

npm install universal-cookie --save

2. Import the Cookies Class, where-ever you want to create a Cookie object in your repository.

import Cookies from 'universal-cookie';

Create a Cookie object at the same time in the file you want to use it,

const cookies = new Cookies();

3. We make use of the set function of the package first, where we try to set the cookie while the User is trying to login to the account.

Note – The cookie value can be set to any value one wants, however, here I am setting it to the access token which is generated by the server so that I can access it throughout the application.

$.ajax({ options: options,
        success: function (response) {
//Get the response token generated from the server
                let accessToken = response.access_token;                       // store the current state
                 let state = this.state;
// set the time for which the session needs to be valid
            let time = response.valid_seconds;
//set the access token in the state
             state.accessToken = accessToken;
// set the time in the state
             state.time = time;           
// Pass the accessToken and the time through the binded function
             this.handleOnSubmit(accessToken, time);
            }.bind(this),
        error: function ( jqXHR, textStatus, errorThrown) {
                   // Handle errors
                   }
        });

Function –  handleOnSubmit()

// Receive the accessToken and the time for which it needs to be valid
handleOnSubmit = (loggedIn, time) => {
        let state = this.state;
        if (state.success) {
              // set the cookie of with the value of the access token at path ‘/’ and set the time using the parameter ‘maxAge’
            cookies.set('loggedIn', loggedIn, { path: '/', maxAge: time });
// Redirect the user to logged in state and reload
            this.props.history.push('/', { showLogin: false });
            window.location.reload();
        }
        else {
        // Handle errors
    }
}

4.  To access the value set to the cookie, we make use of the get function. To check the logged in state of the User we check if get method is returning a null value or an undefined value, this helps in maintaining the User behaviour at every point in the application.

if(cookies.get('loggedIn')===null||
    cookies.get('loggedIn')===undefined) {
    // Handle User behaviours do not send chat queries with access token if the cookie is null
    url = BASE_URL+'/susi/chat.json?q='+
          createdMessage.text+
          '&language='+locale;
  }
  else{
   //  Send the messages with User’s access token
    url = BASE_URL+'/susi/chat.json?q='
          +createdMessage.text+'&language='
          +locale+'&access_token='
          +cookies.get('loggedIn');
  }

5. To delete the cookies, we make use of the remove function, which deletes that cookie. This function is called while logging the user out of the application.

cookies.remove('loggedIn');
this.props.history.push('/');
window.location.reload();

Here’s the full code in the repository. Feel free to contribute:https://github.com/fossasia/chat.susi.ai

Resources

Continue ReadingCreating and Maintaining User Sessions Using Universal-Cookies in SUSI Web Chat

Establishing Communication between PSLab and an Android Device using the USB Host API

In this post, we are going to learn how to establish communication between the PSLab USB device and a connected Android device. We will implement our own custom read & write methods by using functions provided by USB Host API of Android SDK.

At first we need to enable communication to PSLab device by connecting it to Android Phone by an On-The Go (OTG) cable. We are communicating via the USB Host API of Android.

About Android USB

Android supports USB peripherals through two modes:

  • Android Accessory: In this mode external USB device acts as host.
  • Android Host: In this mode Android Device acts as host and powers the external device.
Source : Android Developers Docs

Obtaining Permission to access USB device

When a USB device is connected to Android device, you need to obtain permissions to access the USB device. You have two ways, I have used intent-filter method to obtain permission in PSLab project, but you can also use the approach to implement a broadcast receiver.

Option 1:

Add a intent filter in the activity which would handle that connected USB device. This is an implicit way to obtain permission.

<activity ...>
...
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
    </intent-filter>
    <meta-data android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
        android:resource="@xml/device_filter" />
</activity>

And add device details like your vendor ID and product ID in device_filter.xml

<resources>

    <usb-device vendor-id="1240" product-id="223" />

</resources>

Now when you connect your USB device, permission dialog like below would pop up:

Option 2:

  • If you want to obtain permission explicitly, first create broadcastreceiver which would be broadcasted which you call requestPermission().

    private static final String ACTION_USB_PERMISSION =
        "com.android.example.USB_PERMISSION";
    private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {
        public void onReceive(Context context, Intent intent) {
            String action = intent.getAction();
            if (ACTION_USB_PERMISSION.equals(action)) {
                synchronized (this) {
                    UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
    
                    if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
                        if(device != null){
                       }
                    }
                    else {
                        Log.d(TAG, "permission denied for device " + device);
                    }
                }
            }
        }
    };

    Register this broadcastreceiver in your onCreate method of your activity.

    UsbManager mUsbManager = (UsbManager) getSystemService(Context.USB_SERVICE);
    private static final String ACTION_USB_PERMISSION =
        "com.android.example.USB_PERMISSION";
    ...
    mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
    IntentFilter filter = new IntentFilter(ACTION_USB_PERMISSION);
    registerReceiver(mUsbReceiver, filter);

    And call requestPermission method to show a dialog for permission

    UsbDevice device;
    ...
    mUsbManager.requestPermission(device, mPermissionIntent);

    Now when you open your App permission dialog like shown below would pop up:

Obtain Read & Write Endpoints

Now that you have permission to communicate with a USB device connected. Next step is to obtain read and write Endpoints to read and write to USB device by using bulkTransfer() function.

The definition of bulkTransfer() methods is

int bulkTransfer (UsbEndpoint endpoint, 
                byte[] buffer, 
                int length, 
                int timeout)

endpoint : Usb Endpoint ( the endpoint for this transaction )

buffer : byte ( buffer for data to send or receive )

length : int ( length of data to send/receive )

timeout : int ( in milliseconds, 0 is infinite )

For code to obtain read, write Endpoint through Data Interface of USB device. Open() method of PSLab can be referenced.

There are two ways for communication :

  • Synchronous
  • Asynchronous

In PSLab, we use synchronous communication using bulkTransfer() method. Create a USB device connection object

mConnection = mUsbManager.openDevice(mUsbDevice);

As bulkTransfer methods are exposed by USB connection object. Using these you can implement your read & write functions to meet your project’s requirements. Or use bulkTransfer() directly to read & write data.

For example:

mConnection.bulkTransfer(mReadEndpoint, mReadBuffer, bytesToRead, timeoutMillis)

So this covers the required for obtaining permission to access USB device and basics of how you can read data from and write data to USB device.

Also if this project interest you, feel free to contribute or raise any issue. PSLab-Android.

Resources

Continue ReadingEstablishing Communication between PSLab and an Android Device using the USB Host API

Integration Testing of an Ember Component in Open Event Frontend

Open Event Frontend uses ember components which are reused several times throughout the project, making these components is one thing but we should also ensure they work as expected. To ensure this we need integration tests.How to write an integration test for event-map component? Three files are generated when we generate our event-map component from the command shell. They are namely:

event-map.js

event-map.hbs

event-map-test.js

We are familiar with the above three files as:

  1. event-map.js helps us to provide properties to our event-map.hbs
  2. event-map.hbs is where we define the structure of our component.
  3. event-map-test.js is where we define integration test for our component. Also, which is the current topic for discussion.

Testing

In order to test the rendering of the component, we define an integration test for that component. In integration tests we don’t have to launch our whole application and navigate to the location where our component is present. This makes it ideal for testing components. Have a look at the following sample integration test file.

import { test } from 'ember-qunit';
import moduleForComponent from 'open-event-frontend/tests/helpers/component-helper';
import hbs from 'htmlbars-inline-precompile';

moduleForComponent('public/event-map', 'Integration | Component | public/event map');

let event = Object.create({ latitude: 37.7833, longitude: -122.4167, locationName: 'Sample event location address' });

test('it renders', function(assert) {
  this.set('event', event);
  this.render(hbs `{{public/event-map event=event}}`);
  assert.equal(this.$('.address p').text(), 'Sample event location address');
});

So let’s break down this code line by line-

In the first line we have imported test from ‘ember-qunit’ (default unit testing helper suite     for Ember) which contains all the required test functions. For example, here we are using test function to check to check the rendering of our component. We can use test function multiple times to check multiple components.

Next, we are importing moduleForComponent from ‘open-event-frontend/tests/helpers/component-helper’ helper which helps in finding the component by its name.

Next,  we are importing hbs from ‘htmlbars-inline-precompile’ which basically imports precompile HTMLBars template strings within the tests via ES6 tagged template strings.

The moduleForComponent helper will find the component by name (event-map) and its template. The component we are testing here is a map-related. Therefore, to test this, we need to pass a dummy object consisting of latitude, longitude and locationName. The object data must render correctly in our app.

Inside our test function, this.set(‘event’, event) assigns a variable ( here event ) to our test context.

this.render (hbs `{{public/event-map event=event}}`) lets us create a new instance of the component by declaring the component in template syntax, as we would in our application.

assert.equal(this.$(‘.address p’).text(), ‘Sample event location address’) is basically a check between actual and expected arguments.

For a simple component like a table or a basic UI only component, it is not necessary to pass any object to the component. Only the rendered component can be tested as well. Apart from checking the component rendering we can perform tests on it by using test functions as described above. After writing the integration tests for components, simply run ember test –server on the terminal to see if all the tests have passed or not.      

Find out more about Integration testing in ember –

Ember guides, EmberIgniter

Continue ReadingIntegration Testing of an Ember Component in Open Event Frontend

Using Dynamic segments to Reduce Code Redundancy of Recurring HTML in Open Event ember Frontend

While developing web apps, at times we require the same HTML for different pages in our app. This leads to redundancy and low code-reusability. This can be well managed in ember.js by using dynamic segments in our routes.

In Open Event Front-end we have a route named /sessions where we want to show the details of the event’s sessions and we want to categorize the sessions in all, pending, accepted, confirmed and rejected sessions hence want to create the following subroutes under it.

events/<event-id>/sessions
events/<event-id>/sessions/pending
events/<event-id>/sessions/accepted
events/<event-id>/sessions/confirmed
events/<event-id>/sessions/rejected

All of these subroutes show different data based on the routes in a table with exactly same fields. So if we use dynamic segments, we can decrease code redundancy and increase code reusability. So let us see how to add these subroutes as dynamic segments.

Firstly, we have to add a dynamic part (pending, accepted, confirmed, rejected) under our URL /sessions . For this, edit the following code snippet to the router.js file. In place of list write the name of route handler which handles our subroutes and in place of session_status write any identifier you want as session_status is the dynamic part which changes according to subroutes. In our case it will be pending, accepted, confirmed or rejected.

     this.route('sessions', function() {
        this.route('list', { path: '/:session_status' });
      });

To display all the sessions in our /sessions route, we have to edit index route handler and return data from model hook. Now when we hit  /sessions end point, the template which we are using redundantly, i.e. list.hbs, gets data from a model hook of this route handler.

Next we need to define a model hook in our list.js file which returns data for the dynamic routes. In list.js, we would want to change the title of our page according to the dynamic segments. These dynamic segments are available under model hook in this route under param parameter. We are using this.set which sets the provided key or path to the value. Make this available in titleToken function and by applying simple switch case, we can change the title dynamically.

Till now, we could access our dynamic segments by manually changing the URL. Let’s add a link to do this transition automatically for us. For this, we edit our session.hbs file and provide links using a linkto helper. One thing to take care here is that we should pass the dynamic segments along with the link-to helper.

{{#link-to 'events.view.sessions.list' 'pending' class='item'}}

Here, pending is the dynamic segment which we are passing to our route handler. Similarly, we can make the links for all our dynamic segments. Also in this template, we should provide outlets for the common template i.e list.hbs which will be reused by other dynamic subroutes. And finally, we define our reusable template in list.hbs file.

Now when we hit on different links we are redirected to different routes which are using different data but same templates. Also, we can see this transition in our URL and title.

To know more about dynamic segments refer to Ember guide.

Continue ReadingUsing Dynamic segments to Reduce Code Redundancy of Recurring HTML in Open Event ember Frontend