Fixing Infinite Scroll Feature for Susper using Angular

In Susper, we faced a unique problem. Every time the image tab was opened, and the user scrolled through the images, all the other tabs in the search engine, such as All, Videos etc, would stop working. They would continue to display image results as shown:

Since this problem occurred only when the infinite scroll action was called in the image tab, I diagnosed that the problem probably was in the url parameters being set.

The url parameters were set in the onScroll() function as shown:

onScroll () {
let urldata = Object.assign({}, this.searchdata);
this.getPresentPage(1);
this.resultDisplay = ‘images’;
urldata.start = (this.startindex) + urldata.rows;
urldata.fq = ‘url_file_ext_s:(png+OR+jpeg+OR+jpg+OR+gif)’;
urldata.resultDisplay = this.resultDisplay;
urldata.append = true;
urldata.nopagechange = true;
this.store.dispatch(new queryactions.QueryServerAction(urldata));
};

The parameters append and nopagechange were to ensure that the images are displayed in the same page, one after the other.
To solve this bug I first displayed the query call each time a tab is clicked on the web console.
Here I noticed that for the tab videos, nopagechange and append attributes still persisted, and had not been reset. The start offset had not been set to 0 either.
So adding these few lines before making a query call from any tab, would solve the problem.

urldata.start = 0;
urldata.nopagechange = false;
urldata.append = false;

Now the object is displayed as follows:

Now videos are displayed in the videos tab, text in the text tab and so on.
Please refer to results.component.ts for the entire code.

References:

  1. On how to dispatch queries to the store: https://gist.github.com/btroncone/a6e4347326749f938510
  2. Tutorial on the ngrx suite:http://bodiddlie.github.io/ng-2-toh-with-ngrx-suite/

Implementing Sort By Date Feature In Susper

 

Susper has been given ‘Sort By Date’ feature which provides the user with latest results with the latest date. This feature enhances the search experience and helps users to find desired results more accurately. The sorting of results date wise is done by yacy backend which uses Apache Solr technology.

The idea was to create a ‘Sort By Date’ feature similar to the market leader. For example, if a user searches for keyword ‘Jaipur’ then results appear to be like this:

If a user wishes to get latest results, they can use ‘Sort By Date’ feature provided under ‘Tools’.

The above screenshot shows the sorted results.

You may however notice that results are not arranged year wise. Currently, the backend work for this is being going on Yacy and soon will be implemented on the frontend as well once backend provide us this feature.

Under ‘Tools’ we created an option for ‘Sort By Date’ simply using <li> tag.

<ul class=dropdownmenu>
  <li (click)=filterByDate()>Sort By Date</li>
</ul>

When clicked, it calls filterByDate() function to perform the following task:

filterByDate() {
  let urldata = Object.assign({}, this.searchdata);
  urldata.query = urldata.query.replace(/date, “”);
  this.store.dispatch(new queryactions.QueryServerAction(urldata));
}
Earlier we were using ‘last_modified desc’ attribute provided by Solr for sorting out dates in descending order. In June 2017, this feature was deprecated with a new update of Solr. We are using /date attribute in query for sorting out results which is being provided by Solr.

 

Doing a table join in Android without using rawQuery

The Open Event Android App, downloads data from the API (about events, sessions speakers etc), and saves them locally in an SQLite database, so that the app can work even without internet connection.

Since there are multiple entities like Sessions, Speakers, Events etc, and each Session has ids of speakers, and id of it’s venue etc, we often need to use JOIN queries to join data from two tables.

 

Android has some really nice SQLite helper classes and methods. And the ones I like the most are the SQLiteDatabase.query, SQLiteDatabase.update, SQLiteDatabase.insert ones, because they take away quite a bit of pain for typing out SQL commands by hand.

But unfortunately, if you have to use a JOIN, then usually you have to go and use the SQLiteDatabase.rawQuery method and end up having to type your commands by hand.

But but but, if the two tables you are joining do not have any common column names (actually it is good design to have them so – by having all column names prefixed by tablename_ maybe), then you can hack the usual SQLiteDatabase.query() method to get a JOINed query.

Now ideally, to get the Session where speaker_id was 1, a nice looking SQL query should be like this –

SELECT * FROM speaker INNER JOIN session
ON speaker_id = session_speaker_id
WHERE speaker_id = 1

Which, in android, can be done like this –

String rawQuery = "SELECT * FROM " + SpeakerTable.TABLE_NAME + " INNER JOIN " + SessionTable.TABLE_NAME
        + " ON " + SessionTable.EXP_ID + " = " + SpeakerTable.ID
        + " WHERE " + SessionTable.ID + " = " +  id;
Cursor c = db.rawQuery(
        rawQuery,
        null
);

But of course, because of SQLite’s backward compatible support of the primitive way of querying, we turn that command into

SELECT *
FROM session, speaker
WHERE speaker_id = session_speaker_id AND speaker_id = 1

Now this we can write by hacking the terminology used by the #query() method –

Cursor c = db.query(
        SessionTable.TABLE_NAME + " , " + SpeakerTable.TABLE_NAME,
        Utils.concat(SessionTable.PROJECTION, SpeakerTable.PROJECTION),
        SessionTable.EXP_ID + " = " + SpeakerTable.ID + " AND " + SpeakerTable.ID + " = " +  id,
        null,
        null,
        null,
        null
);

To explain a bit, the first argument String tableName can take table1, table2 as well safely, The second argument takes a String array of column names, I concatenated the two projections of the two classes. and finally, put by WHERE clause into the String selection argument.

You can see the code for all database operations in the android app here  https://github.com/fossasia/open-event-android/blob/master/android/app/src/main/java/org/fossasia/openevent/dbutils/DatabaseOperations.java