Implementing Sort By Date Feature In Susper

 

Susper has been given ‘Sort By Date’ feature which provides the user with latest results with the latest date. This feature enhances the search experience and helps users to find desired results more accurately. The sorting of results date wise is done by yacy backend which uses Apache Solr technology.

The idea was to create a ‘Sort By Date’ feature similar to the market leader. For example, if a user searches for keyword ‘Jaipur’ then results appear to be like this:

If a user wishes to get latest results, they can use ‘Sort By Date’ feature provided under ‘Tools’.

The above screenshot shows the sorted results.

You may however notice that results are not arranged year wise. Currently, the backend work for this is being going on Yacy and soon will be implemented on the frontend as well once backend provide us this feature.

Under ‘Tools’ we created an option for ‘Sort By Date’ simply using <li> tag.

<ul class=dropdownmenu>
  <li (click)=filterByDate()>Sort By Date</li>
</ul>

When clicked, it calls filterByDate() function to perform the following task:

filterByDate() {
  let urldata = Object.assign({}, this.searchdata);
  urldata.query = urldata.query.replace(/date, “”);
  this.store.dispatch(new queryactions.QueryServerAction(urldata));
}
Earlier we were using ‘last_modified desc’ attribute provided by Solr for sorting out dates in descending order. In June 2017, this feature was deprecated with a new update of Solr. We are using /date attribute in query for sorting out results which is being provided by Solr.

 

Filtering out a query in Solr for Susper

In this blog, I would like to share you how- we have implemented tabs feature in Susper.

When a search is attempted, results are fetched in results.component. Since there are many kinds of results being fetched from the server, how should we filter them in separate tabs? For example under the tab ‘All’: All kind of results are being fetched but .png or .mp3 which should be loaded under tabs ‘Images’ and ‘Videos’. The yacy server is written in Java on Solr technology. Going through Solr documents, we found the solution to filter queries using ‘fq’ parameter.

What is fq parameter?

‘fq’ stands for filter query. It returns the document which we want or in simple words returns filtered document. Documents will only be included in the result if they are at the intersection of the document sets resulting from each fq.

How we did we use fq parameter in Susper?

To explain it better, I’m sharing a code snippet here which has been written to filter out queries :

if (query[‘fq’]) {
  if (query[‘fq’].includes(‘png’)) {
    this.resultDisplay = ‘images’;
    urldata.fq = ‘url_file_ext_s:(png+OR+jpeg+OR+jpg+OR+gif)’;
} else if (query[‘fq’].includes(‘avi’)) {
    this.resultDisplay = ‘videos’;
} else {
  this.resultDisplay = ‘all’;
}
}

What is ongoing here is that we have subscribed to a query and used if and else conditions. query[‘fq’] simply filters out the query which has been subscribed. include(‘png’) and include(.avi) is clear that we are filtering out the documents with these tabs. This action happens when the user clicks on a tab.

If the user clicks on images tab: files with .png are displayed. If the user clicks on videos tab: files with .avi are displayed.

url_file_ext_s:() is simply another solr syntax to provide the document format.

The flowchart above explains more clearly, how fq parameter is filtering out documents without affecting the total number of documents which yacy fetches by indexing web pages based on the query.

Resources: