Implementing Advanced Search Feature In Susper

Susper has been provided ‘Advanced Search’ feature which provides the user a great experience to search for desired results. Advanced search has been implemented in such a way it shows top authors, top providers, and distribution regarding protocols. Users can choose any of these options to get best results.

We receive data of each facet name from Yacy using yacy search endpoint. More about yacy search endpoint can be found here:  http://yacy.searchlab.eu/solr/select?query=india&fl=last_modified&start=0&rows=15&facet=true&facet.mincount=1&facet.field=host_s&facet.field=url_protocol_s&facet.field=author_sxt&facet.field=collection_sxt&wt=yjson

For implementing this feature, we created Actions and Reducers using concepts of Redux. The implemented actions can be found here: https://github.com/fossasia/susper.com/blob/master/src/app/actions/search.ts

Actions have been implemented because these actually represent some kind of event. For e.g. like the beginning of an API call here.

We also have created an interface for search action which can be found here under reducers as filename index.ts: https://github.com/fossasia/susper.com/blob/master/src/app/reducers/index.ts

Reducers are a pure type of function that takes the previous state and an action and returns the next state. We have used Redux to implement actions and reducers for the advanced search.

For advanced search, the reducer file can be found here: https://github.com/fossasia/susper.com/blob/master/src/app/reducers/search.ts

The main logic has been implemented under advancedsearch.component.ts:

export class AdvancedsearchComponent implements OnInit {
  querylook = {}; // array of urls
  navigation$: Observable<any>;
  selectedelements: Array<any> = []; // selected urls by user
changeurl
(modifier, element) {
// based on query urls are fetched
// if an url is selected by user, it is decoded
  this.querylook[‘query’] = this.querylook[‘query’] + ‘+’ + decodeURIComponent(modifier);
  this.selectedelements.push(element);
// according to selected urls
// results are loaded from yacy
  this.route.navigate([‘/search’], {queryParams: this.querylook});
}

// same method is implemented for removing an url
removeurl(modifier) {
  this.querylook[‘query’] = this.querylook[‘query’].replace(‘+’ + decodeURIComponent(modifier), );

  this.route.navigate([‘/search’], {queryParams: this.querylook});
}

 

The changeurl() function replaces the query with a query and selected URL and searches for the results only from the URL provider. The removeurl() function removes URL from the query and works as a normal search, searching for the results from all providers.

The source code for the implementation of advanced search feature can be found here: https://github.com/fossasia/susper.com/tree/master/src/app/advancedsearch

Resources

Writing Browser Specific CSS for Susper in Angular

In Susper, we were facing a unique problem for Information box and Analytics box alignment.
At a width of around 1290 px, the boxes fit perfectly in Firefox as shown:

However, they were slipping to the next line in Google Chrome browsers for the same dimension(1290 px)

The solution to this issue was to write browser-specific CSS.
The two most commonly used browser specific tags are

  1. @-moz-document url-prefix() { }: This tag is used to target the Mozilla Firefox browser in particular. Anything written within the curly braces will not apply to any other browser.
  2. @media screen and (-webkit-min-device-pixel-ratio:0) { }: This tag is used to target all browsers that support webkit such as Chrome, Safari etc.

For our problem, we need to use @media screen and (-webkit-min-device-pixel-ratio:0) { }
This was how the code was written for both the components (Information box and Analytics box). Please refer to infobox.component.css and statsbox.component.css for the entire code.

@media screen and (-webkit-min-device-pixel-ratio:0) {
@media screen and (max-width: 1300px) {
.card {
width: 366px;
}
}
}
@media screen and (max-width: 1280px) {
.card {
width: 366px;
}

As a result of this snippet of code, we see the following effects:

  • In Chrome, the boxes change to a smaller width at 1300px itself, thus preventing it from slipping to the next line
  • In Firefox, the boxes change to a smaller width only at 1280px, and not at 1300px, thus achieving the exact design we envisioned.

This is how the display finally looks in Chrome:

References:

  1. Stack overflow on specific CSS tags for Chrome: https://stackoverflow.com/questions/9328832/how-to-apply-specific-css-rules-to-chrome-only
  2. Stack overflow on specific CSS tags for Firefox: https://stackoverflow.com/questions/952861/targeting-only-firefox-with-css

Making a Sticky Top Navigation bar for Susper using Angular

A lot of websites, require a top navigation bar that sticks to the top, irrespective of the screen dimension size. This blog deals with how the top navigation bar was made sticky in Susper.

  1. Using the correct Bootstrap classes. Notice the code enveloping the navigation bar.

<nav class=“top-nav navbar navbar-static-top navbar-default”>

class=“container-fluid”>

class=“navbar-header” id=“navcontainer”>

</div>
</div>
</nav>

Points to note:

  • Using navbar and navbar-default creates a standard gray navigation bar.
  • Using navbar-static-top makes the navbar stick only to the top of the page and disappear on scrolling down.
  • Using container-fluid creates a container for the contents of the navbar with wide margins
  1.  Now we also need to write some personalized CSS code. Notice the classes navcontainer and  top-nav. This is the CSS code for these classes:

.top-nav{
margin-bottom: 0;
}
#navcontainer {
height: 65px;
width: 100vw;
}#navcontainer ul {
margin: 0;
padding: 0;
list-style-type: none;
}

Points to note:

  • Margin and padding can be set according to how the navbar should look. Click here to know the difference between margins and padding.
  • The height has been customized to 65px in Susper, with a width of 100vw(entire viewpost width).
  1. Lastly, if your navigation bar is inside the body tag, remember that by default, body has a top margin of 57 px. As a result you may see an extra white space on top of your navigation bar. To remove this:
  • Move the navigation bar code out of the body tag. If you can’t then,
  • Place your navigation bar in a container ( resultContainer on the Susper result page) and write this in your CSS file.

.resultContainer{
margin-top: -57px;
}

References:

Making Customized and Mobile Responsive Drop-down Menus in Susper using Angular

In  Susper, the drop-down menu is customized with colorful search icons and we wanted to maintain the same menu for mobile screens too, however the drop-down menu disappeared for all screens with width less than 767px. This blog can be used to learn how to create css classes for such drop-down menus without using any bootstrap.
This is how the issue was solved.

  1. Replacing standard bootstrap classes : The drop-down menu blocks had a source code as follows:

class=“dropdown-menu”>

class=“row”>

class=“col-sm-4”>

class=“block”>

</div>
</div>

Using col-sm-4 will do the following

  • For widths greater than 767px: Divide each row into four equally sized columns.
  • For widths smaller than 767px: Stack all the columns on top of each other.

Since the drop-down menu’s design was to remain intact, I made the following changes:

  • Replace row with menu-row
  • Replace col-sm-4 with menu-item

Now I wrote personalized css for these classes.

.menu-row{
width: 267px;
gridtemplatecolumns: 1fr 1fr 1fr;
background-color: white;
}
.menu-item{
display: inlineblock;
width: 86px;
}
  • Width: It is used to set the width of the div class, each row now has a width of 267px, with each column in it having a width of 86px.
  • Grid-template-columns: It is used to layout the structure of the template, here 1fr 1fr 1fr represents that there will be three columns in a row.
  • Display: The display is set to inline block to overwrite the default property of the div element to start in a new line.
  1. Custom css for small screens : In standard bootstrap, for screen sizes less than 767px, dropdown class has properties like transparent background, no border etc. that need to be over written. So we add a new id for the div tag as shown:

<div id=“small-drop” class=“dropdown-menu”>

/** Now we add css for it, as shown: **/
@media screen and (max-width: 767px) {
#small-drop{
position: absolute;
background-color: white;
border: 1px solid #cccccc;
right: -38px;
left: auto;
}

  • Position : absolute is used to make sure all our values are absolute and not relative to the higher div hierarchically
  • Border: The values for the border represent the following respectively: Thickness, Style and Color.
  • Auto: Here the value auto for left signifies that there is no fixed value for the left margin, it can take the default value

References:

  1. For working of grids in Bootstrap: https://www.w3schools.com/bootstrap/bootstrap_grid_examples.asp
  2. A useful article for difference between id and class: https://css-tricks.com/the-difference-between-id-and-class

 

Designing a Remote Laboratory with PSLab using Python Flask Framework

In the introductory post about remote laboratories, a general set of tools to create a framework and handle its various aspects was also introduced. In this blog post, we will explore the implementation of several aspects of the backend app designed with python-flask, and the frontend based on EmberJS. A clear separation of the frontend and backend facilitates minimal disruption of either sections due to the other.

Implementing API methods in Python-Flask

In the Flask web server, page requests are handled via ‘routes’ , which are essentially URLs linked to a python function. Routes are also capable of handling payloads such as POST data, and various return types are also supported.

We shall use an example to demonstrate how a Sign-Up request sent from the sign-up form in the remote lab frontend for PSLab is handled.

@app.route('/signUp',methods=['POST'])
def signUp():
	"""Sign Up for Virtual Lab

	POST: Submit sign-up parameters. The following must be present:
	 inputName : The name of your account. does not need to be unique
	 inputEmail : e-mail ID used for login . must be unique.
	 inputPassword: password .
	Returns HTTP 404 when data does not exist.
	"""
	# read the posted values from the UI
	_name = request.form['inputName']
	_email = request.form['inputEmail']
	_password = request.form['inputPassword']

	# validate the received values
	if _name and _email and _password:
		_hashed_password = generate_password_hash(_password)
		newUser = User(_email, _name,_hashed_password)
		try:
			db.session.add(newUser)
			db.session.commit()
			return json.dumps({'status':True,'message':'User %s created successfully. e-mail:%s !'%(_name,_email)})
		except Exception as exc:
			reason = str(exc)
			return json.dumps({'status':False,'message':str(reason)})

 

In this example, the first line indicates that all URL requests made to <domain:port>/signUp will be handled by the function signUp . During development, we host the server on localhost, and use the default PORT number 8000, so sign-up forms must be submitted to 127.0.0.1:8000/signUp .

For deployment on a globally accessible server, a machine with a static IP, and a DNS record must be used. An example for such a deployment would be the heroku subdomain where pslab-remote is automatically deployed ; https://pslab-stage.herokuapp.com/signUp

A closer look at the above example will tell you that POST data can be accessed via the request.form dictionary, and that the sign-up routine requires inputName,inputEmail, and inputPassword. A password hash is generated before writing the parameters to the database.

Testing API methods using the Postman chrome extension

The route described in the above example requires form data to be submitted along with the URL, and we will use a rather handy developer tool called Postman to help us do this. In the frontend apps , AJAX methods are usually employed to do such tasks as well as handle the response from the server.

 

The above screenshot shows Postman being used to submit form data to /signUp on our API server running at localhost:8000 . The fields inputName, inputDescription, and inputPassword are also posted along with it.

In the bottom section, one can see that the server returned a positive status variable, as well as a descriptive message.

Submitting the sign up form via an Ember controller.
  • Setting up a template
    We first need to set up a template that we shall call sign-up.hbs , and add the following form to it. This form contains the details essential for signing up , and its submit action is linked to an action called `signMeUp` . This action will be defined in the controller which we shall explore shortly

<form class="form-signin" {{action "signMeUp" on="submit"}} >
        <label for="inputName" class="sr-only">Your Name</label>
        {{input value=inputName type="text" name="inputName" id="inputName" class="form-control" placeholder="name" required=true autofocus=true}}
        <label for="inputEmail" class="sr-only">Email address</label>
        {{input value=inputEmail type="email" name="inputEmail" id="inputEmail" class="form-control" placeholder="Email address" required=true autofocus=true}}
        <label for="inputPassword" class="sr-only">Password</label>
        {{input value=inputPassword type="password" name="inputPassword" id="inputPassword" class="form-control" placeholder="Password" required=true autofocus=true}}
         
        <button class="btn btn-lg btn-primary btn-block" type="submit">Sign Up</button>
</form>

 

  • Defining the controller
    The controller contains the actions and variables that the template links to. In this case, we require an action called signMeUp. The success, failure, and error handlers are hidden for clarity.

import Ember from 'ember';
export default Ember.Controller.extend({
  actions:{
    signMeUp() {
        var request = Ember.$.post("/signUp",
 this.getProperties("inputName","inputEmail","inputPassword"),this,'json');
        request.then(this.success.bind(this), this.failure.bind(this),
this.error.bind(this));
    },
  },
});

The signMeUp action submits the contents of the form to the signUp route on the API server, and the results are handled by functions called success, failure, or error depending on the type of response from the backend server.

Resources:

 

Reducing Initial Load Time Of Susper

Susper used to take long time to load the initial page. So, there was a discussion on how to decrease the initial loading time of Susper.

Later on going through issues raised in official Angular repository regarding the time takento load angular applications, we found some of the solutions. Those include:

  • Migrating from development to production during build:
    • This shrinks vendor.js and main.js by minimising them and also removing all packages that are not required on production.
    • Enables inline html and extracting CSS.
    • Disables source maps.
  • Enable Ahead of Time (AoT) compilation.
  • Enable service workers.

After these changes we found the following changes in Susper:

File Name Before (content-length) After
vendor.js 709752 216764
main.js 56412 138361

LOADING TIMES GRAPHS:

After:

Vendor file:

Main.js

Before:

Vendor file:

Main.js

Also we could see that all files are now initiated by service worker:

More about Service Workers could be read at Mozilla and Service Workers in Angular.

Implementation:

While deploying our application, we have added –prod and –aot as extra attributes to ng build .This enables angular to use production mode and ahead of time compilation.

For service workers we have to install @angular/service-worker module. One can install it by using:

npm install @angular/service-worker --save
ng set apps.0.serviceWorker=true

The whole implementation of this is available at this pull:

https://github.com/fossasia/susper.com/pull/597/files

Resources:

 

 

Hiding Intelligence of Susper When a Query is Empty or Erased with Angular

Recently, we have implemented intelligence feature in Susper using SUSI chat API to provide users answer a question without going deeper in search results. When a user types “height of Trump”, it shows answer like this:

Problem which we faced after implementing the feature:

When a user was erasing a query or query field was empty, Susper was still showing the answer of the intelligence component like this:

The answer should not be displayed when a query is empty because the user is not asking any question. The answer was still displayed because it had received a response from SUSI API.

How did we solve the problem?

The problem was solved in two ways.

  1. By using if/else condition: We checked if the statement shown inside the component is similar to the if-and-else condition. If the condition is true, it should hide the component.
  2. Using [hidden] attribute method: The Angular 4 supports [hidden] attribute which acts as { display:none; } . [hidden] attribute generally works as ngShow and ngHide which was earlier supported by Angular 2.

We preferred both the methods to solve the problem. The intelligence component is being loaded inside results component using <app-intelligence> element. Further, we added [hidden] attribute to this element like this :

<appintelligence [hidden]=“hideIntelligence”></app-intelligence>
We created hideIntelligence as variable and assign it as boolean. To check if a query is empty, searchdata variable was used.
searchdata: any = {
  query: ‘ ‘,
  rows: 10,
  start: 0
};
And then checked if a query is empty using if-else condition :
// checks if query is empty or erased
if (this.searchdata.query === ‘ ‘) {// display: none; is true
  this.hideIntelligence = true;

} else {
// display: none; is false
  this.hideIntelligence = false;
}

 

Applying this solution, we succeeded in hiding the intelligence component. We would also had used *ngIf statement but we preferred using [hidden]. [hidden] modifies the display property.  *ngIf is a structural directive which creates or destroys content inside DOM.

The source code for the implementation can be found here: https://github.com/fossasia/susper.com/pull/613

Resources:

 

Implementing Intelligence Feature in Susper

Susper gives answers to your questions using SUSI AI. We want to give users best experience while they are searching for solutions to their questions. To achieve this, we have incorporated with features like infobox and intelligence using SUSI.

Google has this feature where users can ask questions like ‘Who is president of USA?’ and get answers directly without encouraging the users to deep-dive into the search results to know the answer.

Similarly Susper gives answer to the user:

It also gives answer to question which is related to real time data like temperature.

 

How we have implemented this feature?

We used the API Endpoint of SUSI at http://api.asksusi.com/

Using SUSI API is as simple as sending query as a URL parameter in GET request http://api.susi.ai/susi/chat.json?q=YOUR_QUERY

You can also get various action types in the response. Eg: An anwser type response for http://api.susi.ai/susi/chat.json?q=hey%20susi is:

actions: [
  {
    type: "answer",
    expression: "Hi, I'm Susi"
  }
],

 

Documentation regarding SUSI is available at here.

Implementation in Susper:

We have created an Intelligence component to display answer related to a question. You can check it here: https://github.com/fossasia/susper.com/tree/master/src/app/intelligence

It takes care about rendering the information and styling of the rendered data received from SUSI API.

The intelligence.component.ts makes a call to Intelligence Service with the required query and the intelligence service makes a GETrequest to the SUSI API and retrieves the results.

Intelligence.component.ts

this.intelligence.getintelligentresponse(data.query).subscribe(res => {
  if (res && res.answers && res.answers[0].actions) {
     this.actions = res.answers[0].actions;
       for (let action of this.actions) {
         if (action.type === 'answer' && action.mood !== 'sabta') {
           this.answer = action.expression;
         } else {
             this.answer = '';
         }
      }
   } else {
       this.answer = '';
   }
});

 

Intelligence.service.ts

export class IntelligenceService {
 server = 'http://api.susi.ai';
 searchURL = 'http://' + this.server + '/susi/chat.json';
 constructor(private http: Http, private jsonp: Jsonp, private store: Store<fromRoot.State>) {
 }
 getintelligentresponse(searchquery) {
   let params = new URLSearchParams();
   params.set('q', searchquery);
   params.set('callback', 'JSONP_CALLBACK');
   return this.jsonp
     .get('http://api.asksusi.com/susi/chat.json', {search: params}).map(res =>
       res.json()

     );
 }

Whenever the getintelligenceresponse of intelligenceService is called, it creates a URLSearchParams() object and set required parameters in it and send them in jsonp.get request. We also set callback to ‘JSONP_CALLBACK’ to inform the API to send us data in JSONP.

Thereby, the intelligence component retrieves the answer and displays it with search resultson Susper.

Source code for this implementation could be found in this pull:

https://github.com/fossasia/susper.com/pull/569

Resources:

Aligning Images to Same Height Maintaining Ratio in Susper

In this blog, I’ll be sharing the idea how we have perfectly aligned images to the same height in Susper without disturbing their ratio. When it comes to aligning images perfectly, they should have:

  • Same height.
  • A proper ratio to maintain the image quality. Many developers apply same width and height without keeping in mind about image ratio which results in:
    • Blurred image,
    • Image with a lot of pixels,
    • Cropping of an image.

Earlier Susper was having image layout like this:

In the screenshot, images are not properly aligned.  They are also not having the same height. We wanted to improve the layout of images just like market leaders Google and DuckDuckGo.

  • How we implemented a better layout for images?

<div class=“container”>
  <div class=“grid” *ngIf=“Display(‘images’)”>
    <div class=“cell” *ngFor=“let item of item$ | async”>
      <a class=“image-pointer” href=“{{item.link}}”>
        <img class=“responsive-image” src=“{{item.link}}”></a>
    </div>
  </div>
</div>
I have created a container, in which images will be loaded from yacy server. Then I have created a grid with an equal number of rows and column. I have adjusted the height and width of rows and columns to obtain a grid which contains each division as a cell. The image will load inside the cell. Each cell consists of one image.
.grid {
  paddingleft: 80px;
}
.container {
  width: 100%;
  margin: 0;
  top: 0;
  bottom: 0;
  padding: 0;
}

After implementing it, we were facing issues like cropping of the image inside a cell. So, to avoid cropping and maintain the image ratio we introduced .responsive-image class which will avoid cropping of images inside cell.

.responsiveimage {
  maxwidth: 100%;
  height: 200px;
  paddingtop: 20px;
  padding: 0.6%;
  display: inlineblock;
  float: left;
}

This is how Susper’s image section looks now:

It took some time to align images, but somehow we succeeded in creating a perfect layout for the images.

We are facing some issues regarding images. Some of them don’t appear due to broken link. This issue will be resolved soon on the server.

Resources

Handlebars.js used in Open Event Web App

I recently started working in the Open Event Webapp project. One of the initial issues that I took up was a trivial UI bug. It was about adding sponsor names beneath sponsor images for better representation. The issue can be found here. On reading up the code base and exploring the project a bit, I came across a new template – Handlebars.js. Handlebars is a template which has it’s base with the Mustache templating language. One of the early discoveries that I made with Handlebars.js was the use of {{ }} and {{{ }}} and the basic difference between them. In general, all Handlebar.js expressions, just like in Mustache templating, are written between {{ }} or {{{ }}} type of brackets. That is how I learned to identify and distinguish Handlebars from core HTML, even though they are inter-linked. The official Handlebars documentation describes Handlebars expressions in the following way:

A handlebars expression is a {{, some contents, followed by a }} ”

Getting started with Handlebars.js

Installation:

For a basic Linux installation, type the following in your command line:

npm install --save handlebars

Including Handlebars in HTML:

<script src="handlebars-v4.0.10.js"></script>

Handlebars templates are often stored in .hbs files for better readability and accessibility. The Open Event Webapp project consists of a handlebars .hbs file for each of the tracks, events, rooms, schedule, sessions and speakers templates. These can be found here, that is under src/backend/templates folder.

Difference between {{ }} and {{{ }}}: 

Handlebars enables developers to print raw HTML tags or code with the help of {{{ }}}. On the contrary, if you don’t want to print HTML (which is usually the case), use {{ }}. For better understanding, let’s take an example.

If our JS has an object that looks something like:

$(function () {
   var templateScript = $("#title-template").html();

   var temp = Handlebars.compile(templateScript);

var Title= {
“title”: <a> Handlebars</a>
}

Then, HTML of the following kind will help to distinguish the {{ }} and {{{ }}} brackets.

<script id=”title-template” type=”text/x-handlebars-template”>
{{title}}
{{{title}}}
</script>

//the first line will contain an anchor tag with the name “Handlebars”
//the second line will contain “<a>Handlebars</a>”

Block helpers in Handlebars:

Block helpers are identified by a ‘ #’ and they help to define and access custom iterators.

Handlebars allow calling JavaScript functions with the help of ‘helpers’. It doesn’t allow direct JavaScript code in the HTML with templates. We can create our own helpers using Handlebars.registerHelper () in our JavaScript. We generally pass a function to the helper. A good example was provided in the Handlebars.js documentation:

Handlebars.registerHelper('noop', function(options) {
  return options.fn(this);
});

By default, Handlebars helpers take the current context as the context to pass(“this”). Other fields are overshadowed. Incase, we want to access one of the fields that is masked by the default “this” context, we have to use a path reference.

Iterations using helpers:

Helpers can be a great way  to iterate over lists or objects. I will demonstrate it with an example from the Open Event Webapp project. To display all the sponsors of an event in the home page of the event Webapp, we use the following handlebars code, where we iterate over the object list “sponsorpics” that we have. It looks something like this:

{'1': ['Oreilly', 'Amazon'], '2': ['Huawei', 'Google'],'3': ['RedHat', 'GitHub']}
     
{{#if eventurls.sponsorsection}}
<div class="sponsor-container">
       <section class="sponsorscont">
         <div class="row sponsor-row">
           <div class="col-sm-12 col-md-12 col-xs-12 text-center">
             <h1 class="section-header">Proudly supported by</h1><br>
           </div>
         </div>
         <div class="row">
           <div class="col-sm-10 col-sm-offset-1">



             <div class="row">
               {{#each sponsorpics}}
                 {{#each this}}
                   <div class="{{{divclass}}}">
                     <div class=" {{{sponsorimg}}} text-center">
                       <a href="{{{url}}}" data-toggle="tooltip" title="{{{type}}}">
                         <img class="lazy centre {{{imgsize}}}" alt="{{{name}}}" data-original="{{{logo}}}">
                       </a>
                       {{{name}}}
                     </div>
                   </div>
                 {{/each}}
               {{/each}}
             </div> <!-- sponsor-row -->
           </div>
         </div>
       </section>
     </div>
   {{/if}}

For your reference, you can view a sample Webapp for the OSCON 2017 event here.
For further information, please refer to Handlebars.js .
An interesting tutorial about Handlebars in 10 mins or less can be found here.