Adding Susper as standard search engine in Firefox and Chrome

Have you ever seen on the sites like Google and DuckDuckGo, when a user visits their website they provide an option to add their search engine as default search provider. Similarly, we raised up an issue regarding the implementation of this feature.

Currently, we have implemented this feature but facing some issues. Here is a screenshot:

Code for implementing this feature:

  $(document).ready(function() {
    var isFirefox = typeof InstallTrigger!=='undefined';

     if (isFirefox === false) {
       $("#set-susper-default").remove();
       $(".input-group-btn").addClass("align-search-btn");
       $("#navbar-search").addClass("align-navsearch-btn");
     }

     if (window.external && window.external.IsSearchProviderInstalled) {
       var isInstalled = window.external.IsSearchProviderInstalled("http://susper.com");

     if (!isInstalled) {
       $("#set-susper-default").show();
     }
    }

    $("#install-susper").on("click", function() {
      window.external.AddSearchProvider("http://susper.com/susper.xml");
    });

    $("#cancel-installation").on("click", function() {
      $("#set-susper-default").remove();
    });
  });


You can see, right side a box to add search provider. But this feature was not working for Chrome since it has some different feature to add search providers because of AddSearchProvider() and window.external.IsSearchProviderInstalled() works only with some versions of Firefox and earlier they worked with Chrome as well but now they don’t. So to remove this feature from Chrome browser, we simply commented out the lines which are highlighted in the code as bold.

We are still having an issue like, if a user adds the search provider in Firefox by installing it and then he/she visits the site next time, that option still appears to be there. It should not appear again if Susper has been already added as default search provider. We’re working on it though.

  • What else we did for other browsers compatibility?
    We picked up the idea of adding an OpenSearch feature. It auto-discovers the search providers. What we did simply:
  1. Create an XML file as  yoursitename.xml and configure it like this:
    <?xml version="1.0" encoding="UTF-8"?>
    <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
    <ShortName>[Name of your website]</ShortName>
    <Description>[Description of your website]</Description>
    <Url type="text/html" template="http://www.yourwebsite.com/search?q=site:[Site host] {searchTerms}"/>
    </OpenSearchDescription>
  2. Add an auto-discovery link in your main index.html file:
<link rel="search" href="//susper.com/susper.xml" type="application/opensearchdescription+xml" title="susper.com"/>

We’re done! To see it in action, refresh the browser. The browser will auto-detect and tell you that a custom search engine was discovered and you can customise it according to your choice.

Check our susper.xml here:
https://github.com/fossasia/susper.com/blob/master/susper.xml

Continue ReadingAdding Susper as standard search engine in Firefox and Chrome

How we implemented an InfoBox similar to Google in Susper

Research Work: This was initially proposed by @mariobehling , https://github.com/fossasia/susper.com/issues/181, where he proposed an idea of building an infobox similar to Google or Duckduckgo.

Later Michael Christen 0rb1t3r referenced DBpedia API, which can get a structured data from Wikipedia information.

One example of using the DBpedia API is: http://lookup.dbpedia.org/api/search/KeywordSearch?QueryClass=place&QueryString=berlin

More information about the structured Knowledge Graphs is available at https://en.wikipedia.org/wiki/Knowledge_Graph

Implementation:

We created an infobox component to display the data related to infobox https://github.com/fossasia/susper.com/tree/master/src/app/infobox

It takes care about rendering the information, styling of the rendered data retrieved from the DBpedia API

Infobox.component.html :

<div *ngIf="results?.length > 0" class="card">

<div>

<h2><b>{{this.results[0].label}}</b></h2>

<p>{{this.results[0].description}}</p>

</div>

<div class="card-container">

<h3><b>Related Searches</b></h3>




<div *ngFor="let result of results">

   <a [routerLink]="resultsearch" [queryParams]="{query: result.label}">{{result.label}}</a>

</div>

</div>

</div>

The infobox.component.ts makes a call to Knowledge service with the required query, and the knowledge service makes a get request to the DBpedia API and retrieves the results.

infobox.component.ts

this.query$.subscribe( query => {

if (query) {

   this.knowledgeservice.getsearchresults(query).subscribe(res => {

     if (res.results) {

       this.results = res.results;

     }

   });

 }






knowledeapi.service.ts

getsearchresults(searchquery) {




let params = new URLSearchParams();

params.set('QueryString', searchquery);




let headers = new Headers({ 'Accept': 'application/json' });

let options = new RequestOptions({ headers: headers, search: params });

return this.http

   .get(this.searchURL, options).map(res =>




     res.json()




   ).catch(this.handleError);




}

For passing params in an HTTP object, we should create URLSearchParams() object, set the parameters in it, and send them as RequestOptions in http.get method. If you observe the line let headers = new Headers({ ‘Accept’: ‘application/json’ }); . we informed the API to send us data in JSON format.

Thereby finally the infobox component retrieves the results and displays them on susper.

Whole code for this implementation could be found in this pull:

https://github.com/fossasia/susper.com/pull/288

Continue ReadingHow we implemented an InfoBox similar to Google in Susper

Calling an API in Angular: Using Ngrx/Redux Architecture and Yacy API for Susper

Initially, in Susper we retrieved data from Yacy using a service in Angular 2, but later we introduced redux architecture, which resolved many issues and also made the code structured. In the past when Web APIs were not standardised people used to make their own architecture to implement each functionality. Web APIs have simplified the process of sending a query to an external server and standardised the process of sharing one’s own work with others.

The rise of Internet and mobile content in the recent past has resulted in many developers decoupling the front end and back end of their projects by exposing APIs that they create so that Android and iOS devices can interact with them using Web APIs. If you are new to building web APIs, A good place would be to look is here https://zapier.com/learn/apis/chapter-1-introduction-to-apis/ .

To understand how the Susper front end implements API calls from Yacy, it’s essential to understand the ngrx redux architecture inspired by react redux which helps manage the state. In case you are new to redux, please go through this to learn more about it and look at this sample app before proceeding with the rest of this blog post

In Susper we have implemented a front end for peer-to-peer decentralised Search Engine Yacy using Yacy Search API.

The services here are very similar to the angular services that seasoned angular js developers are familiar with. This service implementation in the project is responsible for making the calls to the API whenever a query is made.

https://github.com/fossasia/susper.com/blob/master/src/app/search.service.ts

where we implemented a searchService –

 

getsearchresults(searchquery) {

let params = new URLSearchParams();

for (let key in searchquery) {

if (searchquery.hasOwnProperty(key)) {

params.set(key, searchquery[key]);

}

}

params.set('wt', 'yjson');

params.set('callback', 'JSONP_CALLBACK');

params.set('facet', 'true');

params.set('facet.mincount', '1');

params.append('facet.field', 'host_s');

params.append('facet.field', 'url_protocol_s');

params.append('facet.field', 'author_sxt');

params.append('facet.field', 'collection_sxt');

return this.jsonp

.get('http://yacy.searchlab.eu/solr/select', {search: params}).map(res =>

res.json()[0]

).catch(this.handleError);

}

Now that you have seen the above service it contains JSONP_CALLBACK as a parameter, which tells the server “Hey Yacy, I can understand JSON, so you could communicate or send me data in JSON”. Some servers need one to send a header Accept: application/json

*JSONP is JSON with padding, that is, you put a string at the beginning and a pair of parenthesis around it*

so what about Redux where have we used it then? Basically, every redux based project will have an action and a reducer for each state in the store. Especially for search implementation we have our reducers and actions at https://github.com/fossasia/susper.com/blob/master/src/app/reducers

And

https://github.com/fossasia/susper.com/blob/master/src/app/actions

Now going through the architecture when a user types something in the search bar a call to the action query is made this.store.dispatch(new query.QueryServerAction(event.target.value));

Which is of type QUERYSERVER

 

export class QueryServerAction implements Action {

type = ActionTypes.QUERYSERVER;

constructor(public payload: any) {}

}

export type Actions

= QueryAction|QueryServerAction ;

 

Now on the above action below effect gets called

@Injectable()

export class ApiSearchEffects {

@Effect()

search$: Observable<any>

= this.actions$

.ofType(query.ActionTypes.QUERYSERVER)

.debounceTime(300)

.map((action: query.QueryServerAction) => action.payload)

.switchMap(querypay => {

if (querypay === '') {

return empty();

}

const nextSearch$ = this.actions$.ofType(query.ActionTypes.QUERYSERVER).skip(1);

this.searchService.getsearchresults(querypay)

.takeUntil(nextSearch$)

.subscribe((response) => {

this.store.dispatch(new search.SearchAction(response));

return empty();

});

return empty();

});

If you check with the above lines it makes a call to the service we have built before. that is: searchService.getsearchresults()

On response, it dispatches the response as a payload to SearchAction

On receiving the payload with the searchAction, the searchReducer takes off the responsibility and stores the payload in a state in the store.

export function reducer(state: State = initialState, action: search.Actions): State {

switch (action.type) {

case search.ActionTypes.CHANGE: {

const search = action.payload;

return Object.assign({}, state, {

searchresults: search,

items: search.channels[0].items,

totalResults: Number(search.channels[0].totalResults) || 0,

navigation: search.channels[0].navigation,

});

}

default: {

return state;

}

}

}

Thereby results could be displayed in the results page by subscribing to the store as in this.items$ = store.select(fromRoot.getItems);

Why should we do this all why not a direct service call? There are two reasons to use ngrx store along with ngrx effects.

  1. Using a store the search results will be available to all components.
  2. When we have implemented an instant search, a query call to the server goes for each character input, thereby if the response from the server is not in order, it leads to different results. For instance, if one searches for ‘India’ they might get results shown for ‘Ind’. Which was faced by us while developing the server https://github.com/fossasia/susper.com/issues/256 where “When a user searches, there is a search performed while typing. The search results that are often shown do not always reflect the final search term, they show a result that appeared while the user was typing it in.” we solved this issue using takeUntil(nextSearch$) in the search-effect.ts
Continue ReadingCalling an API in Angular: Using Ngrx/Redux Architecture and Yacy API for Susper

How to make your Website as a default Search Engine

A huge number of users are forcefully made to use predefined search engines on their browsers. On the Firefox browser, you usually see a search box at the top right. In Chrome, you can simply put in anything into the URL bar and it will go to the standard search engine. The browser companies predefined these search boxes, e.g. Google on Chrome, Firefox depending on which language version you have Yahoo or another and on Internet Explorer/Edge it is Bing. This shuts out new and independent searches like our Susper search engine. However, we want to help users and provide them with a choice.

 

At Susper, we integrated a small toast modal which allows users to make Susper their default search engine. They can simply go to susper.com and they will see a small option to change their search engine.

 Search box on Firefox

Implementation

We have Implemented this feature in a simple three-step procedure.

1.Generate Plugin for your search engine

create a plugin for your search engine at:-

http://mycroftproject.com/submitos.html

checked in with “show full instructions” checkbox at the top of the form to understand what those fields are.

Finally, a plugin got generated in XML format that we have to distribute to our user base.

2.Distributing the plugin to our user base

We at susper.com have implemented a toast onto the right bottom of the website’s homepage.

Where, when the user clicks on install Susper, the button triggers and displays an alert box.

Check the box “make this the current search engine”, and make the Susper search engine as your default search engine.

 

Implementation Code:-

 

 

We first check whether the search engine is installed already using window.external.IsSearchProviderInstalled , if not we show the toast for the user. When the user clicks Install button, this will call the window.external.AddSearchProvider API and installs susper using that.

 

<div id="set-susper-default">

<h3>Set Susper as your default search engine on Mozilla!</h3>

<ol>

<li><button id="install-susper">Install susper</button></li>

<li>Mark the checkbox to set Susper as your default search engine</li>

<li>Start searching!</li>

</ol>

<button id="cancel-installation">Cancel</button>

</div>

<script>

$(document).ready(function () {

if (window.external && window.external.IsSearchProviderInstalled) {

var isInstalled = window.external.IsSearchProviderInstalled("http://susper.com");

if (!isInstalled) {

$("#set-susper-default").show();

}

}

$("#install-susper").on("click", function () {

window.external.AddSearchProvider("http://susper.com/susper.xml");

});

$("#cancel-installation").on("click", function () {

$("#set-susper-default").remove();

});

});

</script>

In this way, we are able to give users an option to choose Susper as a default search engine.

More details regarding the implementation of this feature in susper could be checked at this pull https://github.com/fossasia/susper.com/pull/62 .

Continue ReadingHow to make your Website as a default Search Engine

Adding a page to Susper

As a project grows, it’s complexity increases. It suddenly seems to have a higher number of components and linked files. Performing a basic task, such as adding a new page to the website, might take more intricate knowledge. This blog deals with adding a new page to any Angular 2 project, in this case to Susper project. You can also check any of the sample components in the code on the Github Repository for further reference.

STEP 1:

Use ng g component <component-name> command to generate a new component with your desired name. Make sure the component-name is relevant and describes the page you wish to add. Once the above command is run in the Angular CLI project, it will automatically generate the following files:

  • component-name.html
  • component-name.css
  • component-name.ts
  • component-name.spec.ts file

It also adds the new component name to src/app/app.module.ts in the declarations section, after importing it.

You can also do all of this manually without the ng g component command too.

STEP 2:

Write your HTML and CSS files. Ensure that your page looks how you intend it to. Using bootstrap for your CSS classes might help you. Ensure that you link your bootstrap modules in index.html and not in the individual component files.

STEP 3:

If your page uses any typescript functions, please link your functions to your HTML page, after defining them in typescript.

This is how your typescript file might look (This is how it looks in Susper):

You may want to import modules you will need first.

Notice this snippet of code:

export class DemoComponent implements OnInit {

constructor() { }

ngOnInit() {

 }

}

You can define all your variables and functions in the component class. ngOnInit() has already been listed as a demo.

Also, note that including anything in the constructor function will run as soon as the page is loaded or initialized.

STEP 4:

If you have a unit tests in place, then component-name.spec.ts is where they are listed. Make sure to update it.

This is the procedure in Susper. Initially, your component-name.spec.ts will look like this:

import { async, ComponentFixture, TestBed } from '@angular/core/testing'

import { DemoComponent } from './demo.component'



describe('DemoComponent', () => {

let component: DemoComponent;

let fixture: ComponentFixture<DemoComponent>;



beforeEach(async(() => {

   TestBed.configureTestingModule({

     declarations: [ DemoComponent ]

   })

   .compileComponents();

 }));

beforeEach(() => {

   fixture = TestBed.createComponent(DemoComponent);

   component = fixture.componentInstance;

   fixture.detectChanges();

 });

it('should create', () => {

   expect(component).toBeTruthy();

 });

});

 

Here is what you need to add:

  1. Add imports under Testbed.configureTestingModule.

TIP:  Make sure to import all the modules from their file locations first.

imports: [

 RouterTestingModule,

 BrowserModule,

 CommonModule,

 FormsModule,

 HttpModule,

 JsonpModule,

 StoreModule.provideStore(reducer),

 StoreDevtoolsModule.instrumentOnlyWithExtension()

],
  1. Add all the other components under the declarations heading.

   TIP: Make sure to import all the components first.

It should look something like this:

declarations: [

   AppComponent,

   NavbarComponent,

   IndexComponent,

   ResultsComponent,

   NotFoundComponent,

   AdvancedsearchComponent,

   SearchBarComponent,

   FooterNavbarComponent,

   AboutComponent,

   ContactComponent,

   ModalComponent,

   InfoboxComponent,

 DemoComponent,

 ],

})

 

  1. Now add service providers under the providers heading if any.
  2. Finally, add any additional test cases using the standard syntax proved by Jasmine with it and expect statements.

STEP 5:

Update all .spec.ts files with your new component name under the declarations heading as seen in point 2 of step 4. This will notify all other spec.ts files about your new component, allowing ng test to run smoothly.

Make sure to import it each time you use it, to avoid compilation errors.

STEP 6:

To be able to reach your page, you can either

  • Embed it in another page using the selector mentioned in its .ts file.
@Component({

selector: 'app-demo',

Simply include the following tag in whichever page you wish to use the demo component, in the .html file:

<app-demo></app-demo>

  • Give it a route through which the user can reach it. Do this, by adding it in the Routes in app.module.ts
const appRoutes: Routes = [

 {path: 'search', component: ResultsComponent},

{path: '', component: IndexComponent},

You are done! You have successfully added a page to Susper!

Continue ReadingAdding a page to Susper

Fixing the scroll position in the Susper Frontend

An interesting problem that I encountered in the Susper frontend repository is the problem of the scroll position in SPAs (Single Page Applications). Since most websites now use Single page applications, such a hack, might prove useful to a lot of the readers.
Single page applications (SPAs) provide a better user experience. But, they are significantly harder to design and build. One major problem they cause is that they do not remember the scroll position on a page, like traditional browsers do. In traditional browsers, if we open a new page, by clicking on a link, it opens the page at the top.
Then on clicking back, it goes to not just to the previous link, but also the last position scrolled to on it. The issue we faced in Susper, was that when we opened a link, Susper being a SPA did not realise it was on a new page, and hence did not scroll to the top again. This was observed on every page, of the appliance.
Clicking on Terms on the footer for instance,

would open the bottom of the Terms page, which was not what we wanted.

FIX: Since all the pages required the fix, I ran a script in the main app component. Whenever an event occurs, the router instance detects it. Once the event has been identified as the end of a navigation action, I scroll the window to (0,0).
Here is the code snippet:

import {Component, OnInit} from '@angular/core';

import { RouterModule, Router, NavigationEnd } from '@angular/router';

@Component({

selector: 'app-root',

templateUrl: './app.component.html',

styleUrls: ['./app.component.css']

})

export class AppComponent implements OnInit {

title = 'Susper';

constructor(private router: Router) { }

ngOnInit() {

   this.router.events.subscribe((evt) => {

     if (!(evt instanceof NavigationEnd)) {

       return;

     }

     window.scrollTo(0, 0);

   });

 }

}

“NavigationEnd” is triggered on the end of a Navigation action, in Angular2. So if the “NavigationEnd” hasn’t been triggered, our function need not do anything else and can simply return.  If a Navigation action has just finished the window is made to scroll up to (0,0) coordinates.
Now, this is how the Terms page opens:

 

Done! Now every time a link is clicked it scrolls to the top.

Continue ReadingFixing the scroll position in the Susper Frontend

Deploying Angular 2 application using GitHub Pages

In recent months I have started working with Angular 2 technology as my project is based on this tech stack. Angular 2 is one of the famous frameworks of JavaScript. The project name is ‘Susper’ which is currently being in development stages under FOSSASIA. In FOSSASIA, to be a good developer everyone follows good practices. One of the good practice is providing a live preview of the fix done in a pull request related to a particular issue. It was not simple to deploy test pages as it looks on GitHub pages. I read a lot of StackOverflow answers and surfed google a lot to find a solution. Then I came to the solution, which I’ll be sharing with you in this blog.

I’m assuming your Angular 2 app must be using webpack services and the latest version of Angular has been installed. Firstly, be sure Angular CLI must be updated. If not, then update the Angular CLI to a new version. You must update both the global package and local package of your project.

Global package:

npm uninstall g @angular/cli
npm cache clean
npm install g @angular/cli@latest

NOTE – Make sure to install local packages, you must be inside the project folder.

To make deployments easier, follow these steps after updating global and local packages –

Install angular-cli-ghpages :

npm i g angularclighpages

This command is similar to the old github pages:deploy command of @angular/cli and this script works great with Travis CI.
After installing you should see the changes in the package.json as well:

“devDependencies”: {
    “angular-cli-ghpages”: “^0.5.0”
}

After updating the global and local package you will notice a new folder named ‘node_modules’ has been created. Now the magic part comes to play here!

Add deploy script:

In package.json file add the following deploy script –

“scripts”: {
    “deploy”: “ng build –prod –aot –base-href=/project_repo_name/ && cp ./dist/index.html ./dist/404.html && ./node_modules/.bin/angular-cli-ghpages –no-silent”
}

We have setup the required dependencies to deploy test page. Now, here it comes to generate a live preview:

Steps :

git checkout working_branch
ng build
npm run deploy

We have successfully deployed the repository to GitHub pages. To refer live preview go here –

https://yourusername.github.io/project_name

How did it work out?

Well, this is the easiest way to deploy any angular 2 apps on GitHub pages. The only disadvantage of deploying to GitHub pages is that we have to always perform a manual build before providing a live preview whenever some changes have been done in that particular branch.

Continue ReadingDeploying Angular 2 application using GitHub Pages