Extending the News Feature to Show results from multiple organisations

News Tab in Susper was earlier implemented to show results only from a single organisation using  site: modifier for query facet. In this blog I will discuss about how I have modified the current News Tab to show results from various News organisation like BBC, Al Jazeera, The Guardian etc.

Implementation:

Step 1:

Creating a JSON file to store organisations detail:

We need to decide from which organisations  we will fetch results using YaCy Server and then display it in Susper. We have provided a JSON file where user can add or delete News sources easily. The results will only limited to the organisations which are present in the JSON file.

Step 2:

Creating a service to fetch details from JSON file:

Now after creating the the JSON file we need a service which will fetch results from the JSON file according to our need. This service will be a simple  Angular Service having a class GetJsonService and it will access the newsFile.json and map the results in JSON format. Here is the service which does this task for us.

Step 3:

Creating a service to fetch news accordingly:

Now after fetching the JSON result we need a service to fetch the News Results from YaCy Server. I have created a separate service to do this task where I have fetched results using site: modifier from each organisation and returned the results.The code for the news.service.ts is below.

export class NewsService {
 constructor(private jsonp: Jsonp) { }
 getSearchResults(searchquery, org) {
   let searchURL = 'https://yacy.searchlab.eu/solr/select?query=';
   searchURL += searchquery.query + ' site:' + org;
   let params = new URLSearchParams();
   for (let key in searchquery) {
     if (searchquery.hasOwnProperty(key)) {
       params.set(key, searchquery[key]); } }
  //Set other parameters
   return this.jsonp
     .get(searchURL, {search: params}).map(res =>
       res.json()[0]
     ).catch(this.handleError); }

 

Step 4:

Updating the results section:

Now we have a service that gives results from a single organisation and a JSON list of organisations. Now in results.component.ts we can simply subscribe to getJsonService and in a loop we will call getNewsService by changing the organisation in every iteration. We will then check that whether we are getting the valid results or not (undefined). The results which are not valid can cause errors when we will try to read any field of an undefined variable. Then, We will simply append the 2 result items from each organisation in an empty array and later use this array to show results.

this.getJsonService.getJSON().subscribe(res => { this.newsResponse = [];
for (let i = 0; i < res.newsOrgs.length; i++) { this.getNewsService.getSearchResults(querydata,res.newsOrgs[i].provider).subscribe( response => {
    if (response.channels[0].items[0] !== undefined) {
    this.newsResponse.push(response.channels[0].items[0]); }
    if (response.channels[0].items[1] !== undefined) {
    this.newsResponse.push(response.channels[0].items[1]); } } ); }
    });

 

The newsClick() function is activated on clicking News Tab and it updates the query and its details in store.

Step 5

Displaying the results:

Now we will modify the results.component.html to show results from new newsResponse array which have 2 results each from 5 organisations.

For this we will use each item of newsResponse using *ngFor and display its title,link and description in html template. We will also use [style.color] property of our element and set the color according to theme.

<div *ngFor="let item of newsResponse" class="result"> <div class="title">
<a class="title-pointer" href="{{item.link}}" [style.color]="themeService.titleColor">{{item.title}}</a>
</div> <div class="link">
<p  [style.color]="themeService.linkColor">{{item.link}}</p>
 </div> </div>

 

Here is the view of Susper’s News Tab where we are getting results from 5 different organisations.

Resources

  1. YaCy Modifiers: http://www.yacy-websuche.de/wiki/index.php/En:SearchParameters
  1. Angular Services: https://angular.io/tutorial/toh-pt4
  2. Reading JSON data in Angular: https://stackoverflow.com/questions/43275995/angular4-how-do-access-local-json

 

Continue ReadingExtending the News Feature to Show results from multiple organisations

Structure of Open Event Frontend

In Open Event Frontend, new contributors always fall into a dilemma of identifying the proper files where they have to make changes if they want to contribute. The project structure is quite complex and which is obvious because it is a large project. So, in this blog, we will walk through the structure of Open Event Frontend.

Following are the different folders of the project explained:

Root:
The root of the project contains folders like app, config, kubernetes, tests, scripts. Our main project is in the app folder where all the files are present. The config folder in the root has files related to the deployment of the app in development, production, etc. It also has the environment setup such as host, api keys, etc. Other files such as package.json, bower.json, etc are basically to store the current versions of the packages and to ease the installation of the project.

App:
The app folder has all the files and is mainly classified into the following folder:
adapters
components
controllers
helpers
Initializers
mixins
models
routes
serializers
services
styles
templates
transforms
utils

The folders with their significance are listed below:

Adapters: This folder contains the files for building URLs for our endpoints. Sometimes it happens to have a somewhat customised URL for an endpoint which we pass through adapter to modify it.
Components: This folder contains different components which we reuse in our app. For example, the image uploader component can be used at multiple places in our app, so we keep such elements in our components. This folder basically contains the js files of all the components(since when we generate a component, a js file and a hbs template is generated).
Controllers: This folder contains the controller associated with each route. Since the main principle of ember js is DDAU i.e data down actions up, all the actions are written in the files of this folder.
Helpers: Many a time it happens that, we want to format date, time, encode URL etc. There are some predefined helpers but sometimes custom helpers are also needed. All of them have been written in helpers folder.
Initializers: This folder has a file for now called ‘blanket.js’ which basically injects the services into our routes, components. So if you want to write any service and want to inject it into routes/components, it should go in here.
Mixins: In EmberJS the Mixin class can create objects whose properties and functions can be shared amongst other classes and instances. This allows for an easy way to share behavior between objects as well as design objects that may need multiple inheritance. All of them used for the application are in the mixins folder.
Models: This folder contains the schema’s for our data. Since we are using ember data, we need to have proper skeleton of the data. All of this goes it this folder. Observing this folder will show you some models like user, event, etc.
Routes: This folder contains the js files of the routes created. Routes handle which template to render and what to return from the model, etc.
Serializers: We use serializers to modify the data that ember sends automatically in a request. Consider we want to get a user with the help of user model, and don’t want to get the password attribute present in it. We can thus omit that by defining it in a serializer.
Services: Services are the ember objects which are available throughout the running time of the application. These are used to perform tasks like getting current user model, making third party API calls etc. All such services go in this folder.
Styles: As the name infers, all the style sheets go in here.
Templates: A template is generated with generation of each route and component. All of them go here. Thus, the markup will be written over here.
Transforms: Ember Data has a feature called transforms that allow you to transform values before they are set on a model or sent back to the server. In our case, we have a transform called moment.
Utils: This folder contains some functions exported as modules which are reusable. There is some JSON data as well.

References: Ember JS official guide: https://guides.emberjs.com/v2.17.0/
Blog posts: https://spin.atomicobject.com/2015/09/17/ember-js-clean/
http://www.programwitherik.com/ember-pods/

Continue ReadingStructure of Open Event Frontend

Stubbed Routing Inbuilt Service used in Open Event Frontend

In Open Event Frontend, we have used services like ‘auth-manager’, ‘l10n’, ‘loader’, ‘sanitizer’, etc to ease our work with the help of predefined-functions in those services. However, while dealing with an issue in the project, there was a need to use ‘Routing’ as a service.

In the issue, we wanted to generate an access link dynamically from the access code entered by the user. The format of the access link was as follows:

“base_url + event_id + access_code”

So, for the above URL, we needed to have ‘event_id’ and ‘access_code’.

The ‘access_code’ can be readily accessed from the user’s input itself, whereas to get the event_id, we used the ‘Routing’ service in Ember.

Generally to use a service in Ember, it has to be written first,then registered, injected and then used.

‘Routing’ service in Ember is an inbuilt service unlike the ones listed at the beginning.

There is no need to write it. It can be simply registered, injected and used.

this.register('service:routing', routingStub);
this.inject.service('routing', { as: 'routing' });

where ‘register’ and ‘inject’ are the methods on Ember objects.

The integration tests in Open Event Frontend are written such that the services can be used without injecting, but the tests will fail. To pass those tests, we had to register and inject the service in the required component.

The Routing service could thus be registered and injected into the specific component( injection in the component’s integration test ) only but for future needs, this service might be needed in any other component too. For this purpose, this service was registered and injected in ‘component-helper.js’.

const routingStub = Service.extend({
  router: {
    router: {
      state: {
        params: {
          'events.view': {
            event_id: 1
          }
        }
      }
    },
    generate() {
      return 'http://dummy-url.com';
    }
  }
});


export default function(path, name, testCase = null) {
  moduleForComponent(path, name, {
    integration: true,

    beforeEach() {
      this.register('service:routing', routingStub);
      this.inject.service('routing', { as: 'routing' });
      this.register('service:l10n', L10n);
      this.inject.service('l10n', { as: 'l10n' });
      this.application = startApp();
      l10nTestHelper(this);
      run(() => fragmentTransformInitializer.initialize(getOwner(this)));
    }
  }
}

Stubbing a Service: This is a process of faking an app of importing a service when no path is available to import. Stubbing of a service is mainly done when one needs to deal with the testing of the app. In our case, the same is done. We have stubbed the ‘Routing’ service in order to deal with the testing part. It can be seen from the above code that we have generated a ‘routingStub’ which fakes the app while registering the service in the ‘beforeModel’. The next line of code shows the ‘injection’ of service into the app.

Now we are just left with one task i.e to pass ‘routing’ from our integration tests to the component.

test('it renders', function(assert) {
  this.render(hbs`{{forms/events/view/create-access-code routing=routing}}`);
  assert.ok(this.$().html().trim().includes('Save'));
});

Above code shows the same.

Thus we can stub the services in Ember when any component depends on them.

Resources:

Official Ember guide: https://guides.emberjs.com/v2.1.0/testing/testing-components

Blog by Todd Jordan: http://presentationtier.com/stubbing-services-in-emberjs-integration-tests/

Source codehttps://github.com/sumedh123/open-event-frontend/blob/0b193ca679ce3b51f65e19ee0d03ac6a679258de/tests/helpers/component-helper.js

Continue ReadingStubbed Routing Inbuilt Service used in Open Event Frontend

Updating Page Titles Dynamically in Loklak Search

Page titles are native in the web platform, and are prime ways to identify any page. The page titles have been in the web platform since ages. They tell the browsers, the web scrapers and search engines about the page content in 1-2 words. Since the titles are used for wide variety of things from presentation of the page, history references and most importantly by the search engines to categorise the pages, it becomes very important for any web application to update the title of the page appropriately. In earlier implementation of loklak search the page title was a constant and was not updated regularly and this was not a good from presentation and SEO perspective.

Problem of page titles with SPA

Since loklak search is a single page application, there are few differences in the page title implementation in comparison to a server served multi page application. In a server served multi page application, the whole application is divided into pages and the server knows what page it is serving thus can easily set the title of the page while rendering the template. A simple example will be a base django template which holds the place to have a title block for the application.

<!-- base.html -->

<title>{% block title %} Lokalk Search {% endblock %}</title>

<!-- Other application blocks -->

Now for any other page extending this base.html it is very simple to update the title block by simply replacing it with it’s own title.

<!-- home.html -->

{% extendsbase.html%}

{% block title %} Home Page - Loklak Search {% endblock %}

<!-- Other page blocks -->

When the above template is rendered by the templating engine it replaces the title block of the base.html with the updated title block specific to the page, thus for each page at the rendering time server is able to update the page title, appropriately.

But in a SPA, the server just acts as REST endpoints, and all the templating is done at the client side. Thus in an SPA the page title never changes automatically, from the server, as only the client is in control of what page (route) it is showing. Thus it becomes the duty of the client side to update the title of the page, appropriately, and thus this issue of static non informative page titles is often overlooked.

Updating page titles in Loklak Search

Before being able to start solving the issue of updating the page titles it is certainly very important to understand what all are the points of change in the application where we need to update the page title.

  • Whenever the route in the application changes.
  • Whenever new query is fetched from the server.

These two are the most important places where we definitely want to update the titles. The way we achieved is using the Angular Title Service. The title service is a platform-browser service by angular which abstracts the workflow to achieve the title updation. There are are two main methods of this service get and set title, which can be used to achieve our desired behaviour. What title service do is abstract the extraction of Title Node and get and set the title values.

For updation of title for each page which is loaded we just attach an onInit lifecycle hook to the parent component of that page and, onInit we use the title service to update the title accordingly.

@Component({
selector: 'app-home',
templateUrl: './home.component.html',
})
export class HomeComponent implements OnInit, OnDestroy {
constructor(
private titleService: Title
) { }

ngOnInit() {
this.titleService.setTitle(Loklak Search');

// Other initialization attributes and methods
}
}

Similarly other pages according to their context update the page titles accordingly using the simple title service. This solves the basic case of updation of the titles of the page when the actual route path changes, and thus component’s onInit lifecycle hook is the best place to change the title of the page.

@Component({
selector: 'app-home',
templateUrl: './home.component.html',
})
export class HomeComponent implements OnInit, OnDestroy {
constructor(
private titleService: Title
) { }

ngOnInit() {
this.titleService.setTitle(Loklak Search');

// Other initialization attributes and methods
}
}

But when the actual route path doesn’t change and we want to update the title according to the query searched then it is not possible to do it using lifecycle hooks of the component. But fortunately, we are using the ngrx effects in our application and thus this task also again becomes much simpler to achieve in the application. In this situation again what we do is hook up a title change effect to SearchCompleteSuccessAction, and there we change the title accordingly.

@Effect({ dispatch: false })
resetTitleAfterSearchSuccess$: Observable<void>
= this.actions$
.ofType(apiAction.ActionTypes.SEARCH_COMPLETE_SUCCESS,
apiAction.ActionTypes.SEARCH_COMPLETE_FAIL)
.withLatestFrom(this.store$)
.map(([action, state]) => {
const displayString = state.query.displayString;
let title = `${displayString} - Loklak Search`;
if (action.type === apiAction.ActionTypes.SEARCH_COMPLETE_FAIL) {
title += ' - No Results';
}
this.titleService.setTitle(title);
});

Now if we look closely this effect is somewhat different from all the other effects. Firstly, the effect observable is of type void instead of type Action which is the case with other effects, and also there is is a { dispatch: false } argument passed to the constructor. Both these things are important of our resetTitle effect. As our reset title effect has no action to dispatch on it it’s execution the the observable is of type void instead of type Action, and we never want to dispatch an effect whose type is not an Action thus we set dispatch to false. Rest of the code for the effect is fairly simple, we filter all the actions and take SearchSuccess and SearchFail actions, then we get the latest value of the query display string from the store, and we use our title service to reset the title accordingly.

Conclusion

The titles are the important part of the web platform and are used by browsers and search engines to present and rank the relevance of the page. While developing a SPA it is even more important to maintain an updated title tag, as it is the only thing which actually changes about the page according to the context of the page. In loklak search, the title service is now used to update the titles of the page according to the search results.

Resources and Links

Continue ReadingUpdating Page Titles Dynamically in Loklak Search