Deploying loklak search on Heroku & Using config to store and load external URLs

It is really important to have a separate configuration for all the hardcoded URLs having multiple instances across the project. It would help in testing and comparing projects like loklak with a separate configuration for hardcoded URLs. We would also be discussing deployment of Angular based loklak on heroku through this blog post.

Creating shared URL config

The idea here is to export a const Object containing all the collective hardcoded URLs used inside the project. We would try to store similar/collective URLs inside a same key e.g. all the github URLs must go inside key: ‘github’.

export const defaultUrlConfig = {
	fossasia: {
		root: 'https://fossasia.org',
		blog: 'https://blog.fossasia.org'
	},
	loklak: {
		apiServer: 'https://api.loklak.org',
		apiBase: 'loklak.org',
		blog: 'http://blog.loklak.net',
		dev: 'https://dev.loklak.org',
		apps: 'https://apps.loklak.org'
	},
	github: {
		loklak: 'https://github.com/loklak',
		fossasia: 'https://github.com/fossasia'
	},
	phimpme: {
		root: 'https://phimp.me'
	},
	susper: {
		root: 'https://susper.com'
	},
	susiChat: {
		root: 'https://chat.susi.ai'
	},
	pslab: {
		root: 'https://pslab.fossasia.org'
	},
	eventyay: {
		root: 'https://eventyay.com'
	}
};

 

Storing URLs with a key instead of storing it inside a simple Array, makes it easier to add a new URL at required place and easy to modify the existing ones. Now, this configuration can easily be called inside any Component file.

Using config inside the component

Now, the work is really simple. We just need to import the configuration file inside the required Component and store/use directly the respective URLs.

First step would be to import the configuration file as follows:

import { defaultUrlConfig } from ‘../shared/url-config’;

 

Note: Respective path for url-config file for different components might differ.

Now, we would need to store the defaultUrlConfig inside a variable.

public configUrl = defaultUrlConfig;

 

At this point, we have all the configuration URLs which could be extracted from configUrl.

Displaying URLs in HTML

We would use Angular’s interpolation binding syntax to display the URLs from configUrl in HTML. Let’s say we want to display FOSSASIA’s github url in HTML, then we would simply need to do:

{{ configUrl.github.fossasia }}

 

This could be used as an example to to replace all the hardcoded URLs inside the project.

Deploying loklak search on Heroku

Note: I am skipping the initial steps of creating a project on heroku. It is very easy to setup a project on heroku, for initial steps please follow up here.

First step in this direction would be to add a server.js file in root directory and add the following express server code in it:

const express = require(‘express’);
const path = require(‘path’);
const app = express();
app.use(express.static(__dirname + ‘/dist/<name-of-app>’));
app.get(‘/*’, function(req,res) {
res.sendFile(path.join(__dirname+‘/dist/<name-of-app>/index.html’));
});
app.listen(process.env.PORT || 8080);

 

Second step would be to add the following commands inside package.json at respective attributes.

“postinstall”: “ng build –aot -prod”
“start”: “node server.js”

 

Testing

Click on the respective URL link inside the project UI to test the configuration for hardcoded URLs. To check the deployment on heroku, please open the following URLs:

Development branch: loklak-search-dev.herokuapp.com

Master branch: loklak-search.herokuapp.com

Resources

Continue ReadingDeploying loklak search on Heroku & Using config to store and load external URLs

Query check using RegExp

Specific results can be obtained in loklak using different combinations of type associated with query (e.g. from:user, @user, #query). For identifying the requirement of user through query and maintaining the flow of API requests to the loklak server, RegExp is being used. Through this blog post, I’ll be discussing about its use in the project.

Using RegExp in Services

Patterns for all possible query types has already been defined in reg-exp typescript file inside utils/ folder. We would need to import these expressions to check type of query anywhere inside the codebase.

The query (q) arguments of search endpoints of api.loklak depends on type associated with query i.e. argument for from:user would be different from that of @user. To keep check on the same, I have used RegExp.

if ( hashtagRegExp.exec(query) !== null ) {
       // Check for hashtag query
       jsonpUrl += ‘&q=%23 + hashtagRegExp.
       exec(query)[1] +  + hashtagRegExp.exec(query)[0];
   } else if ( fromRegExp.exec(query) !== null ) {
       // Check for from user query
       jsonpUrl += ‘&q=from%3A’ + fromRegExp.exec(query)[1];
   } else if ( mentionRegExp.exec(query) !== null ) {
       // Check for mention query
       jsonpUrl += ‘&q=%40 + mentionRegExp.exec(query)[1];
   } else if ( nearRegExp.exec(query) !== null ) {
       // Check for near query
       jsonpUrl += ‘&q=near%3A’ + nearRegExp.exec(query)[1];
   } else {
       // for other queries
       jsonpUrl += ‘&q=’ + query;
   }

 

A similar architecture has been used in UserService and SuggestService.

Using RegExp in info-box

A new feature has been added in loklak to provide RSS and JSON feed through api.loklak.org. It requires a query check which is then passed through anchor tag link to provide RSS and JSON feed based on the query and type associated with it. It is being included under info-box and the query check is done in ngOnChanges() which means it will keep updating itself with updating query.

   this.store.select(fromRoot.getQuery).subscribe(query => this.stringQuery = query.displayString);
   this.parseApiResponseData();
   if ( hashtagRegExp.exec(this.stringQuery) !== null ) {
       // Check for hashtag this.stringQuery
       this.queryString = %23 + hashtagRegExp.
       exec(this.stringQuery)[1] +  + hashtagRegExp.exec(this.stringQuery)[0];
   } else if ( fromRegExp.exec(this.stringQuery) !== null ) {
       // Check for from user this.stringQuery
       this.queryString = from%3A + fromRegExp.exec(this.stringQuery)[1];
   } else if ( mentionRegExp.exec(this.stringQuery) !== null ) {
       // Check for mention this.stringQuery
       this.queryString = %40 + mentionRegExp.exec(this.stringQuery)[1];
   } else {
       // for other queries
       this.queryString = this.stringQuery;
   }
}

 

A similar architecture is followed throughout the project in different features implemented so far. It would not be easy to provide each of those implementation in one blog. For actual implementations, please go through the codebase.

Resources

Continue ReadingQuery check using RegExp

Implementing news feature in loklak

The idea is to bring out an useful feature out of the enriched high quality of data provided by api.loklak.org. Through this blog post, I’ll be discussing about the implementation of a news feature similar to Google to provide latest news relevant to the query in loklak.

Creating news Component

First step in implementing news feature would be to create a news component which would be rendered on clicking of news tab.

ng g component feed/news

Creating an action & reducer for news status

This step involves creating an action & reducer function for tracking the status of news component on click of news tab. When the user will click on news tab, corresponding action would be dispatched to display the news component in feed.

import { Action } from ‘@ngrx/store’;
export const ActionTypes = {
    NEWS_STATUS: ‘[NEWS] NEWS STATUS’
};
export class NewsStatusAction implements Action {
    type = ActionTypes.NEWS_STATUS;
    constructor (public payload: boolean) { }
}
export type Actions
    = NewsStatusAction;

 

Corresponding reducer function for news status to store current news status would be:

export function reducer(state: State = initialState, 
   action: newsStatusAction.NewsStatusAction): State {
switch (action.type) {
    case newsStatusAction.ActionTypes.NEWS_STATUS: {
    const newsStatusPayload: boolean = action.payload;
    return Object.assign({}, state, {
                newsStatus: newsStatusPayload
            });
        }
        default: {
            return state;
        }
    }
}

 

I would not be getting into each line of code here, only the important portions would be discussed here.

Creating news-org to store the names of news organizations

A new file news-org would be created inside app/shared/ folder containing list of news organizations in an array newsOrgs from which we would extract and filter results.

export const newsOrgs = [
    ‘CNN’,
    ‘nytimes’
];

 

Note: This is a dynamic implementation of news service and hence, the list of news organizations can be easily modified/updated here.

We would also need to export the newsOrgs in the index file inside shared/ folder as:

export { newsOrgs } from ‘./news-org’;

Creating ngrx action, reducer & effect for news success

Now comes the main part. We would need an action to dispatch on successful response of news search and corresponding reducer to keep appending the async news results.

News action would be created following the simple ngrx composition as:

import { Action } from ‘@ngrx/store’;
import { ApiResponse } from ‘../models/api-response’;
export const ActionTypes = {
    NEWS_SEARCH_SUCCESS: ‘[NEWS] SEARCH SUCCESS’
};
export class NewsSearchSuccessAction implements Action {
    type = ActionTypes.NEWS_SEARCH_SUCCESS;
    constructor (public payload: ApiResponse) { }
}
export type Actions
    = NewsSearchSuccessAction;

 

One of the important parts of news implementation is the reducer function corresponding to the news success action. We would actually append the async results provided by news effect into the store.

import * as newsSuccessAction from ‘../actions/newsSuccess’;
import { ApiResponseResult } from ‘../models’;

export interface State {
newsResponse: ApiResponseResult[];
}
export const initialState: State = {
newsResponse: [],
};
export function reducer(state: State = initialState,
   action: newsSuccessAction.NewsSearchSuccessAction): State {
    switch (action.type) {
        case newsSuccessAction.ActionTypes.NEWS_SEARCH_SUCCESS: {
            // Appending the news result
            return Object.assign({}, state, {
                newsResponse: […action.payload.statuses]
            });
        }
        default: {
            return state;
        }
    }
}
export const getNewsResponse = (state: State) => state.newsResponse;

 

Now, we would create an effect which would actually use the search service to fetch results for the list of organizations one by one will keep dispatching a corresponding action to append the response into the store to be displayed in news feed. Instead of showing whole code for the effect, only the main logic is being displayed.

import { newsOrgs } from ‘../shared/news-org’;
...
@Injectable()
export class DisplayNewsEffects {
    @Effect({ dispatch: false })
        searchNews$: Observable<void> = this.actions$
        .pipe(
        ofType(
        newsAction.ActionTypes.NEWS_STATUS
        ),
        map((action) => {
            if (action[‘payload’]) {
                const orgs = newsOrgs;
                orgs.forEach((org) => {
                const searchServiceConfig:
                    SearchServiceConfig = 
                    new SearchServiceConfig();
                this.apiSearchService
                .fetchQuery(‘from:’
                  + org, searchServiceConfig)
                    .subscribe(response =>
                this.store$.dispatch(
                     new newsStatusAction
                     .NewsSearchSuccessAction(response)));
                });
            }
        })
    );

Filtering the news response inside news component

The basic idea is to filter the results from news response which contain the query and display them using news template on click of news tab on results page of loklak.

import { Query } from ‘../../models/query’;
import { Observable } from ‘rxjs’;
...
public query: string;
public query$: Observable<Query>;
...
ngOnInit() {
   const texts = [];
   this.store.select(fromRoot.getQuery).subscribe(res => 
       this.query = res.displayString);
   this.query$ = this.store.select(fromRoot.getQuery);
   this.store.select(fromRoot.getNewsResponse).subscribe(
       v => {
       for ( let i = 0; i < v.length; i++ ) {
           this.newsResponse.push(v[i]);
           if (v[i][‘text’].includes(this.query
               .replace(/\s/g, ”).toLowerCase())) {
               if (!texts.includes(v[i][‘text’]
                   .replace(/\s/g, ”).toLowerCase())) {
                   texts.push(v[i][‘text’]
                       .replace(/\s/g, ”).toLowerCase());
                   this.newsResponse.push(v[i]);
               }
           }
       }
   });
}

 

In above configuration, firstly we needed the current query which we extracted from ngrx store and then checking whether the text of news results contain the query string or not. If the query is present inside the text, we would add it into an array which would be used to display news feed using feed-card.

Displaying the news response

Displaying the news response is very easy, we just need to pass the news response items with index inside feed-card component which would display the news feed similar to the normal feed under the news section.

<div class=“wrapper feed-results”>
   <div *ngFor=“let item of newsResponse; let i = index “>
     <feed-card [feedItem]=“item” [feedIndex]=“i”></feed-card>
   </div>
</div>

Creating News tab to display news feed

To provide a tab to user to access news feed, we would need to create a News tab along with the other searching tools inside feed-advanced-search component as:

<button [class.selected]=“selectedTab === ‘news'” class=“tab” value=“news”
   (click)=“getFilterResults($event.currentTarget.value)”>
   News
</button>

 

Note: Complete code for news implementation can be found here and the news filter can be found here.

Testing news feature

Search a query on loklak, and then click on news tab to get the latest relevant news matching the query.

Resources

Continue ReadingImplementing news feature in loklak

Create new route ‘/settings’ to display project’s configuration

Having a separate route to display all the hardcoded project’s configuration makes it very easy to compare web apps with each other. A similar architecture has been followed in loklak to create a ‘/settings’ route to display configs used inside it. Respective implementation would be discussed in this blog.

Creating Settings Component

First step would be to create a Settings component to render under /settings route.

ng g component settings

 

This command will create new settings component. We have to create settings-routing and settings module inside settings component separately. This route module will be called inside the root app module. The settings-routing module would import Router module as follows:

import { NgModule } from ‘@angular/core’;
import { Routes, RouterModule } from ‘@angular/router’;
import { SettingsComponent } from ‘./settings.component’;
const routes: Routes = [
    {
        path: '',
        component: SettingsComponent
    }
];
@NgModule({
    imports: [RouterModule.forChild(routes)],
    exports: [RouterModule],
    providers: []
})
export class LoklakSettingsRoutingModule { }

 

The settings module would follow the basic module configuration of Angular, and import the settings-routing module as follows:

import { NgModule } from ‘@angular/core’;
import { CommonModule } from ‘@angular/common’;
import { LoklakSettingsRoutingModule }
    from ‘./settings-routing.module’;
import { SettingsComponent } from ‘./settings.component’;
@NgModule({
    imports: [
        CommonModule,
        LoklakSettingsRoutingModule
    ],
    declarations: [
        SettingsComponent
    ]
})
export class SettingsModule { }

 

At this point of time, we are having a basic settings component to render under /settings route.

Loading /settings as child route under root app module

Now, we need to add path of ‘/settings’ route inside root app module as follows:

path: ‘settings’,
loadChildren: ‘./settings/settings.module#SettingsModule’

 

Which means when user is on path /settings, load settings component.

Fetching project configs inside settings component

At this point, we have successfully established /settings route and settings component. Now we need to fetch the project configs inside the settings component to display. We would actually import the config and store it in an array to loop through it in template file using *ngFor to display the items under config array.

Note: We would import newsOrgs containing twitter username of news organizations, and the defaultUrlConfig containing the list of all hardcoded URLs used inside the project. It can be taken as an example to add and display more configurations.

import { Component, OnInit } from ‘@angular/core’;
import { defaultUrlConfig } from ‘../shared/url-config’;
import { newsOrgs } from ‘../shared/news-org’;
@Component({
    selector: ‘app-settings’,
    templateUrl: ‘./settings.component.html’,
    styleUrls: [‘./settings.component.scss’]
})
export class SettingsComponent implements OnInit {
    public urlObject = [];
    public newsConfigOrgs = newsOrgs;
    ngOnInit() {
    let i = 0;
    for (const key in defaultUrlConfig) {
        if (key) {
            const data = [];
            const value = defaultUrlConfig[key];
            const data2 = [];
            for (const key2 in value) {
                if (key2) {
                    const value2 = value[key2];
                    data2.push(key2, value2);
                }
            }
            data.push(key, data2);
            this.urlObject.push(data);
            i++;
            }
        }
    }
}

 

urlObject is an array used to store the list of hardcoded URLs, and newsConfigOrgs is an array containing all the list of news organizations.

Displaying array items in template file

Up to this point, we have successfully stored the data inside the component file of settings component and now we need to display this data in template file. For a sake of an example, we would display news orgs as follows:

<!– News Config –>
<h1><u>News Configuration</u></h1>
<div class=“url-container” *ngFor=“let key of newsConfigOrgs”>
   <h4 class=“news”>{{key}}</h4>
</div>

The above code could be used as an example to display more config stored inside component file. Only the basic implementation has been followed and mentioned inside this blog, complete code can be found here: code.

Creating settings tab in Advanced-Feed

Now we need to provide a link to user to check /settings route. We would actually add another tab as settings inside Advanced-Feed as follows:

<a routerLink=“/settings” target=“_blank” class=“tab”>
   Settings
</a>

Testing /settings route

Search a query on loklak.org, and then click on settings tab which would be similar to:

On clicking settings tab, user should be directed to the /settings route containing non-editable (read-only) configuration.

Resources

Continue ReadingCreate new route ‘/settings’ to display project’s configuration

Add RSS feed and JSON output based on type specified with query param

The idea behind writing this blog post is to discuss the method on how RSS feed and JSON output sources have been included in loklak to provide respective data sources based on the type specified with query as a parameter.

Accessing Current Query in Info-box

Accessing link to RSS feed and JSON output of loklak requires the Query to be passed as a value with parameter ‘q’ (e.g. api/search.json?q=FOSSASIA or api/search.rss?q=FOSSASIA). In order to represent links as buttons in Info-box at sidebar of loklak.org, current Query needs to be accessed/stored inside Info-box from ngrx store.

public stringQuery;
...
this.store.select(fromRoot.getQuery).subscribe(
    query => this.stringQuery = query.displayString);

 

Firstly stringQuery variable is created to store the current Query from store. As the Query can be changed in store (User might search for several Queries), storing of current Query needs to be done inside ngOnChanges().

Checking type associated with Query

There are various types associated with Query to get different type of results like ‘from:FOSSASIA will give results specifically from ‘FOSSASIA’ which will have different query parameter from other types like ‘@FOSSASIA’ or ‘#FOSSASIA’. To assign appropriate Query parameter to each of these types, we need to check the Query pattern to apply Query param based on the type.

import { hashtagRegExp, fromRegExp, mentionRegExp }
      from ‘../../utils/reg-exp’;
...

if ( hashtagRegExp.exec(this.stringQuery) !== null ) {
	// Check for hashtag this.stringQuery
	this.queryString = ‘%23 + hashtagRegExp.exec(
	this.stringQuery)[1] + '' + hashtagRegExp.exec(
	this.stringQuery)[0];
} else if ( fromRegExp.exec(this.stringQuery) !== null ) {
	// Check for from user this.stringQuery
	this.queryString = ‘from%3A’ + fromRegExp.exec(
	this.stringQuery)[1];
} else if ( mentionRegExp.exec(this.stringQuery) !== null ) {
	// Check for mention this.stringQuery
	this.queryString = ‘%40 + mentionRegExp.exec(
	this.stringQuery)[1];
} else {
	// for other queries
	this.queryString = this.stringQuery;
}

 

Note: hashtagRegExp, fromRegExp, mentionRegExp are the utility functions created to match the pattern of given string (Query) in order to classify the preceding type associated with Query. These are provided here as a reference, which can be used to add more of the types.

Passing current queryString in RSS and JSON link

General link for both RSS and JSON data remains same for each Query passed, only the type associated would be changed in the Query value. So representing the link in an anchor tag in UI would be as –

<a class=“data rss” href=“
		http://api.loklak.org/api/search.
		rss?timezoneOffset=-330&q=
		{{stringQuery}}” target=“_blank”>
</a>
<a class=”data json” href=”http://api.
		loklak.org/api/search
		.json?timezoneOffset=-330&q=
		{{stringQuery}}” target=”_blank“>
</a>

 

{{stringQuery}} is the actual query parameter to be passed to get the required results.

Testing RSS feed and JSON data

Search for a query on loklak, and click on the the RSS or JSON button below sidebar on results page and compare with the results.

RSS and JSON button should be similar to –

Resources

Continue ReadingAdd RSS feed and JSON output based on type specified with query param

Displaying Top Hashtags by sorting Hashtags based on the frequency

It is a good idea to display top hashtags on sidebar of loklak. To represent them, it is really important to sort out all unique hashtags on basis of frequency from results obtained from api.loklak. The implementation of the process involved would be discussed in this blog.

Raw Hashtag result

The Hashtags obtained as a parameter of type array containing array of strings into the sortHashtags() method inside the component typescript file of Info-box Component is in Raw Hashtag form.

Making Array of all Hashtags

Firstly, all the Hashtags would be added to a new Array – stored.

sortHashtags( statistics ) {
    let stored = [];
    if (statistics !== undefined && 
        statistics.length !== 0) {
        for (const s in statistics) {
            if (s) {
                for (let i = 0;i <
                    statistics[s].length; i++) {
                    stored.push(statistics[s][i]);
                }
            }
        }
    }
}

 

stored.push( element ) will add each element ( Hashtag ) into the stored array.

Finding frequency of each unique Hashtag

array.reduce() method would be used to store all the Hashtags inside the stored array with frequency of each unique Hashtag (e.g. [ ‘Hashtag1’: 3, ‘Hashtag2’: 2, ‘Hashtag3’: 5, … ]), where Hashtag1 has appeared 3 times, Hashtag2 appeared 2 times and so on.

stored = stored.reduce(function (acc, curr) {
    if (typeof acc[curr] === undefined’) {
        acc[curr] = 1;
    } else {
        acc[curr] += 1;
    }
    return acc;
}, []);

 

stored.reduce() would store the result inside stored array in the format mentioned above.

Using Object to get the required result

Object would be used with different combination of associated methods such as map, filter, sort and slice to get the required Top 10 Hashtags sorted on the basis of frequency of each unique Hashtag.

this.topHashtags = Object.keys(stored)
    .map(key => key.trim())
    .filter(key => key !== ”)
    .map(key => ([key, stored[key]]))
    .sort((a, b) => b[1]  a[1])
    .slice(0, 10);

 

At last, the result is stored inside topHashtags array. First map method is used to trim out spaces from all the keys ( Hashtags ), first filter is applied to remove all those Hashtags which are empty and then mapping each Hashtag as an array with a unique index inside the containing array. At last, sorting each Hashtag on basis of the frequency using sort method and slicing the results to get Top 10 Hashtags to be displayed on sidebar of loklak.

Testing Top Hashtags

Search something on loklak.org to obtain sidebar with results. Now look through the Top 10 Hashtags being displayed on the Sidebar info-box.

Resources

Continue ReadingDisplaying Top Hashtags by sorting Hashtags based on the frequency

Holding query search in Loklak

Continuously searching user’s input while the user kept typing is not good. It has certain drawbacks which result in continuous action dispatch and loading of the result page which depicts the site to be slow, but actually it is not. To prevent continuous loading and searching while user types a query in Loklak Search, it was important to keep hold on query until user completes typing and then start searching for the results. I am writing this blog post to emphasise a proper solution to constantly keep track of the query and search on an appropriate event.

Adding keypress event

First and foremost task would be to add keypress event to input tag on home and results page.

<input     
     

     (keypress)=“onEnter($event)” 
     [formControl]=“searchInputControl

/>

 

formControl will constantly keep track of the user input and will be stored in searchInputControl, while keypress event will activate onEnter() method firing an $event as an input to onEnter() method.

Adding onEnter() method

Note: Keeping track of special keypress event code 13 which is equivalent for an enter key press.

Add a new method onEnter() with event as an input which will execute its function only when the event’s keypress code is 13 (Enter key press). One point should be noted here that query to be searched should not be empty or containing leading and trailing spaces or should not contain only spaces (treated as an incomplete query to be searched).

onEnter(event: any) {  
   if (event.which === 13) {  

      if (this.searchInputControl.value.trim() !== ”) {

        // Actual work of query search will go here.

  }

}

Add query searching and migration code under onEnter()

Now comes the last part to shift query search and route migration code from setupSearchField() to the portion mentioned above in comments. We need to take one point in consideration that from now on we actually do not need to subscribe to formControl in ngOnInit() since now it depends on user query input.

Complete onEnter() in home component file

After completion, onEnter() method should be similar to:

onEnter(event: any) {
   if (event.which === 13) {
       if (this._queryControl.value.trim() !== ”) {
           this.store.dispatch(new 
               queryAction.RelocationAfterQuerySetAction());
           this.store.dispatch(new suggestAction
               .SuggestAction(this._queryControl.value.trim()));
           this.store.dispatch(new queryAction
               .InputValueChangeAction(this._queryControl.value.trim()
           ));
           this.router.navigate([`/search`],
               { queryParams: { query: this._queryControl.value.trim() }
               , skipLocationChange: true } );
           this.store.dispatch(new titleAction
           .SetTitleAction(this._queryControl.value.trim()
               +   Loklak Search’));
           this.getDataFromStore();
       }
   }
}

 

Since we are not depending on ngOnInit() therefore we also need to append getDataFromStore() to onEnter() method.

Complete onEnter() in feed-header component file

After completion, onEnter() method should be similar to:

onEnter(event: any) {
   if (event.which === 13) {
       if (this.searchInputControl.value.trim() !== '') {
           this.searchEvent.emit(this.searchInputControl
               .value.trim());
           this.setupSuggestBoxClosing();
       }
   }
}

Testing

Try typing different formats of query on home and results page of loklak.

Resources

Continue ReadingHolding query search in Loklak

Adding Push endpoint to send data from Loklak Search to Loklak Server

To provide enriched and sufficient amount of data to Loklak, Loklak Server should have multiple sources of data. The api/push.json endpoint of loklak server is used in Loklak to post the search result object to server. It will increase the amount and quality of data on server once the Twitter api is supported by Loklak (Work is in progress to add support for twitter api in loklak).

Creating Push Service

The idea is to create a separate service for making a Post request to server. First step would be to create a new ‘PushService’ under ‘services/’ using:

ng g service services/push

Creating model for Push Api Response

Before starting to write code for push service, create a new model for the type of response data obtained from Post request to ‘api/push.json’. For this, create a new file push.ts under ‘models/’ with the code given below and export the respective push interface method in index file.

export interface PushApiResponse {
   status: string;
   records: number;
   mps: number;
   message: string;
}

Writing Post request in Push Service

Next step would be to create a Post request to api/push.json using HttpClient module. Import necessary dependencies and create an object of HttpClient module in constructor and write a PostData() method which would take the data to be send, makes a Post request and returns the Observable of PushApiResponse created above.

import { Injectable } from ‘@angular/core’;
import {
   HttpClient,
   HttpHeaders,
   HttpParams
} from ‘@angular/common/http’;
import { Observable } from ‘rxjs’;
import {
	ApiResponse,
	PushApiResponse
} from ‘../models’;

@Injectable({
   providedIn: ‘root’
})
export class PushService {

   constructor( private http: HttpClient ) { }
   public postData(data: ApiResponse):
   		Observable<PushApiResponse> {

	const httpUrl = ‘https://api.loklak.org/
		api/push.json’;
	const headers = new HttpHeaders({
		‘Content-Type’: ‘application/
			x-www-form-urlencoded’,
		‘Accept’: ‘application/json’,
		‘cache-control’: ‘no-cache’
	});
	const {search_metadata, statuses} = data;
	
	// Converting the object to JSON string.
	const dataToSend = JSON.stringify({
		search_metadata: search_metadata,
		statuses});
	
	// Setting the data to send in
	// HttpParams() with key as ‘data’
	const body = new HttpParams()
		.set(‘data’, dataToSend);
	
	// Making a Post request to api/push.json
	// endpoint. Response Object is converted
	// to PushApiResponse type.
	return this.http.post<PushApiResponse>(
		httpUrl, body, {headers:
		headers
	});
   }
}

 

Note: Data (dataToSend) send to backend should be exactly in same format as obtained from server.

Pushing data into service dynamically

Now the main part is to provide the data to be send into the service. To make it dynamic, import the Push Service in ‘api-search.effects.ts’ file under effects and create the object of Push Service in its constructor.

import { PushService } from ‘../services’;
constructor(
   
   private pushService: PushService
) { }

 

Now, call the pushService object inside ‘relocateAfterSearchSuccess$’ effect method and pass the search response data (payload value of search success action) inside Push Service’s postData() method.

@Effect()
relocateAfterSearchSuccess$: Observable<Action>
   = this.actions$
       .pipe(
           ofType(
               apiAction.ActionTypes
			   	.SEARCH_COMPLETE_SUCCESS,
               apiAction.ActionTypes
			   	.SEARCH_COMPLETE_FAIL
           ),
           withLatestFrom(this.store$),
           map(([action, state]) => {
               this.pushService
			   .postData(action[‘payload’]);
           
       );

Testing Successful Push to Backend

To test the success of Post request, subscribe to the response data and print the response data on console. You should see something like:

Where each of these is a response of one successful Post request.

Resources

Continue ReadingAdding Push endpoint to send data from Loklak Search to Loklak Server

Creating new config to set Title of Loklak Webpages

For a Search Engine and Progressive Web Application (PWA) like Loklak, it is important to change the title of each web page with the appropriate query term or configuration of the current active page (A great title can result in higher click-throughs and more traffic to the site, and hence its followed by search engines like Google). For solving this problem, I am writing this blog post which creates a new configuration to change the title dynamically across the whole project by using just one Action thereby maximizing reusability of the code and minimizing duplication.

Create Action for title

We would need to create a new separate title action following the architecture in loklak.

Create a new title.ts file under Actions with the following code:

import { Action } from ‘@ngrx/store’;
export const ActionTypes = {
    SET_TITLE: ‘[Title] SET TITLE’
};
export class SetTitleAction implements Action {
    type = ActionTypes.SET_TITLE;
    constructor (public payload: string) { }
}
export type Actions
    = SetTitleAction;

Create reducer for title

For accessing the current title of the webpage, it is important to create a reducer function for the title following the pattern in loklak.

import * as titleAction from ‘../actions/title’;
export interface State {
	title: string;
}

export const initialState: State = {
	title: '',
};

export function reducer(state: State =
	initialState, action: titleAction.Actions):
	State {
	switch (action.type) {
		case titleAction.ActionTypes.SET_TITLE:
		{
			const titleObt: string =
				action.payload;
			return Object.assign({}, state,
			{
				title: titleObt
			});
		}
		default: {
			return state;
		}
	}
}
export const getTitle =
	(state: State) => state.title;

Create selector and add reducer function to root reducer

The title can be accessed from the current state from store when it would be added to the root state and reducer. Firstly import the title reducer created above and define the title property in root reducer and create a selector for the same.

import * as fromTitle from ‘./title’;
export interface State {
	
	title: fromTitle.State;
}

export const reducers: ActionReducerMap<State> = {
	
	title: fromTitle.reducer
};

export const getTitleState = (state: State) => state.title;

export const getTitle = createSelector(getTitleState,
   fromTitle.getTitle);

Create Effect to change actual title

Now comes the actual part which controls the changing of webpage title using ‘Title’ property of Platform-Browser. Create a new title.effects.ts file under effects using pattern in loklak.

import { Injectable } from ‘@angular/core’;
import { Title } from ‘@angular/platform-browser’;
import { Effect, Actions, ofType }
	from ‘@ngrx/effects’;
import { Observable } from ‘rxjs’;
import { map } from ‘rxjs/operators’;
import * as titleAction from ‘../actions/title’;

@Injectable()
export class SetTitleEffects {

	@Effect({ dispatch: false })
	resetTitle$: Observable<void> = this.actions$
		.pipe(
			ofType(
				titleAction.ActionTypes
					.SET_TITLE
			),
			map((action) => {
			const title =
		action[‘payload’];
			this.titleService
		.setTitle(title);
			})
		);

	constructor(
		private actions$: Actions,
		private titleService: Title
	) { }
}

 

Here the ‘payload’ actually contains the query searched by the user, which will be used update the title.

Note: Before adding a new Effect, the old Effect needs to be removed.

After creating the Effect, add it into app.module.ts and export it in the index.ts.

import {
	
	SetTitleEffects
} from ‘./effects’;

EffectsModule.forRoot([
	
	SetTitleEffects
])

How to set title using this configuration?

Now comes the interesting part. It becomes very easy to set title from any service or component by following the procedure given below.

  • Import the created title action.

import * as titleAction from ‘../actions/title’;

 

  • Set the required title by dispatching the SetTitleAction.
this.store.dispatch(new titleAction
	.SetTitleAction(‘Loklak Search’));

 

You’re all set now! The title has been changed to ‘Loklak Search’ which is what we expect.

Resources

Continue ReadingCreating new config to set Title of Loklak Webpages