Implementing news feature in loklak

The idea is to bring out an useful feature out of the enriched high quality of data provided by api.loklak.org. Through this blog post, I’ll be discussing about the implementation of a news feature similar to Google to provide latest news relevant to the query in loklak.

Creating news Component

First step in implementing news feature would be to create a news component which would be rendered on clicking of news tab.

ng g component feed/news

Creating an action & reducer for news status

This step involves creating an action & reducer function for tracking the status of news component on click of news tab. When the user will click on news tab, corresponding action would be dispatched to display the news component in feed.

import { Action } from ‘@ngrx/store’;
export const ActionTypes = {
    NEWS_STATUS: ‘[NEWS] NEWS STATUS’
};
export class NewsStatusAction implements Action {
    type = ActionTypes.NEWS_STATUS;
    constructor (public payload: boolean) { }
}
export type Actions
    = NewsStatusAction;

 

Corresponding reducer function for news status to store current news status would be:

export function reducer(state: State = initialState, 
   action: newsStatusAction.NewsStatusAction): State {
switch (action.type) {
    case newsStatusAction.ActionTypes.NEWS_STATUS: {
    const newsStatusPayload: boolean = action.payload;
    return Object.assign({}, state, {
                newsStatus: newsStatusPayload
            });
        }
        default: {
            return state;
        }
    }
}

 

I would not be getting into each line of code here, only the important portions would be discussed here.

Creating news-org to store the names of news organizations

A new file news-org would be created inside app/shared/ folder containing list of news organizations in an array newsOrgs from which we would extract and filter results.

export const newsOrgs = [
    ‘CNN’,
    ‘nytimes’
];

 

Note: This is a dynamic implementation of news service and hence, the list of news organizations can be easily modified/updated here.

We would also need to export the newsOrgs in the index file inside shared/ folder as:

export { newsOrgs } from ‘./news-org’;

Creating ngrx action, reducer & effect for news success

Now comes the main part. We would need an action to dispatch on successful response of news search and corresponding reducer to keep appending the async news results.

News action would be created following the simple ngrx composition as:

import { Action } from ‘@ngrx/store’;
import { ApiResponse } from ‘../models/api-response’;
export const ActionTypes = {
    NEWS_SEARCH_SUCCESS: ‘[NEWS] SEARCH SUCCESS’
};
export class NewsSearchSuccessAction implements Action {
    type = ActionTypes.NEWS_SEARCH_SUCCESS;
    constructor (public payload: ApiResponse) { }
}
export type Actions
    = NewsSearchSuccessAction;

 

One of the important parts of news implementation is the reducer function corresponding to the news success action. We would actually append the async results provided by news effect into the store.

import * as newsSuccessAction from ‘../actions/newsSuccess’;
import { ApiResponseResult } from ‘../models’;

export interface State {
newsResponse: ApiResponseResult[];
}
export const initialState: State = {
newsResponse: [],
};
export function reducer(state: State = initialState,
   action: newsSuccessAction.NewsSearchSuccessAction): State {
    switch (action.type) {
        case newsSuccessAction.ActionTypes.NEWS_SEARCH_SUCCESS: {
            // Appending the news result
            return Object.assign({}, state, {
                newsResponse: […action.payload.statuses]
            });
        }
        default: {
            return state;
        }
    }
}
export const getNewsResponse = (state: State) => state.newsResponse;

 

Now, we would create an effect which would actually use the search service to fetch results for the list of organizations one by one will keep dispatching a corresponding action to append the response into the store to be displayed in news feed. Instead of showing whole code for the effect, only the main logic is being displayed.

import { newsOrgs } from ‘../shared/news-org’;
...
@Injectable()
export class DisplayNewsEffects {
    @Effect({ dispatch: false })
        searchNews$: Observable<void> = this.actions$
        .pipe(
        ofType(
        newsAction.ActionTypes.NEWS_STATUS
        ),
        map((action) => {
            if (action[‘payload’]) {
                const orgs = newsOrgs;
                orgs.forEach((org) => {
                const searchServiceConfig:
                    SearchServiceConfig = 
                    new SearchServiceConfig();
                this.apiSearchService
                .fetchQuery(‘from:’
                  + org, searchServiceConfig)
                    .subscribe(response =>
                this.store$.dispatch(
                     new newsStatusAction
                     .NewsSearchSuccessAction(response)));
                });
            }
        })
    );

Filtering the news response inside news component

The basic idea is to filter the results from news response which contain the query and display them using news template on click of news tab on results page of loklak.

import { Query } from ‘../../models/query’;
import { Observable } from ‘rxjs’;
...
public query: string;
public query$: Observable<Query>;
...
ngOnInit() {
   const texts = [];
   this.store.select(fromRoot.getQuery).subscribe(res => 
       this.query = res.displayString);
   this.query$ = this.store.select(fromRoot.getQuery);
   this.store.select(fromRoot.getNewsResponse).subscribe(
       v => {
       for ( let i = 0; i < v.length; i++ ) {
           this.newsResponse.push(v[i]);
           if (v[i][‘text’].includes(this.query
               .replace(/\s/g, ”).toLowerCase())) {
               if (!texts.includes(v[i][‘text’]
                   .replace(/\s/g, ”).toLowerCase())) {
                   texts.push(v[i][‘text’]
                       .replace(/\s/g, ”).toLowerCase());
                   this.newsResponse.push(v[i]);
               }
           }
       }
   });
}

 

In above configuration, firstly we needed the current query which we extracted from ngrx store and then checking whether the text of news results contain the query string or not. If the query is present inside the text, we would add it into an array which would be used to display news feed using feed-card.

Displaying the news response

Displaying the news response is very easy, we just need to pass the news response items with index inside feed-card component which would display the news feed similar to the normal feed under the news section.

<div class=“wrapper feed-results”>
   <div *ngFor=“let item of newsResponse; let i = index “>
     <feed-card [feedItem]=“item” [feedIndex]=“i”></feed-card>
   </div>
</div>

Creating News tab to display news feed

To provide a tab to user to access news feed, we would need to create a News tab along with the other searching tools inside feed-advanced-search component as:

<button [class.selected]=“selectedTab === ‘news'” class=“tab” value=“news”
   (click)=“getFilterResults($event.currentTarget.value)”>
   News
</button>

 

Note: Complete code for news implementation can be found here and the news filter can be found here.

Testing news feature

Search a query on loklak, and then click on news tab to get the latest relevant news matching the query.

Resources

Continue ReadingImplementing news feature in loklak

Creating new config to set Title of Loklak Webpages

For a Search Engine and Progressive Web Application (PWA) like Loklak, it is important to change the title of each web page with the appropriate query term or configuration of the current active page (A great title can result in higher click-throughs and more traffic to the site, and hence its followed by search engines like Google). For solving this problem, I am writing this blog post which creates a new configuration to change the title dynamically across the whole project by using just one Action thereby maximizing reusability of the code and minimizing duplication.

Create Action for title

We would need to create a new separate title action following the architecture in loklak.

Create a new title.ts file under Actions with the following code:

import { Action } from ‘@ngrx/store’;
export const ActionTypes = {
    SET_TITLE: ‘[Title] SET TITLE’
};
export class SetTitleAction implements Action {
    type = ActionTypes.SET_TITLE;
    constructor (public payload: string) { }
}
export type Actions
    = SetTitleAction;

Create reducer for title

For accessing the current title of the webpage, it is important to create a reducer function for the title following the pattern in loklak.

import * as titleAction from ‘../actions/title’;
export interface State {
	title: string;
}

export const initialState: State = {
	title: '',
};

export function reducer(state: State =
	initialState, action: titleAction.Actions):
	State {
	switch (action.type) {
		case titleAction.ActionTypes.SET_TITLE:
		{
			const titleObt: string =
				action.payload;
			return Object.assign({}, state,
			{
				title: titleObt
			});
		}
		default: {
			return state;
		}
	}
}
export const getTitle =
	(state: State) => state.title;

Create selector and add reducer function to root reducer

The title can be accessed from the current state from store when it would be added to the root state and reducer. Firstly import the title reducer created above and define the title property in root reducer and create a selector for the same.

import * as fromTitle from ‘./title’;
export interface State {
	
	title: fromTitle.State;
}

export const reducers: ActionReducerMap<State> = {
	
	title: fromTitle.reducer
};

export const getTitleState = (state: State) => state.title;

export const getTitle = createSelector(getTitleState,
   fromTitle.getTitle);

Create Effect to change actual title

Now comes the actual part which controls the changing of webpage title using ‘Title’ property of Platform-Browser. Create a new title.effects.ts file under effects using pattern in loklak.

import { Injectable } from ‘@angular/core’;
import { Title } from ‘@angular/platform-browser’;
import { Effect, Actions, ofType }
	from ‘@ngrx/effects’;
import { Observable } from ‘rxjs’;
import { map } from ‘rxjs/operators’;
import * as titleAction from ‘../actions/title’;

@Injectable()
export class SetTitleEffects {

	@Effect({ dispatch: false })
	resetTitle$: Observable<void> = this.actions$
		.pipe(
			ofType(
				titleAction.ActionTypes
					.SET_TITLE
			),
			map((action) => {
			const title =
		action[‘payload’];
			this.titleService
		.setTitle(title);
			})
		);

	constructor(
		private actions$: Actions,
		private titleService: Title
	) { }
}

 

Here the ‘payload’ actually contains the query searched by the user, which will be used update the title.

Note: Before adding a new Effect, the old Effect needs to be removed.

After creating the Effect, add it into app.module.ts and export it in the index.ts.

import {
	
	SetTitleEffects
} from ‘./effects’;

EffectsModule.forRoot([
	
	SetTitleEffects
])

How to set title using this configuration?

Now comes the interesting part. It becomes very easy to set title from any service or component by following the procedure given below.

  • Import the created title action.

import * as titleAction from ‘../actions/title’;

 

  • Set the required title by dispatching the SetTitleAction.
this.store.dispatch(new titleAction
	.SetTitleAction(‘Loklak Search’));

 

You’re all set now! The title has been changed to ‘Loklak Search’ which is what we expect.

Resources

Continue ReadingCreating new config to set Title of Loklak Webpages

Reactive Side Effects of Actions in Loklak Search

In a Redux based application, every component of the application is state driven. Redux based applications manage state in a predictable way, using a centralized Store, and Reducers to manipulate various aspects of the state. Each reducer controls a specific part of the state and this allows us to write the code which is testable, and state is shared between the components in a stable way, ie. there are no undesired mutations to the state from any components. This undesired mutation of the shared state is prevented by using a set of predefined functions called reducers which are central to the system and updates the state in a predictable way.

These reducers to update the state require some sort triggers to run. This blog post concentrates on these triggers, and how in turn these triggers get chained to form a Reactive Chaining of events which occur in a predictable way, and how this technique is used in latest application structure of Loklak Search. In any state based asynchronous application, like, Loklak Search the main issue with state management is to handle the asynchronous action streams in a predictable manner and to chain asynchronous events one after the other.  The technique of reactive action chaining solves the problem of dealing with asynchronous data streams in a predictable and manageable manner.

Overview

Actions are the triggers for the reducers, each redux action consists of a type and an optional payload. Type of the action is like its ID which should be purposely unique in the application. Each reducer function takes the current state which it controls and action which is dispatched. The reducer decides whether it needs to react to that action or not. If the user reacts to the action, it modifies the state according to the action payload and returns the modified state, else, it returns the original state. So at the core, the actions are like the triggers in the application, which make one or more reducers to work. This is the basic architecture of any redux application. The actions are the triggers and reducers are the state maintainers and modifiers. The only way to modify the state is via a reducer, and a reducer only runs when a corresponding action is dispatched.

Now, who dispatches these actions? This question is very important. The Actions can be technically dispatched from anywhere in the application, from components, from services, from directives, from pipes etc. But we almost in every situation will always want the action to be dispatched by the component. Component who wishes to modify the state dispatch the corresponding actions.

Reactive Effects

If the components are the one who dispatch the action, which triggers a reducer function which modifies the state, then what are these effects, cause the cycle of events seem pretty much complete. The Effects are the Side Effects, of a particular action. The term “side effect” means these are the piece of code which runs whenever an action is dispatched. Don’t confuse them with the reducer functions, effects are not same as the reducer functions as they are not associated with any state i.e. they don’t modify any state. They are the independent sources of other actions. What this means is whenever an Action is dispatched, and we want to dispatch some other action, maybe immediately or asynchronously, we use these side effects. So in a nutshell, effects are the piece of code which reacts to a particular action, and eventually dispatches some other actions.

The most common use case of effects is to call a corresponding service and fetch the data from the server, and then when the data is loaded, dispatch a SearchCompleteAction. These are the simplest of use cases of effects and are most commonly use in Loklak Search. This piece of code below shows how it is done.

@Effect()
search$: Observable<Action>
= this.actions$
.ofType(apiAction.ActionTypes.SEARCH)
.map((action: apiAction.SearchAction) => action.payload)
.switchMap(query => {
return this.apiSearchService.fetchQuery(query)
.map(response => new apiAction.SearchCompleteSuccessAction(response))

This is a very simple type of an effect, it filters out all the actions and react to only the type of action which we are interested in, here SEARCH, and then after calling the respective Service, it either dispatches SearchComplete or a SearchFail action depending on the status of the response from the API. The effect runs on SEARCH action and eventually dispatches, the success or the fail actions.

This scheme illustrates the effect as another point apart from components, to dispatch some action. The difference being, components dispatch action on the basis of the User inputs and events, whereas Effects dispatch actions on the basis of other actions.

Reactive Chaining of Actions

We can thus take the advantage of this approach in a form of reactive chaining of actions. This reactive chaining of actions means that some component dispatches some action, which as a side effects, dispatches some other action, and it dispatches another set of actions and so on. This means a single action dispatched from a component, brings about the series of actions which follow one another. This approach makes it possible to write reducers at the granular level rather than complete state level. As a series of actions can be set up which, start from a fine grain, and reaches out to a coarse grain. The loklak search application uses this technique to update the state of query. The reducers in the loklak search rather than updating the whole query structure update only the required part of the state. This helps in code maintainability as the one type of query attribute has no effect on the other type

@Effect()
inputChange$: Observable<Action>
= this.actions$
.ofType(queryAction.ActionTypes.VALUE_CHANGE)
.map(_ => new queryAction.QueryChangeAction(''));

@Effect()
filterChange$: Observable<Action>
= this.actions$
.ofType(queryAction.ActionTypes.FILTER_CHANGE)
.map(_ => new queryAction.QueryChangeAction(''));

Here the QUERY_CHANGE action further can do other processing of the query and then dispatch the SearchAction, which eventually calls the service and then return the response, then the success or fail actions can be dispatched eventually.

Conclusion

The reactive side effects is one of the most beautiful thing we can do with Redux and Reactive Programming. They provide an easy clean way to chain events in an application, which helps in a cleaner non-overlapping state management along with clean and simple reducers. This idea of the reactive chaining can be extended to any level of sophistication, and that too in a simple and easy to understand manner.

Resources and links

Continue ReadingReactive Side Effects of Actions in Loklak Search

Adding unit tests for effects in Loklak Search

Loklak search uses @ngrx/effects to listen to actions dispatched by the user and sending API request to the loklak server. Loklak search, currently, has seven effects such as Search Effects,  Suggest Effects which runs to make the application reactive. It is important to test these effects to ensure that effects make API calls at the right time and then map the response to send it back to the reducer.

I will  explain here how I added unit tests for the effects. Surprisingly, the test coverage increased from 43% to 55% after adding these tests.

Effects to test

We are going to test effects for user search. This effect listens to the event of type USER_SEARCH and makes a call to the user-search service with the query as a parameter. After a response is received, it maps the response and passes it on the UserSearchCompleteSuccessAction action which performs the later operation. If the service fails to get a response, it makes a call to the UserSearchCompleteFailAction.

Code

ApiUserSearchEffects is the effect which detects if the USER_SEARCH action is dispatched from some component of the application and consequently, it makes a call to the UserSearchService and handles the JSON response received from the server. The effects then, dispatch the action new UserSearchCompleteSuccessAction if response is received from server or either dispatch the action new UserSearchCompleteFailAction if no response is received. The debounce time is set to 400 so that response can be flushed if a new USER_SEARCH is dispatched within the next 400ms.

For this effect, we need to test if the effects actually runs when USER_SEARCH action is made. Further, we need to test if the correct parameters are supplied to the service and response is handled carefully. We also, need to check if the response if really flushed out within the certain debounce time limit.

@Injectable()
export class ApiUserSearchEffects {@Effect()
search$: Observable<Action>
= this.actions$
.ofType(userApiAction.ActionTypes.USER_SEARCH)
.debounceTime(400)
.map((action: userApiAction.UserSearchAction) => action.payload)
.switchMap(query => {
const nextSearch$ = this.actions$.ofType(userApiAction.ActionTypes.USER_SEARCH).skip(1);const follow_count = 10;return this.apiUserService.fetchQuery(query.screen_name, follow_count)
.takeUntil(nextSearch$)
.map(response => new userApiAction.UserSearchCompleteSuccessAction(response))
.catch(() => of(new userApiAction.UserSearchCompleteFailAction()));
});constructor(
private actions$: Actions,
private apiUserService: UserService
) { }

}

Unit test for effects

  • Configure the TestBed class before starting the unit test and add all the necessary imports (most important being the EffectsTestingModule) and providers. This step will help to isolate the effects completely from all other components and testing it independently. We also need to create spy object which spies on the method userService.fetchQuery with provider being UserService.

beforeEach(() => TestBed.configureTestingModule({
imports: [
EffectsTestingModule,
RouterTestingModule
],
providers: [
ApiUserSearchEffects,
{
provide: UserService,
useValue: jasmine.createSpyObj(‘userService’, [‘fetchQuery’])
}
]
}));
  • Now, we will be needing a function setup which takes params which are the data to be returned by the Mock User Service. We can now configure the response to returned by the service. Moreover, this function will be initializing EffectsRunner and returning ApiUserSearchEffects so that it can be used for unit testing.

function setup(params?: {userApiReturnValue: any}) {
const userService = TestBed.get(UserService);
if (params) { userService.fetchQuery.and.returnValue(params.userApiReturnValue);
}return {
runner: TestBed.get(EffectsRunner),
apiUserSearchEffects: TestBed.get(ApiUserSearchEffects)
};
}

 

  • Now we will be adding unit tests for the effects. In these tests, we are going to test if the effects recognise the action and return some new action based on the response we want and if it maps the response only after a certain debounce time.We have used fakeAsync() which gives us access to the tick() function. Next, We are calling the function setup and pass on the Mock Response so that whenever User Service is called it returns the Mock Response and never runs the service actually. We will now queue the action UserSearchAction in the runner and subscribe to the value returned by the effects class. We can now test the value returned using expect() block and that the value is returned only after a certain debounce time using tick() block.

it(‘should return a new userApiAction.UserSearchCompleteSuccessAction, ‘ +
‘with the response, on success, after the de-bounce’, fakeAsync(() => {
const response = MockUserResponse;const {runner, apiUserSearchEffects} = setup({userApiReturnValue: Observable.of(response)});

const expectedResult = new userApiAction.UserSearchCompleteSuccessAction(response);

runner.queue(new userApiAction.UserSearchAction(MockUserQuery));

let result = null;
apiUserSearchEffects.search$.subscribe(_result => result = _result);
tick(399); // test debounce
expect(result).toBe(null);
tick(401);
expect(result).toEqual(expectedResult);
}));

it(‘should return a new userApiAction.UserSearchCompleteFailAction,’ +
‘if the SearchService throws’, fakeAsync(() => {
const { runner, apiUserSearchEffects } = setup({ userApiReturnValue: Observable.throw(new Error()) });

const expectedResult = new userApiAction.UserSearchCompleteFailAction();

runner.queue(new userApiAction.UserSearchAction(MockUserQuery));

let result = null;
apiUserSearchEffects.search$.subscribe(_result => result = _result );

tick(399); // Test debounce
expect(result).toBe(null);
tick(401);
expect(result).toEqual(expectedResult);
}));
});

Reference

Continue ReadingAdding unit tests for effects in Loklak Search