Extending the News Feature to Show results from multiple organisations

News Tab in Susper was earlier implemented to show results only from a single organisation using  site: modifier for query facet. In this blog I will discuss about how I have modified the current News Tab to show results from various News organisation like BBC, Al Jazeera, The Guardian etc. Implementation: Step 1: Creating a JSON file to store organisations detail: We need to decide from which organisations  we will fetch results using YaCy Server and then display it in Susper. We have provided a JSON file where user can add or delete News sources easily. The results will only limited to the organisations which are present in the JSON file. Step 2: Creating a service to fetch details from JSON file: Now after creating the the JSON file we need a service which will fetch results from the JSON file according to our need. This service will be a simple  Angular Service having a class GetJsonService and it will access the newsFile.json and map the results in JSON format. Here is the service which does this task for us. Step 3: Creating a service to fetch news accordingly: Now after fetching the JSON result we need a service to fetch the News Results from YaCy Server. I have created a separate service to do this task where I have fetched results using site: modifier from each organisation and returned the results.The code for the news.service.ts is below. export class NewsService { constructor(private jsonp: Jsonp) { } getSearchResults(searchquery, org) { let searchURL = 'https://yacy.searchlab.eu/solr/select?query='; searchURL += searchquery.query + ' site:' + org; let params = new URLSearchParams(); for (let key in searchquery) { if (searchquery.hasOwnProperty(key)) { params.set(key, searchquery[key]); } } //Set other parameters return this.jsonp .get(searchURL, {search: params}).map(res => res.json()[0] ).catch(this.handleError); }   Step 4: Updating the results section: Now we have a service that gives results from a single organisation and a JSON list of organisations. Now in results.component.ts we can simply subscribe to getJsonService and in a loop we will call getNewsService by changing the organisation in every iteration. We will then check that whether we are getting the valid results or not (undefined). The results which are not valid can cause errors when we will try to read any field of an undefined variable. Then, We will simply append the 2 result items from each organisation in an empty array and later use this array to show results. this.getJsonService.getJSON().subscribe(res => { this.newsResponse = []; for (let i = 0; i < res.newsOrgs.length; i++) { this.getNewsService.getSearchResults(querydata,res.newsOrgs[i].provider).subscribe( response => { if (response.channels[0].items[0] !== undefined) { this.newsResponse.push(response.channels[0].items[0]); } if (response.channels[0].items[1] !== undefined) { this.newsResponse.push(response.channels[0].items[1]); } } ); } });   The newsClick() function is activated on clicking News Tab and it updates the query and its details in store. Step 5 Displaying the results: Now we will modify the results.component.html to show results from new newsResponse array which have 2 results each from 5 organisations. For this we will use each item of newsResponse using *ngFor and display its title,link…

Continue ReadingExtending the News Feature to Show results from multiple organisations

Implementing news feature in loklak

The idea is to bring out an useful feature out of the enriched high quality of data provided by api.loklak.org. Through this blog post, I’ll be discussing about the implementation of a news feature similar to Google to provide latest news relevant to the query in loklak. Creating news Component First step in implementing news feature would be to create a news component which would be rendered on clicking of news tab. ng g component feed/news Creating an action & reducer for news status This step involves creating an action & reducer function for tracking the status of news component on click of news tab. When the user will click on news tab, corresponding action would be dispatched to display the news component in feed. import { Action } from ‘@ngrx/store’; export const ActionTypes = { NEWS_STATUS: ‘[NEWS] NEWS STATUS’ }; export class NewsStatusAction implements Action { type = ActionTypes.NEWS_STATUS; constructor (public payload: boolean) { } } export type Actions = NewsStatusAction;   Corresponding reducer function for news status to store current news status would be: export function reducer(state: State = initialState, action: newsStatusAction.NewsStatusAction): State { switch (action.type) { case newsStatusAction.ActionTypes.NEWS_STATUS: { const newsStatusPayload: boolean = action.payload; return Object.assign({}, state, { newsStatus: newsStatusPayload }); } default: { return state; } } }   I would not be getting into each line of code here, only the important portions would be discussed here. Creating news-org to store the names of news organizations A new file news-org would be created inside app/shared/ folder containing list of news organizations in an array newsOrgs from which we would extract and filter results. export const newsOrgs = [ ‘CNN’, ‘nytimes’ ];   Note: This is a dynamic implementation of news service and hence, the list of news organizations can be easily modified/updated here. We would also need to export the newsOrgs in the index file inside shared/ folder as: export { newsOrgs } from ‘./news-org’; Creating ngrx action, reducer & effect for news success Now comes the main part. We would need an action to dispatch on successful response of news search and corresponding reducer to keep appending the async news results. News action would be created following the simple ngrx composition as: import { Action } from ‘@ngrx/store’; import { ApiResponse } from ‘../models/api-response’; export const ActionTypes = { NEWS_SEARCH_SUCCESS: ‘[NEWS] SEARCH SUCCESS’ }; export class NewsSearchSuccessAction implements Action { type = ActionTypes.NEWS_SEARCH_SUCCESS; constructor (public payload: ApiResponse) { } } export type Actions = NewsSearchSuccessAction;   One of the important parts of news implementation is the reducer function corresponding to the news success action. We would actually append the async results provided by news effect into the store. import * as newsSuccessAction from ‘../actions/newsSuccess’; import { ApiResponseResult } from ‘../models’; export interface State { newsResponse: ApiResponseResult[]; } export const initialState: State = { newsResponse: [], }; export function reducer(state: State = initialState, action: newsSuccessAction.NewsSearchSuccessAction): State { switch (action.type) { case newsSuccessAction.ActionTypes.NEWS_SEARCH_SUCCESS: { // Appending the news result return Object.assign({}, state, { newsResponse: […action.payload.statuses] });…

Continue ReadingImplementing news feature in loklak

Adding News Tab in Susper

Most of the current search engines have a News tab where results are displayed which are fetched from different News Organisations. The latest results are displayed on top followed by the older ones. Current market leader, Google even uses AI and Machine Learning to analyze the contents before delivering it as results to the user. In this blog, I will describe how I have implemented a basic News Tab in Susper which show news results from dailymail.co.uk . This News tab can be improved in future to match the market leader. Implementation of News tab in Susper: Implementation of News tab in UI: To implement News tab the current design pattern has been followed in Susper. Here is the code to show News tab on the results page. <ul type="none" id="search-options"> <li [class.active_view]="Display('news')" (click)="newsClick()">News</li> Angular attribute binding is used to bind the active_view property with the "Display('news')" function and the click event is associated with "newsClick()" function. Here is the code for newsClick() function which filters the result using 'site:' parameter of yacy and then dispatches the 'urldata' object to 'QueryServerAction(urldata)' function in Query Action. newsClick() { let urldata = Object.assign({}, this.searchdata); this.getPresentPage(1); this.resultDisplay = 'news'; delete urldata.fq; urldata.rows = 10; if (urldata.query.substr(urldata.query.length - 25, 25) !== " site:www.dailymail.co.uk") { urldata.query += " site:www.dailymail.co.uk"; } else { urldata.query += ""; } urldata.resultDisplay = this.resultDisplay; this.store.dispatch(new queryactions.QueryServerAction(urldata)); } Here is the code for Query Action in query.ts file. It has two actions QueryAction and QueryServerAction export const ActionTypes = { QUERYCHANGE: type('[Query] Change'), QUERYSERVER: type('[Query] Server'), }; export class QueryAction implements Action { type = ActionTypes.QUERYCHANGE; constructor(public payload: any) {} } export class QueryServerAction implements Action { type = ActionTypes.QUERYSERVER; constructor(public payload: any) {} } export type Actions = QueryAction|QueryServerAction ; Now the reducer takes either of the two query actions and the current state and returns the new state to the store. Here is the code for the reducer query.ts export const CHANGE = 'CHANGE'; export interface State { query: string; wholequery: any; } const initialState: State = { query: '', wholequery: { query: '', rows: 10, start: 0, mode: 'text' }, }; export function reducer(state: State = initialState, action: query.Actions): State { switch (action.type) { case query.ActionTypes.QUERYCHANGE: { const changeQuery = action.payload; return Object.assign({}, state, { query: changeQuery, wholequery: state.wholequery }); } case query.ActionTypes.QUERYSERVER: { let serverQuery = Object.assign({}, action.payload); let resultCount = 10; if (localStorage.getItem('resultscount')) { resultCount = JSON.parse(localStorage.getItem('resultscount')).value || 10; } let instantsearch = JSON.parse(localStorage.getItem('instantsearch')); if (instantsearch && instantsearch.value) { resultCount = 10; } serverQuery.rows = resultCount; return Object.assign({}, state, { wholequery: serverQuery, query: state.query }); } default: { return state; } } } export const getpresentquery = (state: State) => state.query; export const getpresentwholequery = (state: State) => state.wholequery;   From the store the modified query is available to all other components of the appand is used by search service to get the filtered results. The search results are then stored in observable items$ in results.components.ts and is then used in results.components.html to display results under…

Continue ReadingAdding News Tab in Susper