Dispatching a search action to get result for a query in Loklak Search

For a new comer in Loklak, it’s not really easy to understand the complete codebase to just get results for a query without explicitly using navigation route i.e. ‘/search’. In order to help newcomers to easily understand a simple way to get results for a query in the codebase, I am writing this blog post.

Setting up a query

A sample searchQuery will be created of type Query. And following discussion will provide a way to get the results for a sample query – ‘from:Fossasia’ (Last 30 days tweets from FOSSASIA).

searchQuery: Query = {
    displayString: ‘from:FOSSASIA,
    queryString: ‘from:FOSSASIA’,
    routerString: ‘from:FOSSASIA’,
    filter: { video: false, image: false },
    location: null,
    timeBound: { since: null, until: null },
    from: true
}

 

Process discussed here can be used to get results in any class file within Loklak Search.

Using necessary imports

First step would be to add all necessary imports.

import { OnInit } from ‘@angular/core’;
import { Store } from ‘@ngrx/store’;
import { Observable } from ‘rxjs/Observable’;
import * as fromRoot from ‘../../reducers’;
import { Query } from ‘../../models/query’;
import * as searchAction from ‘../../actions/api’;
import { ApiResponseResult } from ‘../../models/api-response’;

 

Note: No need to import OnInit, if the class already implements it (OnInit is one of the Angular’s Lifecycle Hooks which controls the changes when the component containing it gets loaded initially).

The Query is used to define the type of searchQuery variable created above. And ApiResponseResult is the type of result which will be obtained on searching the searchQuery.

Declaring response variables and store object

Next step would be to declare the variables with the type of response and create store object of type Store with the current State of the application.

public isSearching$: Observable<boolean>;
public apiResponseResults$: Observable<ApiResponseResult[]>;
constructor(
    private store: Store<fromRoot.State>
) { }

 

isSearching$ holds an Observable which can be subscribed to get a boolean value which states the current status of searching of query. apiResponseResults$ holds the actual response of query.

Dispatching a SearchAction

The most crucial part comes here to dispatch a SearchAction with the created searchQuery as payload. A new method would be created with void as return type to dispatch the SearchAction.

private queryFromURL(): void {
    this.store.dispatch(
        new searchAction.SearchAction(this.searchQuery)
    );
}

Selecting and storing results from store

A new method would be created to store the current searching status along with the response of input query in isSearching$ and apiResponseResults$ respectively. Now the selector for the reducer function corresponding to the search and api response entities would be called.

private getDataFromStore(): void {
    this.isSearching$ = 
        this.store.select(fromRoot.getSearchLoading);
    this.apiResponseResults$ =   
        this.store.select(
            fromRoot.getApiResponseEntities
    );
}

Dispatching and Selecting on view Init

create ngOnInit() method, If you have not already done and call the respective dispatching and select method inside it respectively.

ngOnInit() {
    this.queryFromURL();
    this.getDataFromStore();
}

How should one test the results?

Call these lines inside any method and look for the results on browser console window.

console.log(this.isSearching$);
console.log(this.apiResponseResults$);

Resources

Continue ReadingDispatching a search action to get result for a query in Loklak Search

Adding ‘Scroll to Top’ in Loklak Search

It is really important for a Progressive Web Application like Loklak to be user friendly. To enhance the user experience and help user scroll up through the long list of results, a new ‘Scroll to Top’ feature has been added to Loklak.

Integration Process

The feature is built using HostListener and Inject property of Angular, which allows to use Window and Document functionalities similar to Javascript.

Adding HostListener

First step would be to import HostListener and Inject, and create property of document using with Inject.

import { HostListener, Inject } from ‘@angular/core’;
navIsFixed: boolean;
constructor(
    @Inject(DOCUMENT) private document: Document
) { }

 

Next step would be to create method to detect the current state of window, and scroll to the top of window.

@HostListener(‘window:scroll’, [])
onWindowScroll() {
    if (window.pageYOffset
        || document.documentElement.scrollTop
        || document.body.scrollTop > 100) {
                this.navIsFixed = true;
    } else if (this.navIsFixed && window.pageYOffset
        || document.documentElement.scrollTop
            || document.body.scrollTop < 10) {
                    this.navIsFixed = false;
            }
        } scrollToTop() {
            (function smoothscroll() {
                const currentScroll =
                    document.documentElement.scrollTop
                        || document.body.scrollTop;
            if (currentScroll > 0) {
            window.requestAnimationFrame(smoothscroll);
            window.scrollTo(0,
                currentScroll  (currentScroll / 5));
        }
    })();
}

 

The last step would be to add user interface part i.e. HTML and CSS component for the Scroll to Top feature.

Creating user interface

HTML

HTML portion contains a div tag with class set to CSS created and a button with click event attached to it to call the scrollToTop() method created above.

CSS

The CSS class corresponding to the div tag provides the layout of the button.

button {
   display: block;
   margin: 32px auto;
   font-size: 1.1em;
   padding: 5px;
   font-weight: 700;
   background: transparent;
   border: none;
   cursor: pointer;
   height: 40px;
   width: 40px;
   font-size: 30px;
   color: white;
   background-color: #d23e42;
   border-radius: 76px;
   &:focus {
       border: none;
       outline: none;
   }
}
.scroll-to-top {
  position: fixed;
  bottom: 15px;
  right: 15px;
  opacity: 0;
  transition: all .2s ease-in-out;
}
.show-scroll {
  opacity: 1;
  transition: all .2s ease-in-out;
}

 

The button style creates a round button with an interactive css properties. scroll-to-top class defines the behaviour of the div tag when user scrolls on the results page.

How to use?

Goto Loklak and search a query. On results page, scroll down the window to see upward arrow contained in a red circle like:

On clicking the button, user should be moved to the top of results page.

Resources

Continue ReadingAdding ‘Scroll to Top’ in Loklak Search

Adding Speech Component in Loklak Search

Speech recognition service for voice search is already embedded in Loklak. Now the idea is to use this service and create a new separate component for voice recognition with an interactive and user friendly interface. This blog will cover every single portion of an Angular’s redux based component from writing actions and reducers to the use of created component in other required components.

Creating Action and Reducer Function

The main idea to create an Action is to control the flow of use of Speech Component in Loklak Search. The Speech Component will be called on and off based on this Action.

Here, the first step is to create speech.ts file in actions folder with the following code:

import { Action } from '@ngrx/store';

export const ActionTypes = {
   MODE_CHANGE: '[Speech] Change',
};

export class SearchAction implements Action {
   type = ActionTypes.MODE_CHANGE;

   constructor(public payload: any) {}
}

export type Actions
   = SearchAction;

 

In the above segment, only one action (MODE_CHANGE) has been created which is like a boolean value which is being returned as true or false i.e. whether the speech component is currently in use or not. This is a basic format to be followed in creating an Action which is being followed in Loklak Search. The next step would be to create speech.ts file in reducers folder with the following code:

import { Action } from '@ngrx/store';
import * as speech from '../actions/speech';
export const MODE_CHANGE = 'MODE_CHANGE';

export interface State {
   speechStatus: boolean;
}
export const initialState: State = {
   speechStatus: false
};
export function reducer(state: State = initialState,
   action: speech.Actions): State {
        switch (action.type) {
            case speech.ActionTypes.MODE_CHANGE: {
                const response = action.payload;
                return Object.assign({}, state,
                {speechStatus: response});
       }
       default: {
           return state;
       }
   }
}
export const getspeechStatus = (state: State) =>
    state.speechStatus;

 

It follows the format of reducer functions created in Loklak Search. Here, the main key point is the state creation and type of value it is storing i.e. State is containing a speechStatus of type boolean. Defining an initial state with speechStatus value false (Considering initially the Speech Component will not be in use). The reducer function a new state by toggling the input state based on the type of Action created above and it returns the input state by default. At last wrapping the state as a function and returning the state’s speechStatus value.

Third and last step in this section would be to create a selector for the above reducer function in the root reducer index file.

Import and add speech from speech reducer file into the general state in root reducer file. And at last export the created selector function for speech reducer.

import * as fromSpeech from './speech';
export interface State {
   ...
   speech: fromSpeech.State;
}
export const getSpeechState = (state: State) =>
    state.speech;
export const getspeechStatus = createSelector(
    getSpeechState, fromSpeech.getspeechStatus);

Creating Speech Component

Now comes the main part to create and define the functioning of Speech Component. For creating the basic Speech Component, following command is used:

ng generate component app/speech --module=app

 

It will automatically create and provide Speech Component in app.module.ts. The working structure of Speech Component has been followed as of Google’s voice recognition feature for voice searching. Rather than providing description of each single line of code the following portion will cover the main code responsible for the functioning of Speech Component.

Importing and defining speech service in constructor:

import {
    SpeechService
} from '../services/speech.service';
constructor(
       private speech: SpeechService,
       private store: Store<fromRoot.State>,
       private router: Router
   ) {
       this.resultspage =this.router.url
       .toString().includes('/search');
       if (this.resultspage) {
           this.shadowleft = '-103px';
           this.shadowtop = '-102px';
       }
       this.speechRecognition();
}
speechRecognition() {
    this.speech.record('en_US').subscribe(voice =>
        this.onquery(voice));
}

 

When the Speech Component is called, speechRecognition() method will start recording speech (It will use the record() method from speech service to record the user voice).

For fluctuating border height and color of voice search icon, a resettimer() method is created.

randomize(min, max) {
    let x;
    x = (Math.random() * (max - min) + min);
    return x;
}
resettimer(recheck: boolean = false) {
    this.subscription.unsubscribe();
    this.timer = Observable.timer(0, 100);
    this.subscription = this.timer.subscribe(t => {
    this.ticks = t;
        if (t % 10 === 0 && t <= 20) {
            this.buttoncolor = '#f44';
            this.miccolor = '#fff';
            this.borderheight =
                this.randomize(0.7, 1);
            if (this.resultspage) {
                this.borderheight =
                    this.randomize(0.35, 0.5);
            }
            if (!recheck) {
                this.resettimer(true);
            }
        }
        if (t === 20) {
            this.borderheight = 0;
        }
        if (t === 30) {
            this.subscription.unsubscribe();
            this.store.dispatch(new speechactions
            .SearchAction(false));
        }
    });
}

 

The randomize() method provides a random number between min and max value.

To put on check and display status as message on things like whether microphone is working, or user has spoken something, or if the speech is being recorded, based on the time elapsed in calling of speech component and actual voice recording, the following portion of code is written in ngOnInit() method.

ngOnInit() {
    this.timer = Observable.timer(1500, 2000);
    this.subscription = this.timer.subscribe(t => {
        this.ticks = t;
        if (t === 1) {
            this.message = 'Listening...';
        }
        if (t === 4) {
            this.message = 'Please check your 
            microphone and volume levels.';
            this.miccolor = '#C2C2C2';
        }
        if (t === 6) {
            this.subscription.unsubscribe();
            this.store.dispatch(new speechactions
                .SearchAction(false));
        }
    });
}

 

The logic can be understood as if the elapsed time is 1 sec, it means it is listening to the speaker’s voice. And if the elapsed time is 4 sec, it means there is something wrong and user will be asked to check for the microphone and volume levels. At last if it tends to 6 seconds, then the Speech Component will be called off with the dispatched Action as false which is defined above (That means it is no longer in use).

Embed Speech Component in main App Component

Now comes the last part to use the created featured component in the required place. Code below describes embedding Speech Component in App Component.

Import SpeechService and required modules.

import {
    SpeechService
} from './services/speech.service';
import { Observable } from 'rxjs/Observable';

 

hidespeech will be used to store the current status of Speech Component (whether its in use or not), and completeQuery$ and searchData store the voice recorded in form Observable and String. completeQuery$ is optional (If the Speech Component is unable to track voice of speaker by any means, then it will not contain any value and hence searchData will be empty).

hidespeech: Observable<any>;
completeQuery$: Observable<any>;
searchData: String;

 

Creating speech parameter in constructor and store the current status of speech and store it into hidespeech. Based on the subscribed value of hidespeech, speech service’s stoprecord() will be called (To stop recording when the speech recognition completes). After recording stops, store the whole query in completeQuery$.

constructor (
    private speech: SpeechService
) {
    this.hidespeech = store.select(
        fromRoot.getspeechStatus);
    this.hidespeech.subscribe(hidespeech => {
        if (!hidespeech) {
            this.speech.stoprecord();
        }
    });
    this.completeQuery$ = store.select(
        fromRoot.getQuery);
    this.completeQuery$.subscribe(data => {
        this.searchData = data;
    });
}

 

Add the Speech Component in app.component.html. Now the main logic of calling Speech Component will be based on the subscribed observable value of hidespeech (If false then call Speech Component else not).

<app-speech *ngIf="hidespeech|async"></app-speech>

Using Speech Component in Home and FeedHeader Component

Import Speech Service and speech Action created above, and create hidespeech to store the current status of Speech Component.

import * as speechactions from '../../actions/speech';
import {
    SpeechService
} from '../../services/speech.service';
hidespeech: Observable<boolean>;

 

Create speech parameter of type SpeechService and store the current status of Speech Component in hidespeech. Dispatch speechactions.SearchAction (payload as true) for inferring that the Speech Component is currently in use.

constructor(
    private speech: SpeechService
) {
    this.hidespeech = store
        .select(fromRoot.getspeechStatus);
}
speechRecognition() {
    this.store.dispatch(
        new speechactions.SearchAction(true));
}

How to use the Speech Component?

Goto Loklak and click on Voice Input Icon. It will popup a screen as below.

Now, speak something to search. E.g. Google, the screen will turn into something like below with the spelled value displayed on screen.

If something goes wrong (Microphone did not work, low volume levels or unrecognisable voice), then screen will show something like:

On successful recognition of speech, the query will be set and the results will be shown as

Similar process is being followed on results page to make a search query using voice.

Resources

Continue ReadingAdding Speech Component in Loklak Search

Adding ‘Voice Search’ Feature in Loklak Search

It is beneficial to have a voice search feature embedded in a search engine like loklak.org based on PWA (Progressive Web Application) architecture. This will allow users to make query using voice on phone or computer. For integrating voice search, JavaScript Web Speech API (also known as webkitSpeechRecognition) is used which allows us to add speech recognition feature in any website or a web application.

Integration Process

For using webkitSpeechRecognition, a separate typescript service is being created along with its relevant unit test

  1. speech.service.ts
  2. speech.service.spec.ts

The main idea to implement the voice search was

  1. To create an injectable speech service.
  2. Write methods for starting and stopping voice record in it.
  3. Import and inject the created speech service into root app module.
  4. Use the created speech service into required HomeComponent and FeedHeaderComponent.

Structure of Speech Service

The speech service mainly consists of two methods

  • record() method which accepts lang as string input which specifies the language code for speech recording and returns a string Observable.

   record(lang: string): Observable<string> {
           return Observable.create(observe => {
                 const webkitSpeechRecognition }: IWindow = 
                        <IWindow>window;
                 this.recognition = new webkitSpeechRecognition();
                 this.recognition.continuous = true;
                 this.recognition.interimResults = true;
                 this.recognition.onresult = take => 
                 this.zone.run(()                            
                 observe.next(take
                 .results.item
                 (take.results.
                 length - 1).
                 item(0).
                 transcript);
                 this.recognition
                 .onerror = 
                 err => observe
                 .error(err);
                 this.recognition.onend 
                 = () => observe
                 .complete();
                 this.recognition.lang = 
                 lang;
                 this.recognition.start( 
             );
        });
     }

 

Using observe.complete() allows speech recognition action to stop when the user stops speaking.

  • stoprecord() method which is used to stop the current instance of recording.

 stoprecord() {
       if (this.recognition) {
           this.recognition.stop();
       }
   }

 

stop() method is used to stop the current instance of speech recognition.

Using Speech Service in required Component

In Loklak Search, the speech service have been included in two main components i.e. HomeComponent and FeedHeaderComponent. The basic idea of using the created speech service in these components is same.

In the TypeScript (not spec.ts) file of the two components, firstly import the SpeechService and create its object in constructor.

constructor( private speech: SpeechService ) { }

 

Secondly, define a speechRecognition() method and use the created instance or object of SpeechService in it to record speech and set the query as the recorded speech input. Here, default language has been set up as ‘en_US’ i.e. English.

 stoprecord() {
       if (this.recognition) {
           this.recognition.stop();
       }
   }

 

After user clicks on speech icon, this method will be called and the recorded speech will be set as the query and there will be router navigation to the /search page where the result will be displayed.

Resources

Continue ReadingAdding ‘Voice Search’ Feature in Loklak Search

Autolinker Component in Loklak Search

In Loklak Search the post items contain links, which are either internal or external. These links include the hashtags, mentions, and URLs. From the backend server we just received the message in the plain text format, and thus there is need to parse the plain text and render it as clickable links. These clickable links can be either internal or external. Thus we need an auto-linker component, which takes the text and render it as links.

The API of the Component

The component takes as the input the plain text, then four arrays of strings. Each containing the text to be linked. These are hashtags, mentions, links and the unshorten attribute which is used to unshorten the shortened URLs in the post. These attributes are used by the component to render the text in the appropriate format.

export class FeedLinkerComponent implements OnInit {
@Input() text: string;
@Input() hashtags: string[] = new Array<string>();
@Input() mentions: string[] = new Array<string>();
@Input() links: string[] = new Array<string>();
@Input() unshorten: Object = {};
}

The Logic of the Component

The basic logic of the component works as the following, we divide the text into chunks known as shards, we have three basic data structures for the component to work

  • The ShardType which is the type of the chunk it specifies whether it is plain, hashtags, mentions, and links.
  • The Shard which is the simple object containing the text to show, its type and the link it refers to

The StringIndexdChunks, they are utilized to index the chunks in the order in which they appear in the text.

const enum ShardType {
plain, // 0
link, // 1
hashtag, // 2
mention // 3
}

class Shard {
constructor (
public type: ShardType = ShardType.plain,
public text: String = '',
public linkTo: any = null,
public queryParams: any = null
) { }
}

interface StringIndexedChunks {
index: number;
str: string;
type: ShardType;
}

First we have a private method of the component which searches for all the elements (strings) in the text. Here we have an array which maintains the index of those chunks in the text.

private generateShards() {
const indexedChunks: StringIndexedChunks[] = [];

this.hashtags.forEach(hashtag => {
const indices = getIndicesOf(this.text, `#${hashtag}`, false);
indices.forEach(idx => {
indexedChunks.push({index: idx, str: `#${hashtag}`, type: ShardType.hashtag});
});
});

this.mentions.forEach(mention => {
const indices = getIndicesOf(this.text, `@${mention}`, false);
indices.forEach(idx => {
indexedChunks.push({index: idx, str: `@${mention}`, type: ShardType.mention});
});
});
}

Then we sort the chunks according to their indexes in the text. This gives us sorted array which consists of all the chunks sorted according to the indexes as they appear in the text.

indexedChunks.sort((a, b) => { return (a.index > b.index) ? 1 : (a.index < b.index) ? -1 : 0; });

The next part of the logic is to generate the shard array, an array which contains each chunk, once. To do this we iterate over the Sorted Indexed array created in the previous step and use it split the text into chunks. We iterate over the text and take substrings using the indexes of each element.

let startIndex = 0;
const endIndex = this.text.length;

indexedChunks.forEach(element => {
if (startIndex !== element.index) {
const shard = new Shard(ShardType.plain, this.text.substring(startIndex, element.index));
this.shardArray.push(shard);
startIndex = element.index;
}
if (startIndex === element.index) {
const str = this.text.substring(startIndex, element.index + element.str.length);
const shard = new Shard(element.type, str);
switch (element.type) {
case ShardType.link: {
if (this.unshorten[element.str]) {
shard.linkTo = str;
shard.text = this.unshorten[element.str];
}
else {
shard.linkTo = str;
}
break;
}

case ShardType.hashtag: {
shard.linkTo = ['/search'];
shard.queryParams = { query : str };
break;
}

case ShardType.mention: {
shard.linkTo = ['/search'];
shard.queryParams = { query : `from:${str.substring(1)}` };
break;
}
}
this.shardArray.push(shard);
startIndex += element.str.length;
}
});

if (startIndex !== endIndex) {
const shard = new Shard(ShardType.plain, this.text.substring(startIndex));
this.shardArray.push(shard);
}

After this we have generated the chunks of the text, now the only task is to write the view of the component which uses this Shard Array to render the linked elements.

<div class="textWrapper">
<span *ngFor="let shard of shardArray">
<span *ngIf="shard.type === 0"> <!-- Plain -->
{{shard.text}}
</span>
<span *ngIf="shard.type === 1"> <!-- URL Links -->
<a>{{shard.text}}</a>
</span>
<span *ngIf="shard.type === 2"> <!-- Hashtag -->
<a [routerLink]="shard.linkTo" [queryParams]="shard.queryParams">{{shard.text}}</a>
</span>
<span *ngIf="shard.type === 3"> <!-- Mention -->
<a [routerLink]="shard.linkTo" [queryParams]="shard.queryParams">{{shard.text}}</a>
</span>
</span>
</div>
  • This renders the chunks and handles the links of both internal and external type.
  • It also also makes sure that the links get unshortened properly using the unshorten API property.
  • Uses routerLink, angular property to link in application URLs, for asynchronous reloading while clicking links.

Resources and Links

This component is inspired from the two main open source libraries.

Earlier these libraries were used in the project, but as the need of unshortening and asynchronous linking appeared in the application, a custom implementation was needed to be implemented.

Continue ReadingAutolinker Component in Loklak Search

Service Workers in Loklak Search

Loklak search is a web application which is built on latest web technologies and is aiming to be a progressive web application. A PWA is a web application which has a rich, reliable, fast, and engaging web experience, and web API which enables us to get these are Service Workers. This blog post describes the basics of service workers and their usage in the Loklak Search application to act as a Network Proxy to and the programmatical cache controller for static resources.

What are Service Workers?

In the very formal definition, Matt Gaunt describes service workers to be a script that the browser runs in the background, and help us enable all the modern web features. Most these features include intercepting network requests and caching and responding from the cache in a more programmatical way, and independent from native browser based caching. To register a service worker in the application is a really simple task, there is just one thing which should be kept in mind, that service workers need the HTTPS connection, to work, and this is the web standard made around the secure protocol. To register a service worker

if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js').then(function(registration) {
// Registration was successful
console.log('ServiceWorker registration successful with scope: ', registration.scope);
}, function(err) {
// registration failed :(
console.log('ServiceWorker registration failed: ', err);
});
});
}

This piece of javascript, if the browser supports, registers the service worker defined by sw.js. The service worker then goes through its lifecycle, and gets installed and then it takes control of the page it gets registered with.

What does service workers solve in Loklak Search?

In loklak search, service workers currently work as a, network proxy to work as a caching mechanism for static resources. These static resources include the all the bundled js files and images. These bundled chunks are cached in the service workers cache and are responded with from the cache when requested. The chunking of assets have an advantage in this caching strategy, as the cache misses only happen for the chunks which are modified, and the parts of the application which are unmodified are served from the cache making it possible for lesser download of assets to be served.

Service workers and Angular

As the loklak search is an angular application we, have used the @angular/service-worker library to implement the service workers. This is simple to integrate library and works with the, CLI, there are two steps to enable this, first is to download the Service Worker package

npm install --save @angular/service-worker

And the second step is to enable the service worker flag in .angular-cli.json

"apps": [
   {
      // Other Configurations
      serviceWorker: true
   }
]

Now when we generate the production build from the CLI, along with all the application chunks we get, The three files related to the service workers as well

  • sw-register.bundle.js : This is a simple register script which is included in the index page to register the service worker.
  • worker-basic.js : This is the main service worker logic, which handles all the caching strategies.
  • ngsw-manifest.json : This is a simple manifest which contains the all the assets to be cached along with their version hashes for cache busting.

Future enhancements in Loklak Search with Service Workers

The service workers are fresh in loklak search and are currently just used for caching the static resources. We will be using service workers for more sophisticated caching strategies like

  • Dynamically caching the results and resources received from the API
  • Using IndexedDB interface with service workers for storing the API response in a structured manner.
  • Using service workers, and app manifest to provide the app like experience to the user.

 

Resources and Links

Continue ReadingService Workers in Loklak Search

Analyzing Production Build Size in Loklak Search

Loklak search being a web application it is critical to keep the size of the application in check to ensure that we are not transferring any non-essential bytes to the user so that application load is faster, and we are able to get the minimal first paint time. This requires a mechanism for the ability to check the size of the build files which are generated and served to the user. Alongside the ability to check sizes it is also critically important to analyze the distribution of the modules along with their sizes in various chunks. In this blog post, I discuss the analysis of the application code of loklak search and the generated build files.

Importance of Analysis

The chunk size analysis is critical to any application, as the chunk size of any application directly determines the performance of any application, at any scale. The smaller the application the lesser is the load time, thus faster it becomes usable at the user side. The time to first-paint is the most important metric to keep in mind while analyzing any web application for performance, though the first paint time consists of many critical parts from loading, parsing, layout and paint, but still the size of any chunk determines all the time it will take to render it on the screen.

Also as we use the 3rd party libraries and components it becomes crucially important to inspect the impact on the size of the application upon the inclusion of those libraries and components.

Development Phase Checking

Angular CLI provides a clean mechanism to track and check the size of all the chunks always at the runtime, these stats simply show the size of each chunk in the application in the terminal on every successful compilation, and this provides us a broad idea about the chunks to look and address.

Deep Analysis using Webpack Bundle Analyzer

The angular cli while generating the production build provides us with an option to generates the statistics about the chunks including the size and namespaces of the each module which is part of that chunk. These stats are directly generated by the webpack at the time of bundling, code splitting, and tree shaking. These statistics thus provide us to peek into the actual deeper level of chunk creation in webpack to analyze sizes of its various components. To generate the statistics we just need to enable the –stats-json flag while building.

ng serve --prod --aot --stats-json

This will generate the statistics file for the application in the /dist directory, alongside all the bundles. Now to have the visual and graphical analysis of these statistics we can use a tool like webpack-bundle-analyzer to analyze the statistics. We can install the webpack-bundle-analyzer via npm,

npm install --save-dev webpack-bundle-analyzer

Now, to our package.json we can add a script, running this script will open up a web page which contains graphical visualization of all the chunks build in the application

// package.json

{
   …
   …
   {
      “scripts”: {
         …
         …
         "analyze": "webpack-bundle-analyzer dist/stats.json"
      }
   }
}

These block diagrams also contain the information about the sub modules contained in each chunk, and thus we can easily analyze and compare the size of each component we add in the application.

Now, we can see in the above distribution, the main.bundle is of the largest size among all the other chunks. And the major part of it is being occupied by, moment.js, this analysis provides us with a deeper insight into the impact of a module like moment.js on the application size. This helps us to reason about the analyze which part of the application is worth, and which parts of the application can be replaced with lighter alternatives and which parts of the application are worth the size they are consuming, as for a 3rd party module which consumes a lot of sizes but is used in some insignificant feature, must be replaced with a lightweight alternative.

Conclusion

Thus being able to see the description of modules in each and every chunk provides us with a method to reason about, and compare the alternative approaches for a particular solution to a problem, in terms of the effect of those approaches on the size of the application so we are able to make the best decision.

Resources and Links

  • Analyzing the builds blog by hackernoon
  • Bundle analysis for webpack applications blog by Nimesh
Continue ReadingAnalyzing Production Build Size in Loklak Search

Using CSS Grid in Loklak Search

CSS Grid is the latest web standard for the layouts in the web applications. This is the web standard which allows the HTML page to be viewed as 2-dimensional for laying out the elements in the page. It is thus used in parts of loklak search for layout. In this blog post, I will discuss the basic naming convention for CSS grid and its usage in Loklak Search for layout structuring and responsiveness.

CSS Grid Basics

There are some basic terminologies regarding grid few major ones are the following

Grid Container

The grid container is the container which is the wrapper of all the grid items. It is declared by display: grid, this makes all the direct children of that element to become grid items.

Grid Tracks

We define rows and columns of the grid as the lines, the area between any two lines is called a grid track. Tracks can be defined using any length unit. Grid also introduces an additional length unit to help us create flexible grid tracks. The new fr unit represents a fraction of the available space in the grid container.

Grid Cells

The area between any two horizontal and vertical lines is called a grid cell.

Grid Area

The area formed by the combination of two or more cells is called a grid area.

Using CSS grid in Loklak Search

The CSS grid is used in loklak search uses CSS grid in the feeds page to align elements in a responsive way on mobile and desktop. Earlier there was the issue that on small displays the info box of the results appeared after the feed results, and we needed to make sure that it appears on top on smaller displays. This is the outline of the structure of the feed page.

<div class=”feed-wrapper”>
<div class=”feed-results”>
<!-- Feed Results -->
</div>

<div class=”feed-info-box”>
<!-- Feed Info Box -->
</div>
</div>

Now we use the CSS grid to position the items according to the display width. First we declare the “feed-wrapper” as display:grid to make it a Grid Container, and we associate the rows and columns accordingly.

.feed-wrapper {
   display: grid;
   grid-template-columns: 150px 632px 455px 1fr;
   grid-template-rows: auto;
}

This defines the grid to be consisting of 4 columns of width 150px, 632px,  455px and one remaining unit i.e. 1fr. The rows are set to be auto.

Now we define the grid areas i.e. the names of the areas using the grid-area:<area> css property. This gives names to the elements in the CSS grid.

.feed-results {
   grid-area: feed-results;
}

.feed-info-box {
   grid-area: feed-info-box;
}

The last thing which remains now is to specify the position of these grid elements in the grid cells according to the display width, we use simple media queries along with simple grid area positioning property, i.e. grid-template-areas.

.feed-wrapper {
   /* Other Properties */
   @media(min-width: 1200px) {
      grid-template-areas: ". feed-results feed-info-box .";
   }

   @media(max-width: 1199px) {
      grid-template-columns: 1fr;
      grid-template-areas:
         "feed-info-box"
         "feed-results";
   }
}

This positions both the boxes according to the display width, in one column for large displays, and info box on top of results on mobile displays.

This is how it looks on the large desktop displays

 

This is how it looks on small mobile displays

Links and References

 

 

Continue ReadingUsing CSS Grid in Loklak Search

Reactive Side Effects of Actions in Loklak Search

In a Redux based application, every component of the application is state driven. Redux based applications manage state in a predictable way, using a centralized Store, and Reducers to manipulate various aspects of the state. Each reducer controls a specific part of the state and this allows us to write the code which is testable, and state is shared between the components in a stable way, ie. there are no undesired mutations to the state from any components. This undesired mutation of the shared state is prevented by using a set of predefined functions called reducers which are central to the system and updates the state in a predictable way.

These reducers to update the state require some sort triggers to run. This blog post concentrates on these triggers, and how in turn these triggers get chained to form a Reactive Chaining of events which occur in a predictable way, and how this technique is used in latest application structure of Loklak Search. In any state based asynchronous application, like, Loklak Search the main issue with state management is to handle the asynchronous action streams in a predictable manner and to chain asynchronous events one after the other.  The technique of reactive action chaining solves the problem of dealing with asynchronous data streams in a predictable and manageable manner.

Overview

Actions are the triggers for the reducers, each redux action consists of a type and an optional payload. Type of the action is like its ID which should be purposely unique in the application. Each reducer function takes the current state which it controls and action which is dispatched. The reducer decides whether it needs to react to that action or not. If the user reacts to the action, it modifies the state according to the action payload and returns the modified state, else, it returns the original state. So at the core, the actions are like the triggers in the application, which make one or more reducers to work. This is the basic architecture of any redux application. The actions are the triggers and reducers are the state maintainers and modifiers. The only way to modify the state is via a reducer, and a reducer only runs when a corresponding action is dispatched.

Now, who dispatches these actions? This question is very important. The Actions can be technically dispatched from anywhere in the application, from components, from services, from directives, from pipes etc. But we almost in every situation will always want the action to be dispatched by the component. Component who wishes to modify the state dispatch the corresponding actions.

Reactive Effects

If the components are the one who dispatch the action, which triggers a reducer function which modifies the state, then what are these effects, cause the cycle of events seem pretty much complete. The Effects are the Side Effects, of a particular action. The term “side effect” means these are the piece of code which runs whenever an action is dispatched. Don’t confuse them with the reducer functions, effects are not same as the reducer functions as they are not associated with any state i.e. they don’t modify any state. They are the independent sources of other actions. What this means is whenever an Action is dispatched, and we want to dispatch some other action, maybe immediately or asynchronously, we use these side effects. So in a nutshell, effects are the piece of code which reacts to a particular action, and eventually dispatches some other actions.

The most common use case of effects is to call a corresponding service and fetch the data from the server, and then when the data is loaded, dispatch a SearchCompleteAction. These are the simplest of use cases of effects and are most commonly use in Loklak Search. This piece of code below shows how it is done.

@Effect()
search$: Observable<Action>
= this.actions$
.ofType(apiAction.ActionTypes.SEARCH)
.map((action: apiAction.SearchAction) => action.payload)
.switchMap(query => {
return this.apiSearchService.fetchQuery(query)
.map(response => new apiAction.SearchCompleteSuccessAction(response))

This is a very simple type of an effect, it filters out all the actions and react to only the type of action which we are interested in, here SEARCH, and then after calling the respective Service, it either dispatches SearchComplete or a SearchFail action depending on the status of the response from the API. The effect runs on SEARCH action and eventually dispatches, the success or the fail actions.

This scheme illustrates the effect as another point apart from components, to dispatch some action. The difference being, components dispatch action on the basis of the User inputs and events, whereas Effects dispatch actions on the basis of other actions.

Reactive Chaining of Actions

We can thus take the advantage of this approach in a form of reactive chaining of actions. This reactive chaining of actions means that some component dispatches some action, which as a side effects, dispatches some other action, and it dispatches another set of actions and so on. This means a single action dispatched from a component, brings about the series of actions which follow one another. This approach makes it possible to write reducers at the granular level rather than complete state level. As a series of actions can be set up which, start from a fine grain, and reaches out to a coarse grain. The loklak search application uses this technique to update the state of query. The reducers in the loklak search rather than updating the whole query structure update only the required part of the state. This helps in code maintainability as the one type of query attribute has no effect on the other type

@Effect()
inputChange$: Observable<Action>
= this.actions$
.ofType(queryAction.ActionTypes.VALUE_CHANGE)
.map(_ => new queryAction.QueryChangeAction(''));

@Effect()
filterChange$: Observable<Action>
= this.actions$
.ofType(queryAction.ActionTypes.FILTER_CHANGE)
.map(_ => new queryAction.QueryChangeAction(''));

Here the QUERY_CHANGE action further can do other processing of the query and then dispatch the SearchAction, which eventually calls the service and then return the response, then the success or fail actions can be dispatched eventually.

Conclusion

The reactive side effects is one of the most beautiful thing we can do with Redux and Reactive Programming. They provide an easy clean way to chain events in an application, which helps in a cleaner non-overlapping state management along with clean and simple reducers. This idea of the reactive chaining can be extended to any level of sophistication, and that too in a simple and easy to understand manner.

Resources and links

Continue ReadingReactive Side Effects of Actions in Loklak Search

Route Based Chunking in Loklak Search

The loklak search application running at loklak.org is growing in size as the features are being added into the application, this growth is a linear one, and traditional SPA, tend to ship all the code is required to run the application in one pass, as a single monolithic JavaScript file, along with the index.html. This approach is suitable for the application with few pages which are frequently used, and have context switching between those logical pages at a high rate and almost simultaneously as the application loads.

But generally, only a fraction of code is what is accessed most frequently by most users, so as the application size grows it does not make sense to include all the code for the entire application at the first request, as there is always some code in the application, some views, are rarely accessed. The loading of such part of the application can be delayed until they are accessed in the application. The angular router provides an easy way to set up such system and is used in the latest version of loklak search.

The technique which is used here is to load the content according to the route. This makes sure only the route which is viewed is loaded on the initial load, and subsequent loading is done at the runtime as and when required.

Old setup for baseline

Here are the compiled file sizes, of the project without the chunking the application. Now as we can see that the file sizes are huge, especially the vendor bundle, which is of 5.4M and main bundle which is about 0.5M now, these are the files which are loaded on the first load and due to their huge sizes, the first paint of the application suffers, to a great extent. These numbers will act as a baseline upon which we will measure the impact of route based chunking.

Setup for route based chunking

The setup for route based chunking is fairly simple, this is our routing configuration, the part of the modules which we want to lazy load are to be passed as loadChildren attribute of the route, this attribute is a string which is a path of the feature module which, and part after the hash symbol is the actual class name of the module, in that file. This setup enables the router to load that module lazily when accessed by the user.

const routes: Routes = [
{
path: '',
pathMatch: 'full',
loadChildren: './home/home.module#HomeModule',
data: { preload: true }

},
{
path: 'about',
loadChildren: './about/about.module#AboutModule'
},

{
path: 'contact',
loadChildren: './contact/contact.module#ContactModule'
},

{
path: 'search',
loadChildren: './feed/feed.module#FeedModule',
data: { preload: true }
},
{
path: 'terms',

loadChildren: './terms/terms.module#TermsModule'
},
{
path: 'wall',
loadChildren: './media-wall/media-wall.module#MediaWallModule'
}
];

Preloading of some routes

As we can see that in two of the configurations above, there is a data attribute, on which preload: true attribute is specified. Sometimes we need to preload some part of theapplication, which we know we will access, soon enough. So angular also enables us to set up our own preloading strategy to preload some critical parts of the application, which we know are going to be accessed. In our case, Home and Feed module are the core parts of the application, and we can be sure that, if someone has to use our application, these two modules need to be loaded. Defining the preloading strategy is also really simple, it is a class which implements PreloadingStrategy interface, and have a preload method, this method receives the route and load function as an argument, and this preload method either returns the load() observable or null if preload is set to true.

export class CustomPreloadStrategy implements PreloadingStrategy {
preload(route: Route, load: Function): Observable<any> {
return route.data && route.data.preload ? load() : of(null);
}
}

Results of route based chunking

The results of route based chunking are the 50% reduction in the file size of vendor bundle and 70% reduction in the file size of the main bundle this provides the edge which every application needs to perform well at the load time, as unnecessary bytes are not at all loaded until required.

Resources

Continue ReadingRoute Based Chunking in Loklak Search