Service Workers in Loklak Search

Loklak search is a web application which is built on latest web technologies and is aiming to be a progressive web application. A PWA is a web application which has a rich, reliable, fast, and engaging web experience, and web API which enables us to get these are Service Workers. This blog post describes the basics of service workers and their usage in the Loklak Search application to act as a Network Proxy to and the programmatical cache controller for static resources.

What are Service Workers?

In the very formal definition, Matt Gaunt describes service workers to be a script that the browser runs in the background, and help us enable all the modern web features. Most these features include intercepting network requests and caching and responding from the cache in a more programmatical way, and independent from native browser based caching. To register a service worker in the application is a really simple task, there is just one thing which should be kept in mind, that service workers need the HTTPS connection, to work, and this is the web standard made around the secure protocol. To register a service worker

if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js').then(function(registration) {
// Registration was successful
console.log('ServiceWorker registration successful with scope: ', registration.scope);
}, function(err) {
// registration failed 🙁
console.log('ServiceWorker registration failed: ', err);

This piece of javascript, if the browser supports, registers the service worker defined by sw.js. The service worker then goes through its lifecycle, and gets installed and then it takes control of the page it gets registered with.

What does service workers solve in Loklak Search?

In loklak search, service workers currently work as a, network proxy to work as a caching mechanism for static resources. These static resources include the all the bundled js files and images. These bundled chunks are cached in the service workers cache and are responded with from the cache when requested. The chunking of assets have an advantage in this caching strategy, as the cache misses only happen for the chunks which are modified, and the parts of the application which are unmodified are served from the cache making it possible for lesser download of assets to be served.

Service workers and Angular

As the loklak search is an angular application we, have used the @angular/service-worker library to implement the service workers. This is simple to integrate library and works with the, CLI, there are two steps to enable this, first is to download the Service Worker package

npm install --save @angular/service-worker

And the second step is to enable the service worker flag in .angular-cli.json

"apps": [
      // Other Configurations
      serviceWorker: true

Now when we generate the production build from the CLI, along with all the application chunks we get, The three files related to the service workers as well

  • sw-register.bundle.js : This is a simple register script which is included in the index page to register the service worker.
  • worker-basic.js : This is the main service worker logic, which handles all the caching strategies.
  • ngsw-manifest.json : This is a simple manifest which contains the all the assets to be cached along with their version hashes for cache busting.

Future enhancements in Loklak Search with Service Workers

The service workers are fresh in loklak search and are currently just used for caching the static resources. We will be using service workers for more sophisticated caching strategies like

  • Dynamically caching the results and resources received from the API
  • Using IndexedDB interface with service workers for storing the API response in a structured manner.
  • Using service workers, and app manifest to provide the app like experience to the user.


Resources and Links

Analyzing Production Build Size in Loklak Search

Loklak search being a web application it is critical to keep the size of the application in check to ensure that we are not transferring any non-essential bytes to the user so that application load is faster, and we are able to get the minimal first paint time. This requires a mechanism for the ability to check the size of the build files which are generated and served to the user. Alongside the ability to check sizes it is also critically important to analyze the distribution of the modules along with their sizes in various chunks. In this blog post, I discuss the analysis of the application code of loklak search and the generated build files.

Importance of Analysis

The chunk size analysis is critical to any application, as the chunk size of any application directly determines the performance of any application, at any scale. The smaller the application the lesser is the load time, thus faster it becomes usable at the user side. The time to first-paint is the most important metric to keep in mind while analyzing any web application for performance, though the first paint time consists of many critical parts from loading, parsing, layout and paint, but still the size of any chunk determines all the time it will take to render it on the screen.

Also as we use the 3rd party libraries and components it becomes crucially important to inspect the impact on the size of the application upon the inclusion of those libraries and components.

Development Phase Checking

Angular CLI provides a clean mechanism to track and check the size of all the chunks always at the runtime, these stats simply show the size of each chunk in the application in the terminal on every successful compilation, and this provides us a broad idea about the chunks to look and address.

Deep Analysis using Webpack Bundle Analyzer

The angular cli while generating the production build provides us with an option to generates the statistics about the chunks including the size and namespaces of the each module which is part of that chunk. These stats are directly generated by the webpack at the time of bundling, code splitting, and tree shaking. These statistics thus provide us to peek into the actual deeper level of chunk creation in webpack to analyze sizes of its various components. To generate the statistics we just need to enable the –stats-json flag while building.

ng serve --prod --aot --stats-json

This will generate the statistics file for the application in the /dist directory, alongside all the bundles. Now to have the visual and graphical analysis of these statistics we can use a tool like webpack-bundle-analyzer to analyze the statistics. We can install the webpack-bundle-analyzer via npm,

npm install --save-dev webpack-bundle-analyzer

Now, to our package.json we can add a script, running this script will open up a web page which contains graphical visualization of all the chunks build in the application

// package.json

      “scripts”: {
         "analyze": "webpack-bundle-analyzer dist/stats.json"

These block diagrams also contain the information about the sub modules contained in each chunk, and thus we can easily analyze and compare the size of each component we add in the application.

Now, we can see in the above distribution, the main.bundle is of the largest size among all the other chunks. And the major part of it is being occupied by, moment.js, this analysis provides us with a deeper insight into the impact of a module like moment.js on the application size. This helps us to reason about the analyze which part of the application is worth, and which parts of the application can be replaced with lighter alternatives and which parts of the application are worth the size they are consuming, as for a 3rd party module which consumes a lot of sizes but is used in some insignificant feature, must be replaced with a lightweight alternative.


Thus being able to see the description of modules in each and every chunk provides us with a method to reason about, and compare the alternative approaches for a particular solution to a problem, in terms of the effect of those approaches on the size of the application so we are able to make the best decision.

Resources and Links

  • Analyzing the builds blog by hackernoon
  • Bundle analysis for webpack applications blog by Nimesh

Route Based Chunking in Loklak Search

The loklak search application running at is growing in size as the features are being added into the application, this growth is a linear one, and traditional SPA, tend to ship all the code is required to run the application in one pass, as a single monolithic JavaScript file, along with the index.html. This approach is suitable for the application with few pages which are frequently used, and have context switching between those logical pages at a high rate and almost simultaneously as the application loads.

But generally, only a fraction of code is what is accessed most frequently by most users, so as the application size grows it does not make sense to include all the code for the entire application at the first request, as there is always some code in the application, some views, are rarely accessed. The loading of such part of the application can be delayed until they are accessed in the application. The angular router provides an easy way to set up such system and is used in the latest version of loklak search.

The technique which is used here is to load the content according to the route. This makes sure only the route which is viewed is loaded on the initial load, and subsequent loading is done at the runtime as and when required.

Old setup for baseline

Here are the compiled file sizes, of the project without the chunking the application. Now as we can see that the file sizes are huge, especially the vendor bundle, which is of 5.4M and main bundle which is about 0.5M now, these are the files which are loaded on the first load and due to their huge sizes, the first paint of the application suffers, to a great extent. These numbers will act as a baseline upon which we will measure the impact of route based chunking.

Setup for route based chunking

The setup for route based chunking is fairly simple, this is our routing configuration, the part of the modules which we want to lazy load are to be passed as loadChildren attribute of the route, this attribute is a string which is a path of the feature module which, and part after the hash symbol is the actual class name of the module, in that file. This setup enables the router to load that module lazily when accessed by the user.

const routes: Routes = [
path: '',
pathMatch: 'full',
loadChildren: './home/home.module#HomeModule',
data: { preload: true }

path: 'about',
loadChildren: './about/about.module#AboutModule'

path: 'contact',
loadChildren: './contact/contact.module#ContactModule'

path: 'search',
loadChildren: './feed/feed.module#FeedModule',
data: { preload: true }
path: 'terms',

loadChildren: './terms/terms.module#TermsModule'
path: 'wall',
loadChildren: './media-wall/media-wall.module#MediaWallModule'

Preloading of some routes

As we can see that in two of the configurations above, there is a data attribute, on which preload: true attribute is specified. Sometimes we need to preload some part of theapplication, which we know we will access, soon enough. So angular also enables us to set up our own preloading strategy to preload some critical parts of the application, which we know are going to be accessed. In our case, Home and Feed module are the core parts of the application, and we can be sure that, if someone has to use our application, these two modules need to be loaded. Defining the preloading strategy is also really simple, it is a class which implements PreloadingStrategy interface, and have a preload method, this method receives the route and load function as an argument, and this preload method either returns the load() observable or null if preload is set to true.

export class CustomPreloadStrategy implements PreloadingStrategy {
preload(route: Route, load: Function): Observable<any> {
return && ? load() : of(null);

Results of route based chunking

The results of route based chunking are the 50% reduction in the file size of vendor bundle and 70% reduction in the file size of the main bundle this provides the edge which every application needs to perform well at the load time, as unnecessary bytes are not at all loaded until required.


Lazy Loading Images in Loklak Search

In last blog post, I discussed the basic Web API’s which helps us to create the lazy image loader component. I also discussed the structure which is used in the application, to load the images lazily. The core idea is to wrap the <img> element in a wrapper, <app-lazy-img> element. This enables us the detection of the element in the viewport and corresponding loading only if the image is present in the viewport.

In this blog post, I will be discussing the implementation details about how this is achieved in Loklak search in an optimized manner.

The logic for lazy loading of images in the application is divided into a Component and a corresponding Service. The reason for this splitting of logic will be explained as we discuss the core parts of the code for this feature.

Detecting the Intersection with Viewport

The lazy image service is a service for the lazy image component which is registered at the by the modules which intend to use this app lazy image component. The task of this service is to register the elements with the intersection observer, and, then emit an event when the element comes in the viewport, which the element can react on and then use the other methods of services to actually fetch the image.

export class LazyImgService {
private intersectionObserver: IntersectionObserver
= new IntersectionObserver(this.observerCallback.bind(this), { rootMargin: '50% 50%' });
private elementSubscriberMap: Map<Element, Subscriber<boolean>>
= new Map<Element, Subscriber<boolean>>();

The service has two member attributes, one is IntersectionObserver, and the other is a Map which stores the the reference of the subscribers of this intersection observer. This reference is then later used to emit the event when the element comes in viewport. The rootMargin of the intersection observer is set to 50% this makes sure that when the element is 50% away from the viewport.

The obvserve public method of the service, takes an element and pass it to intersection observer to observe, also put the element in the subscriber map.

public observe(element: Element): Observable<boolean> {
const observable: Observable<boolean> = new Observable<boolean>(subscriber => {
this.elementSubscriberMap.set(element, subscriber);
return observable;

Then there is the observer callback, this method, as an argument receives all the objects intersecting the root of the observer, when this callback is fired, we find all the intersecting elements and emit the intersection event. Indicating that the element is nearby the viewport and this is the time to load it.

private observerCallback(entries: IntersectionObserverEntry[], observer: IntersectionObserver) {
entries.forEach(entry => {
if (this.elementSubscriberMap.has( {
if (entry.intersectionRatio > 0) {
const subscriber = this.elementSubscriberMap.get(;;

Now, our LazyImgComponent enables us to uses this service to register its element, with the intersection observer and then reacting to it later, when the event is emitted. This method sets up the IO, to load the image, and subscribes to the event emittes by the service and eventually calls the loadImage method when the element intersects with the viewport.

private setupIntersectionObserver() {
.subscribe(value => {
if (value) {

Loading and rendering the image

Our lazy image service has another public API method fetch to fetch the image resource, this method returns an observable, which on successful fetching of image emits a Base64 image string.

public fetch(resource: string): Observable<string> {
return new Observable<string>(subscriber => {
.then(strBuffer => {;
.catch((error) => {

The intermediate promise then chain is for converting the raw response buffer to a Base64 string, this string is then emited as the observable emmision. The component then subscribes to this fetch Observable, when the load image method is called.

private loadImage() {
this.isLoading = true;
.subscribe(this.handleResponse.bind(this), this.handleError.bind(this));

The handler methods for the response and errors then contain the code to handle the effects of loading of results, ie. rendering the image inside the img element. The intresting thing to note here is, if we give the Base64 string as the src attribute of an img tag, instead of resource path then also it renders the image properly.

private handleResponse(imageStr: string) {
const base64Flag = `data:image/${this.imageType};base64,`;
this.elementRef.nativeElement.querySelector('img').src = base64Flag + imageStr;

And this completes our workflow of the app-lazy-img and gives us, a robust lazy image loader, and also is compliant with accessibility guidelines, including all the necessary attributes like, title, width, height etc. for the generation of proper accessibility tree. This technique can be extended to any level, and is more or less platform and framework independent, as this relies solely on Web Standards API’s. This is an optimized solution, as at a time only one intersection observer is active on a page and is seeing all the images, rather than per component instance based intersection observers which can be a performane bottleneck in low memory devices.

Resources and Links

  • Intersection observer API
  • Intersection Observer polyfill for the browsers which don’t support Intersection Observer
  • Fetch API documentation
  • Fetch API polyfill for the browsers which don’t support fetch.
  • Loklak Search Repo

Lazy loading images in Loklak Search

Loklak Search delivers the media rich content to the users. Most of the media delivered to the users are in the form of images. In the earlier versions of loklak search, these images were delivered to the users imperatively, irrespective of their need. What this meant is, whether the image is required by the user or not it was delivered, consuming the bandwidth and slowing down the initial load of the app as large amount of data was required to be fetched before the page was ready. Also, the 404 errors were also not being handled, thus giving the feel of a broken UI.

So we required a mechanism to control this loading process and tap into its various aspects, to handle the edge cases. This, on the whole, required few new Web Standard APIs to enable smooth working of this feature. These API’s are

  • IntersectionObserver API
  • Fetch API


As the details of this feature are involving and comprise of new API standards, I have divided this into two posts, one with the basics of the above mentioned API’s and the outline approach to the module and its subcomponents and the difficulties which we faced. The second post will mostly comprise of the details of the code which went into making this feature and how we tackled the corner cases in the path.


Our goal here was to create a component which can lazily load the images and provide UI feedback to the user in case of any load or parse error. As mentioned above the development of this feature depends on the relatively new web standards API’s so it’s important to understand the functioning of these AP’s we understand how they become the intrinsic part of our LazyImgComponent.

Intersection Observer

If we see history, the problem of intersection of two elements on the web in a performant way has been impossible since always, because it requires DOM polling constantly for the ClientRect the element which we want to check for intersection, as these operations run on main thread these kinds of polling has always been a source of bottlenecks in the application performance.

The intersection observer API is a web standard to detect when two DOM elements intersect with each other. The intersection observer API helps us to configure a callback whenever an element called target intersects with another element (root) or viewport.

To create an intersection observer is a simple task we just have to create a new instance of the observer.

var observer = new IntersectionObserver(callback, options);

Here the callback is the function to run whenever some observed element intersect with the element supplied to the observer. This element is configured in the options object passed to the Intersection Observer

var options = {
root: document.querySelector('#root'), // Defaults to viewport if null
rootMargin: '0px', // The margin around root within which the callback is triggered
threshold: 1.0

The target element whose intersection is to be tested with the main element can be setup using the observe method of the observer.

var target = document.querySelector('#target');

After this setup whenever the target element intersects with the root element the callback method is fired, and this provides the easy standard mechanism to get callbacks whenever the target element intersects with root element.

How this is used in Loklak Search?

Our goal here is to load the images only if required, ie. we should load the images lazily only if they are in the viewport. So the task of checking whether the element is near the viewport is done in a performant way using the Intersection Observer standard.

Fetch API

Fetch API provides interface for fetching resources. It provides us with generic Request and Response interfaces, which can be used as Streaming responses, requests from Service Worker or CacheAPI. This is a very lightweight API providing us with the flexibility and power to make the AJAX requests from anywhere, irrespective of context of the thread or the worker. It is also a Promise driven API. Thus, providing the remedy from the callback hell.

The basic fetch requests are really very easy to setup, all they require is the path to the resource to fetch, and they return a promise on which we can apply promise chaining to transform the response into desired form. A very simple example illustratuing the fetch API is as follows.

.then(function(response) {
return response.blob();
.then(function(respBlob) {

The role of fetch api is pretty straight forward in the lazy loading of images, ie. to actually fetch the images when requested.

Lazy Image Component

We start designing our lazy image component by structuring our API. The API our component should provide to the outside world must be similar to Native Image Element, in terms of attributes and methods. So our component’s class should look something like this, with attributes src, alt, title, width and height. Also, we have hooked a load event which host can listen to for loading success or failure of the image.

selector: 'app-lazy-img',
templateUrl: './lazy-img.component.html',
styleUrls: ['./lazy-img.component.scss'],
export class LazyImgComponent {
@Input() src: string;
@Input() width: number;
@Input() height: number;
@Input() alt: string;
@Input() title: string;
@Output() load: EventEmitter<boolean> = new EventEmitter<boolean>();

This basic API provides us with our custom <app-lazy-img> tag which we can use with the attributes, instead of standard <img> element to load the images when needed.

So our component is basically a wrapper around the standard img element to provide lazy loading behaviour.

This is the basic outline of how our component will be working.

  • The component registers with an Intersection observer, which notifies the element when it comes near the viewport.
  • Upon this notification, the component fetches the resource image.
  • When the fetch is complete, the retrieved binary stream is fed to the native image element which eventually renders the image on the screen.

This is the basic setup and working logic behind our lazy image component. In next post, I will be discussing the details of how this logic is achieved inside the LazyImgComponent, and how we solve the problems with a large number of elements rendered at once in the application.

Resources and Links

  • Intersection observer API
  • Intersection Observer polyfill for the browsers which don’t support Intersection Observer
  • Fetch API documentation
  • Fetch API polyfill for the browsers which don’t support fetch.
  • Loklak Search Repo

PIL to convert type and quality of image

Image upload is an important part of the server. The images can be in different formats and after applying certain javascript modifications, they can be changed to different formats. For example, when an image is uploaded after cropping in open event organizer server, it is saved in PNG format. But PNG is more than 5 times larger than JPEG image. So when we upload a 150KB image, the image finally reaching the server is around 1MB which is huge. So we need to decide in the server which image format to select in different cases and how to convert them.

Continue reading PIL to convert type and quality of image

Optimizing page load time

The average size of a web page has been growing at an accelerating rate over the last few years. Last week, when the open-event webapp was tested, it was very slow to load because of loading 67.2 MB data ( which contained audio and image files ) on the web page.I have taken following steps which are good to read to make any web application load faster.

1. Stop preloading of audio files

The HTML5 audio player loads all audio files by default on the page. To stop this we have to change the code as

<audio controls preload="none">
 <source src="{{audio}}" type="audio/mpeg">

Setting preload option to none help us to load audio only when it is clicked. Hence it decreases the HTTP calls and decreases the page loading time.


Network tab with Audio files takes 68.0MB  load and 13.6 minute loading time.




Network tab with option <audio controls preloaded=”none”> takes 1.1 MB load and  1 minute loading time. This was a huge improvement but more optimizations can be done.

2. Using Compression Middleware

Express compression middleware works like a charm by compressing the response coming to the web page. It is too simple to add.

$ npm install compression
var compression = require('compression')
var express = require('express')

var app = express()

// compress all requests

// add all routes

This has compressed the responses and decreased the page load time.

3.  Reduce the number of HTTP requests

Another great way of speeding up your web pages is to simply reduce the number of files that need to be loaded.

Combine files

Combining multiple stylesheets into one file is a really useful way of eliminating extra HTTP requests. This strategy can also be adopted for your JavaScript files. A single larger CSS or JavaScript file will often load quicker because more time can be spent downloading the data rather than establishing multiple connections to a server.

After such optimizations, the webapp loads now in 8-15 seconds with DOM loaded in 2 seconds.


The result of these optimizations are awesome. We can test the page speed by using various tools like pagespeed, speedtracer etc.

SASS for the theme based concept

Until yesterday, I was exploring all around the internet, to find the best possible way for the theme concept in the web app. The theme concept allows an organizer to choose the theme for the web app from the set of provided themes.

After some time, I realised that this awesomeness to the web app can be added by using Syntactically Awesome Stylesheets.

How SASS supports the theme concept?


In the web app, a folder name _scss is added which has the directory structure as shown

_scss folder directory structure

There is a file _config.scss inside the _base folder that includes the SASS variables which are used in all the files after importing it.

Each of the SASS variables uses a flag !default at the end, which means it can be overwritten in any other file. This property of SASS leads to the theme concept.


@charset "UTF-8";
// Colors
$black: #000;
$white: #fff;
$red: #e2061c;
$gray-light: #c9c8c8;
$gray: #838282;
$gray-dark: #333333;
$blue: #253652;
// Corp-Colors
$corp-color: $white !default;
$corp-color-dark: darken($corp-color, 15%) !default;
$corp-color-second: $red !default;
$corp-color-second-dark: darken($corp-color-second, 15%) !default;
// Fontbasic
$base-font-size: 1.8 !default;
$base-font-family: Helvetica, Arial, Sans-Serif !default;
$base-font-color: $gray-dark !default;
// Border
$base-border-radius: 2px !default;
$rounded-border-radius: 50% !default;
$base-border-color: $gray !default;

The main file that includes all the required style is the application.scss. Here, $corp-color takes default value from  _config.scss. It can be overwritten by the themes.


@charset 'UTF-8';

// 1.Base
@import '_base/_config.scss';

/* Offsets for fixed navbar */
body {
 margin-top: 50px;
 background-color:$corp-color !important;

Making a light theme 


The light theme will overwrite the $corp-color value to $gray-light which is defined in _config.scss. This will change the background-color defined in application.scss to #c9c8c8. So, in this way a light theme is generated. The similar approach can be followed to generate the dark theme.


@charset 'UTF-8';
@import '../../_base/config';
// 1.Overwrite stuff
$corp-color: $gray-light;

@import '../../application';

Making a dark theme 



@charset 'UTF-8';
@import '../../_base/config';
// 1.Overwrite stuff
$corp-color: $gray;

@import '../../application';

How to compile the CSS from SASS files ?


  1. We can easily compile these SASS files by using command
    sass /path/application.scss /path/schedule.css

    For generating light theme and dark theme:

    sass /path/_light.scss  /path/schedule.css
    sass /path/_dark.scss  /path/schedule.css

Optimization of Code


SASS is very powerful for optimizing the code. It allows the concepts such as nesting, mixins which reduce the line of code. I have rewritten schedule.css into application.scss to make it more optimized.

/* Adjustments for smaller screens */

@media (min-width: 450px) {
 body {
 font-size: 14px
 .session {
         font-size: 90%;
         padding: .2em .6em .3em;
    &-title {
         width: 75%;
    &-location {
         float: right;
         width: 25%;
         text-align: right;

Further Exploration


This is one of the ways to deal with the SASS structure, but there are other resources that can be helpful in dealing with SASS projects.

Also, check out, Jeremy Hexon article here.