Lazy Loading Images in Loklak Search

In last blog post, I discussed the basic Web API’s which helps us to create the lazy image loader component. I also discussed the structure which is used in the application, to load the images lazily. The core idea is to wrap the <img> element in a wrapper, <app-lazy-img> element. This enables us the detection of the element in the viewport and corresponding loading only if the image is present in the viewport.

In this blog post, I will be discussing the implementation details about how this is achieved in Loklak search in an optimized manner.

The logic for lazy loading of images in the application is divided into a Component and a corresponding Service. The reason for this splitting of logic will be explained as we discuss the core parts of the code for this feature.

Detecting the Intersection with Viewport

The lazy image service is a service for the lazy image component which is registered at the by the modules which intend to use this app lazy image component. The task of this service is to register the elements with the intersection observer, and, then emit an event when the element comes in the viewport, which the element can react on and then use the other methods of services to actually fetch the image.

@Injectable()
export class LazyImgService {
private intersectionObserver: IntersectionObserver
= new IntersectionObserver(this.observerCallback.bind(this), { rootMargin: '50% 50%' });
private elementSubscriberMap: Map<Element, Subscriber<boolean>>
= new Map<Element, Subscriber<boolean>>();
}

The service has two member attributes, one is IntersectionObserver, and the other is a Map which stores the the reference of the subscribers of this intersection observer. This reference is then later used to emit the event when the element comes in viewport. The rootMargin of the intersection observer is set to 50% this makes sure that when the element is 50% away from the viewport.

The obvserve public method of the service, takes an element and pass it to intersection observer to observe, also put the element in the subscriber map.

public observe(element: Element): Observable<boolean> {
const observable: Observable<boolean> = new Observable<boolean>(subscriber => {
this.elementSubscriberMap.set(element, subscriber);
});
this.intersectionObserver.observe(element);
return observable;
}

Then there is the observer callback, this method, as an argument receives all the objects intersecting the root of the observer, when this callback is fired, we find all the intersecting elements and emit the intersection event. Indicating that the element is nearby the viewport and this is the time to load it.

private observerCallback(entries: IntersectionObserverEntry[], observer: IntersectionObserver) {
entries.forEach(entry => {
if (this.elementSubscriberMap.has(entry.target)) {
if (entry.intersectionRatio > 0) {
const subscriber = this.elementSubscriberMap.get(entry.target);
subscriber.next(true);
this.elementSubscriberMap.delete(entry.target);
}
}
});
}

Now, our LazyImgComponent enables us to uses this service to register its element, with the intersection observer and then reacting to it later, when the event is emitted. This method sets up the IO, to load the image, and subscribes to the event emittes by the service and eventually calls the loadImage method when the element intersects with the viewport.

private setupIntersectionObserver() {
this.lazyImgService
.observe(this.elementRef.nativeElement)
.subscribe(value => {
if (value) {
this.loadImage();
}
});
}

Loading and rendering the image

Our lazy image service has another public API method fetch to fetch the image resource, this method returns an observable, which on successful fetching of image emits a Base64 image string.

public fetch(resource: string): Observable<string> {
return new Observable<string>(subscriber => {
fetch(resource)
.then(this.processStatus)
.then(this.getBufferResponse)
.then(this.arrayBufferToBase64)
.then(strBuffer => {
subscriber.next(strBuffer);
subscriber.complete();
})
.catch((error) => {
subscriber.error(error);
subscriber.complete();
});
});
}

The intermediate promise then chain is for converting the raw response buffer to a Base64 string, this string is then emited as the observable emmision. The component then subscribes to this fetch Observable, when the load image method is called.

private loadImage() {
this.isLoading = true;
this.lazyImgService
.fetch(this.src)
.subscribe(this.handleResponse.bind(this), this.handleError.bind(this));
}

The handler methods for the response and errors then contain the code to handle the effects of loading of results, ie. rendering the image inside the img element. The intresting thing to note here is, if we give the Base64 string as the src attribute of an img tag, instead of resource path then also it renders the image properly.

private handleResponse(imageStr: string) {
const base64Flag = `data:image/${this.imageType};base64,`;
this.elementRef.nativeElement.querySelector('img').src = base64Flag + imageStr;
}

And this completes our workflow of the app-lazy-img and gives us, a robust lazy image loader, and also is compliant with accessibility guidelines, including all the necessary attributes like, title, width, height etc. for the generation of proper accessibility tree. This technique can be extended to any level, and is more or less platform and framework independent, as this relies solely on Web Standards API’s. This is an optimized solution, as at a time only one intersection observer is active on a page and is seeing all the images, rather than per component instance based intersection observers which can be a performane bottleneck in low memory devices.

Resources and Links

  • Intersection observer API
  • Intersection Observer polyfill for the browsers which don’t support Intersection Observer
  • Fetch API documentation
  • Fetch API polyfill for the browsers which don’t support fetch.
  • Loklak Search Repo
Continue ReadingLazy Loading Images in Loklak Search

Lazy loading images in Loklak Search

Loklak Search delivers the media rich content to the users. Most of the media delivered to the users are in the form of images. In the earlier versions of loklak search, these images were delivered to the users imperatively, irrespective of their need. What this meant is, whether the image is required by the user or not it was delivered, consuming the bandwidth and slowing down the initial load of the app as large amount of data was required to be fetched before the page was ready. Also, the 404 errors were also not being handled, thus giving the feel of a broken UI.

So we required a mechanism to control this loading process and tap into its various aspects, to handle the edge cases. This, on the whole, required few new Web Standard APIs to enable smooth working of this feature. These API’s are

  • IntersectionObserver API
  • Fetch API

 

As the details of this feature are involving and comprise of new API standards, I have divided this into two posts, one with the basics of the above mentioned API’s and the outline approach to the module and its subcomponents and the difficulties which we faced. The second post will mostly comprise of the details of the code which went into making this feature and how we tackled the corner cases in the path.

Overview

Our goal here was to create a component which can lazily load the images and provide UI feedback to the user in case of any load or parse error. As mentioned above the development of this feature depends on the relatively new web standards API’s so it’s important to understand the functioning of these AP’s we understand how they become the intrinsic part of our LazyImgComponent.

Intersection Observer

If we see history, the problem of intersection of two elements on the web in a performant way has been impossible since always, because it requires DOM polling constantly for the ClientRect the element which we want to check for intersection, as these operations run on main thread these kinds of polling has always been a source of bottlenecks in the application performance.

The intersection observer API is a web standard to detect when two DOM elements intersect with each other. The intersection observer API helps us to configure a callback whenever an element called target intersects with another element (root) or viewport.

To create an intersection observer is a simple task we just have to create a new instance of the observer.

var observer = new IntersectionObserver(callback, options);

Here the callback is the function to run whenever some observed element intersect with the element supplied to the observer. This element is configured in the options object passed to the Intersection Observer

var options = {
root: document.querySelector('#root'), // Defaults to viewport if null
rootMargin: '0px', // The margin around root within which the callback is triggered
threshold: 1.0
}

The target element whose intersection is to be tested with the main element can be setup using the observe method of the observer.

var target = document.querySelector('#target');
observer.observe(target);

After this setup whenever the target element intersects with the root element the callback method is fired, and this provides the easy standard mechanism to get callbacks whenever the target element intersects with root element.

How this is used in Loklak Search?

Our goal here is to load the images only if required, ie. we should load the images lazily only if they are in the viewport. So the task of checking whether the element is near the viewport is done in a performant way using the Intersection Observer standard.

Fetch API

Fetch API provides interface for fetching resources. It provides us with generic Request and Response interfaces, which can be used as Streaming responses, requests from Service Worker or CacheAPI. This is a very lightweight API providing us with the flexibility and power to make the AJAX requests from anywhere, irrespective of context of the thread or the worker. It is also a Promise driven API. Thus, providing the remedy from the callback hell.

The basic fetch requests are really very easy to setup, all they require is the path to the resource to fetch, and they return a promise on which we can apply promise chaining to transform the response into desired form. A very simple example illustratuing the fetch API is as follows.

fetch('someResource.xyz’')
.then(function(response) {
return response.blob();
})
.then(function(respBlob) {
doSomethingWithBlob(respBlob);
});

The role of fetch api is pretty straight forward in the lazy loading of images, ie. to actually fetch the images when requested.

Lazy Image Component

We start designing our lazy image component by structuring our API. The API our component should provide to the outside world must be similar to Native Image Element, in terms of attributes and methods. So our component’s class should look something like this, with attributes src, alt, title, width and height. Also, we have hooked a load event which host can listen to for loading success or failure of the image.

@Component({
selector: 'app-lazy-img',
templateUrl: './lazy-img.component.html',
styleUrls: ['./lazy-img.component.scss'],
})
export class LazyImgComponent {
@Input() src: string;
@Input() width: number;
@Input() height: number;
@Input() alt: string;
@Input() title: string;
@Output() load: EventEmitter<boolean> = new EventEmitter<boolean>();
}

This basic API provides us with our custom <app-lazy-img> tag which we can use with the attributes, instead of standard <img> element to load the images when needed.

So our component is basically a wrapper around the standard img element to provide lazy loading behaviour.

This is the basic outline of how our component will be working.

  • The component registers with an Intersection observer, which notifies the element when it comes near the viewport.
  • Upon this notification, the component fetches the resource image.
  • When the fetch is complete, the retrieved binary stream is fed to the native image element which eventually renders the image on the screen.

This is the basic setup and working logic behind our lazy image component. In next post, I will be discussing the details of how this logic is achieved inside the LazyImgComponent, and how we solve the problems with a large number of elements rendered at once in the application.

Resources and Links

  • Intersection observer API
  • Intersection Observer polyfill for the browsers which don’t support Intersection Observer
  • Fetch API documentation
  • Fetch API polyfill for the browsers which don’t support fetch.
  • Loklak Search Repo
Continue ReadingLazy loading images in Loklak Search

Updating Page Titles Dynamically in Loklak Search

Page titles are native in the web platform, and are prime ways to identify any page. The page titles have been in the web platform since ages. They tell the browsers, the web scrapers and search engines about the page content in 1-2 words. Since the titles are used for wide variety of things from presentation of the page, history references and most importantly by the search engines to categorise the pages, it becomes very important for any web application to update the title of the page appropriately. In earlier implementation of loklak search the page title was a constant and was not updated regularly and this was not a good from presentation and SEO perspective.

Problem of page titles with SPA

Since loklak search is a single page application, there are few differences in the page title implementation in comparison to a server served multi page application. In a server served multi page application, the whole application is divided into pages and the server knows what page it is serving thus can easily set the title of the page while rendering the template. A simple example will be a base django template which holds the place to have a title block for the application.

<!-- base.html -->

<title>{% block title %} Lokalk Search {% endblock %}</title>

<!-- Other application blocks -->

Now for any other page extending this base.html it is very simple to update the title block by simply replacing it with it’s own title.

<!-- home.html -->

{% extendsbase.html%}

{% block title %} Home Page - Loklak Search {% endblock %}

<!-- Other page blocks -->

When the above template is rendered by the templating engine it replaces the title block of the base.html with the updated title block specific to the page, thus for each page at the rendering time server is able to update the page title, appropriately.

But in a SPA, the server just acts as REST endpoints, and all the templating is done at the client side. Thus in an SPA the page title never changes automatically, from the server, as only the client is in control of what page (route) it is showing. Thus it becomes the duty of the client side to update the title of the page, appropriately, and thus this issue of static non informative page titles is often overlooked.

Updating page titles in Loklak Search

Before being able to start solving the issue of updating the page titles it is certainly very important to understand what all are the points of change in the application where we need to update the page title.

  • Whenever the route in the application changes.
  • Whenever new query is fetched from the server.

These two are the most important places where we definitely want to update the titles. The way we achieved is using the Angular Title Service. The title service is a platform-browser service by angular which abstracts the workflow to achieve the title updation. There are are two main methods of this service get and set title, which can be used to achieve our desired behaviour. What title service do is abstract the extraction of Title Node and get and set the title values.

For updation of title for each page which is loaded we just attach an onInit lifecycle hook to the parent component of that page and, onInit we use the title service to update the title accordingly.

@Component({
selector: 'app-home',
templateUrl: './home.component.html',
})
export class HomeComponent implements OnInit, OnDestroy {
constructor(
private titleService: Title
) { }

ngOnInit() {
this.titleService.setTitle(Loklak Search');

// Other initialization attributes and methods
}
}

Similarly other pages according to their context update the page titles accordingly using the simple title service. This solves the basic case of updation of the titles of the page when the actual route path changes, and thus component’s onInit lifecycle hook is the best place to change the title of the page.

@Component({
selector: 'app-home',
templateUrl: './home.component.html',
})
export class HomeComponent implements OnInit, OnDestroy {
constructor(
private titleService: Title
) { }

ngOnInit() {
this.titleService.setTitle(Loklak Search');

// Other initialization attributes and methods
}
}

But when the actual route path doesn’t change and we want to update the title according to the query searched then it is not possible to do it using lifecycle hooks of the component. But fortunately, we are using the ngrx effects in our application and thus this task also again becomes much simpler to achieve in the application. In this situation again what we do is hook up a title change effect to SearchCompleteSuccessAction, and there we change the title accordingly.

@Effect({ dispatch: false })
resetTitleAfterSearchSuccess$: Observable<void>
= this.actions$
.ofType(apiAction.ActionTypes.SEARCH_COMPLETE_SUCCESS,
apiAction.ActionTypes.SEARCH_COMPLETE_FAIL)
.withLatestFrom(this.store$)
.map(([action, state]) => {
const displayString = state.query.displayString;
let title = `${displayString} - Loklak Search`;
if (action.type === apiAction.ActionTypes.SEARCH_COMPLETE_FAIL) {
title += ' - No Results';
}
this.titleService.setTitle(title);
});

Now if we look closely this effect is somewhat different from all the other effects. Firstly, the effect observable is of type void instead of type Action which is the case with other effects, and also there is is a { dispatch: false } argument passed to the constructor. Both these things are important of our resetTitle effect. As our reset title effect has no action to dispatch on it it’s execution the the observable is of type void instead of type Action, and we never want to dispatch an effect whose type is not an Action thus we set dispatch to false. Rest of the code for the effect is fairly simple, we filter all the actions and take SearchSuccess and SearchFail actions, then we get the latest value of the query display string from the store, and we use our title service to reset the title accordingly.

Conclusion

The titles are the important part of the web platform and are used by browsers and search engines to present and rank the relevance of the page. While developing a SPA it is even more important to maintain an updated title tag, as it is the only thing which actually changes about the page according to the context of the page. In loklak search, the title service is now used to update the titles of the page according to the search results.

Resources and Links

Continue ReadingUpdating Page Titles Dynamically in Loklak Search

Search Engine Optimization and Meta Tags in Loklak Search

Ranking higher in search results is very important for any website’s, productivity and reach. These days modern search engines use algorithms and scrape the sites for relevant data points, then these data points are processed to result in a relevance number aka the page ranking. Though these days the algorithms which search engines use are very advanced and are able to generate the context of the page by the page content, but still there are some key points which should be followed by the developers to enable a higher page ranking on search results along with better presentation of search results on the pages. Loklak Search is also a web application thus it is very important to get these crucial points correct.

So the first thing which search engines see on a website is their landing or index page, this page gives information to the search engine what the data is about in this site and then search engines follow links to to crawl the details of the page. So the landing page should be able to provide the exact context for the page. The page can provide the context using the meta tags which store the metadata information about the page, which can be used by the search engines and social sites

Meta Charset

  • The first and foremost important tag is the charset tag. It specifies the character set being used on the page and declares the page’s character encoding which is used by browsers encoding algorithm.
  • It very important to determine a correct and compatible charset for security and presentation reasons.
  • Thus the most widely used charset UTF-8 is used in loklak search.
<meta charset="utf-8">

Meta Viewport

  • The mobile browsers often load the page in a representative viewport which is usually larger than the actual screen of the device so that the content of the screen is not crammed into small space.
  • This makes the user to pan-zoom around the page to reach the desired content. But this approach often undesirable design for the most of the mobile websites.
  • For this at loklak search we use
<meta name="viewport" content="width=device-width, initial-scale=1">

 

This specifies the relation between CSS pixel and device pixel. Here the relationship is actually computed by the browser itself, but this meta tag says that the width for calculating that ratio should be equal to device width and the initial-scale should be 1 (or zoom should be 0).

Meta Description

  • The meta description tag is the most important tad for the SEO as this is the description which is used by search engines and social media sites while showing the description of the page
<meta name="description"
   content="Search social media on Loklak Search. Our mission is to make the world’s social media information openly accessible and useful generating open knowledge for all">

 

  • This is how this description tag is used by google on the Google Search

Social Media meta tags

  • The social media meta tags are important for the presentation of the content of the page when the link is shared in these sites.
  • As the link sharing is fairly simple in social media the links should always be able to show some context about the page in a visual form without the need to click the link actually.
  • For this purpose Facebook and Twitter read special kinds of meta tags to read and display information regarding the page whenever the link is shared.
  • For twitter cards data we have used
<!-- Twitter Card data -->
 <meta name="twitter:card" content="summary">
 <meta name="twitter:site" content="@fossasia">
 <meta name="twitter:title" content="Loklak Search">
 <meta name="twitter:description" content="Search social media on Loklak Search. Our mission is to make the world’s social media information openly accessible and useful generating open knowledge for all">
 <meta name="twitter:image" content="http://loklak.net/assets/images/cow_400x466.png">

 

This specifies the card to be of type summary card. There are many types of cards which are provided by twitter and  can be found here.

  • The facebook also uses the similar approach but with different meta tag names for their graph API.
 <!-- Open Graph data -->
 <meta property="og:title" content="Loklak Search">
 <meta property="og:type" content="website">
 <meta property="og:url" content="http://loklak.net">
 <meta property="og:image" content="http://loklak.net/assets/images/cow_400x466.png">
 <meta property="og:description" content="Search social media on Loklak Search. Our mission is to make the world’s social media information openly accessible and useful generating open knowledge for all">
 <meta property="og:site_name" content="Loklak Search">

Conclusion

Presentation of the content in the web plays a crucial role in reach and usage of the application. Wider reach and high usage are the core goals of Loklak Search project so that open data reaches the masses. To achieve this we are slowly moving for better sharing experience for the loklak pages using standard techniques. By using these we hope that we will be able to reach a wider audience, thus delivering the open content to masses.

Resources and Links

Continue ReadingSearch Engine Optimization and Meta Tags in Loklak Search

Accessing Child Component’s API in Loklak Search

Loklak search being an angular application, comprises of components. Components provide us a way to organize the application in a more consistent way, along with providing the ability to reuse code in the application. Each component has two type of API’s public and private. Public API is the API which it exposes to the outer world for manipulating the working of the component, while private API is something which is local to the component and cannot be directly accessed by the outside world. Now when this distinction between the two is clear, it is important to state the need of these API’s, and why are they required in loklak search.

The components can never live in isolation, i.e. they have to communicate with their parent to be able to function properly. Same is the case with components of loklak search. They have to interact with others to make the application work. So how this, interaction looks like,

The rule of thumb here is, data flows down, events flow up. This is the core idea of all the SPA frameworks of modern times, unidirectional data flow, and these interactions can be seen everywhere in loklak search.

<feed-header
   [query]="query"
   (searchEvent)="doSearch($event)"></feed-header>

This is how a simple component’s API looks in loklak search. Here our component is FeedHeader and it exposes some of it’s API as inputs and outputs.

export class FeedHeaderComponent {

 @Input() query: string;

 @Output() searchEvent: EventEmitter<string> = new EventEmitter<string>();

  // Other methods and properties of the component
}

The FeedHeaderComponent ‘s class defines some inputs which it takes. These inputs are the data given to the component. Here the input is a simple query property, and the parent at the time of instantiating the component, passes the value to it’s child as [query]=”query”. This enables the one direction of API, from parent to child. Now, we also need a way for parent to be able to events generated by the child on interaction with user. For example, here we need to have a way to tell the parent to perform a search whenever user presses search button. For this the Output property searchEvent is used. The search event can be emitted by the child component independently. While the parent, if it wants to listen to child components simply do so by binding to the event and running a corresponding function whenever event is emitted (searchEvent)=”doSearch($event)”. Here the event which parent listens to is searchEvent and whenever such an event is emitted by the child a function doSearch is run by the parent. Thus this completes the event flow, from child to parent.

Now it is worth noticing that all these inputs for data and outputs for events is provided by the child component itself. They are the API of the child and parent’s job is just to bind to these inputs and outputs to bind to data and listen to events. This allows the component interactions in both directions.

@ViewChild and triggering child’s methods

The inputs are important to carry data from the parent to the child, declaratively but sometimes it is necessary for the parent to access the public API of it’s child more directly, specially the API methods to trigger an action. These methods require the way for the parent to access its child component. This is done by @ViewChild decorator. The child element which the parent wants access to, have to declare the component as, one of it’s attributes. Like in our example, the FeedHeaderComponent needs access to its child component SuggestBoxComponent, to show/hide suggest box as and when required. So here the feed header component gets the access to its child using viewchild decorator.

export class FeedHeaderComponent {

  @ViewChild(‘#suggestBox) suggestBox: SuggestBoxComponent;

  // Other properties and methods

  toggleSuggestBox() {

     this.suggestBox.toggle();
  }
}

The SuggestBoxComponent here has a public method toggle() which toggles the visibility state of the suggest box. This method is available as a component’s public API method. The parent of this component calls this method using the @ViewChild reference which it grabbed at the time of view instantiation.

export class SuggestBoxComponent {

  private suggestBoxVisible = true;

  public toggle() {

     this.suggestBoxVisible = !this.suggestBoxVisible;
  }
}

Resources and Links

  • Angular document pages
  • Basic Usage in Angular tour of heroes tutorial
  • In depth usage blog for Inputs and Outputs SitePoint Tutorial
  • Loklak Search Repo
Continue ReadingAccessing Child Component’s API in Loklak Search

Configurable Services in Loklak Search

Loklak search being an angular application has a concept of wiring down the code in the special form of classes called Services. These serviced have important characteristics, which make them a powerful feature of angular.

  • Services are shared common object wired together by Dependency Injection.
  • Services are lazily instantiated at the runtime.

 

The DI and the instantiation part of a service are handled by angular itself so we don’t have to bother about it. The parts of the services we are always concerned about is the logical part of the service. As the services are the sharable code at the time of writing a service we have to be 100% sure that this is the part of the code which we want to share with our components, else this can lead to the bad implementation of architecture which makes application harder to debug.

Now, the next question which arises is how services are different from something like redux state? Well, the difference lies in the word itself, services don’t have a persistent state of themselves. They are just a set of methods to separate a common piece of code from all the components into one class. These services have functions which take an input, processes them and spit an output.

Services in Loklak Search

So in loklak search, the main services are the ones which on request, fetch data from the backend API and return the data to the requester. All the services in loklak search have a fixed well-defined task, i.e. to use the API and get the data. This is how all the services must be designed, with a specific set of goals. A service should never try to do what is not necessary, or in other words, each service should have one and only one aim and it should do it nicely.

In loklak search, the services are classified by the API endpoints they hit to retrieve data. These services receive the query to be searched from the requested and they send the AJAX request to correct API endpoint and return the fetched data. This is the common structure of all the Loklak services, they all have a fetchQuery() method which takes a string argument query and requests the API for that query and after completion, it either returns the correct response from the API or throws an error if something goes wrong.

@Injectable()
class SearchService() {
public fetchQuery( query: string ) {  }
private extractData( response ) {  }
private handleError( error ) {  }
}

Problems faced in this structure

This simple structure was good enough for the application in the basic levels, but as the number of features in the application increase, our simple service becomes less and less flexible as the fetchQuery() method takes only a query string as an argument and requests the API for that query, along with some query parameters. These query parameters are the additional information given to the server to process and respond to our request in a particular way, like a number of results to be fetched, aggregations to be carried out, and much more. In the current implementation, the setting up these parameters were solely done by the service itself, so these parameters were fixed inside the service and there was no easy way to modify them. This reduced the flexibility of the service as all the requesters were bound to a fixed set of parameters, thus lacking the usability of the service in other places of the application.

 

Solution – Service Configs

The solution to this problem of service customizability is the Service Config classes. The objects of these classes contain the information about the query parameters which various requesters can configure according to their specific needs, and our services will simply configure the query params accordingly. This idea of having a shared structure for the service configurations plays very nicely with our scenario where we want extra control over the parameters which our service is configuring.

@Injectable()
class SearchService() {
public fetchQuery( query: string, config: SearchServiceConfig ) {  }
private extractData( response ) {  }
private handleError( error ) {  }
}

This small modification to our service structure enables us to have the amount of control which we required. The config class is a fairly simple one.

export class SearchServiceConfig {
private count: number;
private source: Source;
private fields: Set<AggregationFields>;
private aggregationLimit: number;
private maximumRecords: number;
private startRecord: number;
private timezoneOffset: string;
private filters: Set<Filter>;

// Other methods to get/set these attributes
}

Now any requester will instantiate a new object of this class and will set the attributes according to his needs then this object is passed to the fetchQuery() method of our function. Which designs the request to be sent accordingly.

Conclusion

In conclusion, i would like to mention the how these attributes are chosen to be a part of the Config and not as a query string. Our API endpoints accept the query string along with some attributes which filter out the results or run aggregations in various fields. So we should have all these attributes in our config as these all properties may vary according to the requesters need. Therefore, this idea of configurable services makes us not only better reuse the existing models and services in multiple situations but also make us write better predictable code.

Resources and Links

Continue ReadingConfigurable Services in Loklak Search

Query Model Structure of Loklak Search

Need to restructure

The earlier versions of loklak search applications had the issues of breaking changes whenever any new feature was added in the application. The main reason for these unintended bugs was identified to be the existing query structure. The query structure which was used in the earlier versions of the application only comprised of the single entity a string named as queryString.

export interface Query {
 queryString: string;
}

This simple query string property was good enough for simple string based searches which were the goals of the application in the initial phases, but as the application progressed we realized this simple string based implementation is not going to be feasible for the long term. As there are only a limited things we can do with strings. It becomes extremely difficult to set and reset the portions of the string according to the requirements. This was the main vision for the alternate architecture design scheme was to the ease of enabling and disabling the features on the fly.

Application Structure

Therefore, to overcome the difficulties faced with the simple string based structure we introduced the concept of an attribute based structure for queries. The attribute based structure is simpler to understand and thus easier to maintain the state of the query in the application.

export interface Query {
 displayString: string;
 queryString: string;
 routerString: string;
 filter: FilterList;
 location: string;
 timeBound: TimeBound;
 from: boolean;
}

The reason this is called an attribute based structure is that here each property of an interface is independent of one another. Thus each one can be thought of as a separate little key placed on the query, but each of these keys are different and are mutually exclusive. What this means is, if I want to write an attribute query, then it does not matter to me which other attributes are already present on the query. The query will eventually be processed and sent to the server and the corresponding valid results if exists will be shown to the user.

Now the question arises how do we modify the values of these attributes? Now before answering this I would like to mention that this interface is actually instantiated in the the Redux state, so now our question automatically gets answered, the modification to redux state corresponding to the query structure will be done by specific reducers meant for modification of each attribute. These reducers are again triggered by corresponding actions.

export const ActionTypes = {
 VALUE_CHANGE: '[Query] Value Change',
 FILTER_CHANGE: '[Query] Filter Change',
 LOCATION_CHANGE: '[Query] Location Change',
 TIME_BOUND_CHANGE: '[Query] Time Bound Change',
};

This ActionTypes object contains the the corresponding actions which are used to trigger the reducers. These actions can be dispatched in response to any user interaction by any of the components, thus modifying a particular state attribute via the reducer.

Converting from object to string

Now for our API endpoint to understand our query we need to send the proper string in API accepted format. For this there is need to convert dynamically from query state to query string, for this we need a simple function which take in query state as an input return the query string as output.

export function parseQueryToQueryString(query: Query): string {
 let qs: string;
 qs = query.displayString;
 if (query.location) {
qs += ` near:${query.location);

 if (query.timeBound.since) {
   qs += ` since:${parseDateToApiAcceptedFormat(query.timeBound.since)}`;

 if (query.timeBound.until) {
   qs += ` until:${parseDateToApiAcceptedFormat(query.timeBound.until)}`;

 return qs;
}

In this function we are just checking and updating the query string according to the various attributes set in the structure, and then returning the query string. So if eventually we have to convert to the string, then what is the advantage of this approach? The main advantage of this approach is that we know the query structure beforehand and we use the structure to build the string not just randomly selecting and removing pieces of information from a string. Whenever we update any of the attribute of the query state, the query is generated fresh, and not modifying the old string.

Conclusion

This approach makes the application to be able to modify the search queries sent to server in a streamlined and logical way, just by using simple data structure. This query model has provided us with a lot of advantages which are visible in the aspect of application stability and performance. This model has cuts out dirty regex matching, of typed queries and thus again help us to make simpler queries.

Resources and Links

Continue ReadingQuery Model Structure of Loklak Search