Auto Deploying loklak Server on Google Cloud Using Travis

This is a setup for loklak server which want to check in only the source files, but have the development branch in Kubernetes deployment automatically updated with some compiled output every time the push using details from Travis build. How to achieve it? Unix commands and shell script is one of the best option to automate all deployment and build activities. I explored Kubernetes Gcloud which can be accessed through unix command. 1.Checking for Travis build details before deployment: Firstly check whether the repository is loklak_server, pull request is available and branches are either master or development, and then decide to update the docker image or not. The code of the aforementioned things is as follows: if [ "$TRAVIS_REPO_SLUG" != "loklak/loklak_server" ]; then echo "Skipping image update for repo $TRAVIS_REPO_SLUG" exit 0 fi if [ "$TRAVIS_PULL_REQUEST" != "false" ]; then echo "Skipping image update for pull request" exit 0 fi if [ "$TRAVIS_BRANCH" != "master" ] && [ "$TRAVIS_BRANCH" != "development" ]; then echo "Skipping image update for branch $TRAVIS_BRANCH" exit 0 fi 2. Setting up Tag and Decrypting the credentials: For the Kubernetes deployment, each time the travis build is successful, it takes the commit details from travis and appended into tag details for deployment and gcloud credentials is decrypted from the json file. openssl aes-256-cbc -K $encrypted_48d01dc243a6_key -iv $encrypted_48d01dc243a6_iv -in kubernetes/gcloud-credentials.json.enc -out kubernetes/gcloud-credentials.json -d 3. Install, Authenticate and Configure GCloud details with Kubernetes: In this step, initially Google Cloud SDK should be installed with Kubernetes- curl https://sdk.cloud.google.com | bash > /dev/null source ~/google-cloud-sdk/path.bash.inc gcloud components install kubectl   Then, Authenticate Google Cloud using the above mentioned decrypted credentials and finally configure the Google Cloud with the details like zone, project name, cluster details, number of nodes etc. 4. Update the Kubernetes deployment: Since, in this issue it is specific to the loklak_server/development branch, so in here it checks if the branch is development or not and then updates the deployment using following command: if [ $TRAVIS_BRANCH == "development" ]; then kubectl set image deployment/server --namespace=web server=$TAG fi   Conclusion: In this post, how to write a script in such a way that with each successful push after travis build how to update the deployment on Kubernetes GCloud. Resources: Read more about Kubernetes GCloud deployment here: http://thylong.com/ci/2016/deploying-from-travis-to-gce/ Documentation available on Kubernetes: https://kubernetes.io/docs/tutorials/

Continue ReadingAuto Deploying loklak Server on Google Cloud Using Travis

Developing LoklakWordCloud app for Loklak apps site

LoklakWordCloud app is an app to visualise data returned by loklak in form of a word cloud. The app is presently hosted on Loklak apps site. Word clouds provide a very simple, easy, yet interesting and effective way to analyse and visualise data. This app will allow users to create word cloud out of twitter data via Loklak API. Presently the app is at its very early stage of development and more work is left to be done. The app consists of a input field where user can enter a query word and on pressing search button a word cloud will be generated using the words related to the query word entered. Loklak API is used to fetch all the tweets which contain the query word entered by the user. These tweets are processed to generate the word cloud. Related issue: https://github.com/fossasia/apps.loklak.org/pull/279 Live app: http://apps.loklak.org/LoklakWordCloud/ Developing the app The main challenge in developing this app is implementing its prime feature, that is, generating the word cloud. How do we get a dynamic word cloud which can be easily generated by the user based on the word he has entered? Well, here comes in Jqcloud. An awesome lightweight Jquery plugin for generating word clouds. All we need to do is provide list of words along with their weights. Let us see step by step how this app (first version) works. First we require all the tweets which contain the entered word. For this we use Loklak search service. Once we get all the tweets, then we can parse the tweet body to create a list of words along with their frequency. var url = "http://35.184.151.104/api/search.json?callback=JSON_CALLBACK&count=100&q=" + query; $http.jsonp(url) .then(function (response) { $scope.createWordCloudData(response.data.statuses); $scope.tweet = null; }); Once we have all the tweets, we need to extract the tweet texts and create a list of valid words. What are valid words? Well words like ‘the’, ‘is’, ‘a’, ‘for’, ‘of’, ‘then’, does not provide us with any important information and will not help us in doing any kind of analysis. So there is no use of including them in our word cloud. Such words are called stop words and we need to get rid of them. For this we are using a list of commonly used stop words. Such lists can be very easily found over the internet. Here is the list which we are using. Once we are able to extract the text from the tweets, we need to filter stop words and insert the valid words into a list. tweet = data[i]; tweetWords = tweet.text.replace(", ", " ").split(" "); for (var j = 0; j < tweetWords.length; j++) { word = tweetWords[j]; word = word.trim(); if (word.startsWith("'") || word.startsWith('"') || word.startsWith("(") || word.startsWith("[")) { word = word.substring(1); } if (word.endsWith("'") || word.endsWith('"') || word.endsWith(")") || word.endsWith("]") || word.endsWith("?") || word.endsWith(".")) { word = word.substring(0, word.length - 1); } if (stopwords.indexOf(word.toLowerCase()) !== -1) { continue; } if (word.startsWith("#") || word.startsWith("@")) { continue; } if (word.startsWith("http") || word.startsWith("https")) { continue; } $scope.filteredWords.push(word); }…

Continue ReadingDeveloping LoklakWordCloud app for Loklak apps site

Persistently Storing loklak Server Dumps on Kubernetes

In an earlier blog post, I discussed loklak setup on Kubernetes. The deployment mentioned in the post was to test the development branch. Next, we needed to have a deployment where all the messages are collected and dumped in text files that can be reused. In this blog post, I will be discussing the challenges with such deployment and the approach to tackle them. Volatile Disk in Kubernetes The pods that hold deployments in Kubernetes have disk storage. Any data that gets written by the application stays only until the same version of deployment is running. As soon as the deployment is updated/relocated, the data stored during the application is cleaned up. Due to this, dumps are written when loklak is running but they get wiped out when the deployment image is updated. In other words, all dumps are lost when the image updates. We needed to find a solution to this as we needed a permanent storage when collecting dumps. Persistent Disk In order to have a storage which can hold data permanently, we can mount persistent disk(s) on a pod at the appropriate location. This ensures that the data that is important to us stays with us, even when the deployment goes down. In order to add persistent disks, we first need to create a persistent disk. On Google Cloud Platform, we can use the gcloud CLI to create disks in a given region - gcloud compute disks create --size=<required size> --zone=<same as cluster zone> <unique disk name> After this, we can mount it on a Docker volume defined in Kubernetes configurations - ... volumeMounts: - mountPath: /path/to/mount name: volume-name volumes: - name: volume-name gcePersistentDisk: pdName: disk-name fsType: fileSystemType But this setup can’t be used for storing loklak dumps. Let’s see “why” in the next section. Rolling Updates and Persistent Disk The Kubernetes deployment needs to be updated when the master branch of loklak server is updated. This update of master deployment would create a new pod and try to start loklak server on it. During all this, the older deployment would also be running and serving the requests. The control will not be transferred to the newer pod until it is ready and all the probes are passing. The newer deployment will now try to mount the disk which is mentioned in the configuration, but it would fail to do so. This would happen because the older pod has already mounted the disk. Therefore, all new deployments would simply fail to start due to insufficient resources. To overcome such issues, Kubernetes allows persistent volume claims. Let’s see how we used them for loklak deployment. Persistent Volume Claims Kubernetes provides Persistent Volume Claims which claim resources (storage) from a Persistent Volume (just like a pod does from a node). The higher level APIs are provided by Kubernetes (configurations and kubectl command line). In the loklak deployment, the persistent volume is a Google Compute Engine disk - apiVersion: v1 kind: PersistentVolume metadata: name: dump namespace: web spec: capacity: storage:…

Continue ReadingPersistently Storing loklak Server Dumps on Kubernetes

Using Mosquitto as a Message Broker for MQTT in loklak Server

In loklak server, messages are collected from various sources and indexed using Elasticsearch. To know when a message of interest arrives, users can poll the search endpoint. But this method would require a lot of HTTP requests, most of them being redundant. Also, if a user would like to collect messages for a particular topic, he would need to make a lot of requests over a period of time to get enough data. For GSoC 2017, my proposal was to introduce stream API in the loklak server so that we could save ourselves from making too many requests and also add many use cases. Mosquitto is Eclipse’s project which acts as a message broker for the popular MQTT protocol. MQTT, based on the pub-sub model, is a lightweight and IOT friendly protocol. In this blog post, I will discuss the basic setup of Mosquitto in the loklak server. Installation and Dependency for Mosquitto The installation process of Mosquitto is very simple. For Ubuntu, it is available from the pre installed PPAs - sudo apt-get install mosquitto Once the message broker is up and running, we can use the clients to connect to it and publish/subscribe to channels. To add MQTT client as a project dependency, we can introduce following line in Gradle dependencies file - compile group: 'net.sf.xenqtt', name: 'xenqtt', version: '0.9.5' [SOURCE] After this, we can use the client libraries in the server code base. The MQTTPublisher Class The MQTTPublisher class in loklak would provide an interface to perform basic operations in MQTT. The implementation uses AsyncClientListener to connect to Mosquitto broker - AsyncClientListener listener = new AsyncClientListener() { // Override methods according to needs }; [SOURCE] The publish method for the class can be used by other components of the project to publish messages on the desired channel - public void publish(String channel, String message) { this.mqttClient.publish(new PublishMessage(channel, QoS.AT_LEAST_ONCE, message)); } [SOURCE] We also have methods which allow publishing of multiple messages to multiple channels in order to increase the functionality of the class. Starting Publisher with Server The flags which signal using of streaming service in loklak are located in conf/config.properties. These configurations are referred while initializing the Data Access Object and an MQTTPublisher is created if needed - String mqttAddress = getConfig("stream.mqtt.address", "tcp://127.0.0.1:1883"); streamEnabled = getConfig("stream.enabled", false); if (streamEnabled) { mqttPublisher = new MQTTPublisher(mqttAddress); } [SOURCE] The mqttPublisher can now be used by other components of loklak to publish messages to the channel they want. Adding Mosquitto to Kubernetes Since loklak has also a nice Kubernetes setup, it was very simple to introduce a new deployment for Mosquitto to it. Changes in Dockerfile The Dockerfile for master deployment has to be modified to discover Mosquitto broker in the Kubernetes cluster. For this purpose, corresponding flags in config.properties have to be changed to ensure that things work fine - sed -i.bak 's/^\(stream.enabled\).*/\1=true/' conf/config.properties && \ sed -i.bak 's/^\(stream.mqtt.address\).*/\1=mosquitto.mqtt:1883/' conf/config.properties && \ [SOURCE] The Mosquitto broker would be available at mosquitto.mqtt:1883 because of the service that is created…

Continue ReadingUsing Mosquitto as a Message Broker for MQTT in loklak Server

Deploying loklak Server on Kubernetes with External Elasticsearch

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. - kubernetes.io Kubernetes is an awesome cloud platform, which ensures that cloud applications run reliably. It runs automated tests, flawless updates, smart roll out and rollbacks, simple scaling and a lot more. So as a part of GSoC, I worked on taking the loklak server to Kubernetes on Google Cloud Platform. In this blog post, I will be discussing the approach followed to deploy development branch of loklak on Kubernetes. New Docker Image Since Kubernetes deployments work on Docker images, we needed one for the loklak project. The existing image would not be up to the mark for Kubernetes as it contained the declaration of volumes and exposing of ports. So I wrote a new Docker image which could be used in Kubernetes. The image would simply clone loklak server, build the project and trigger the server as CMD - FROM alpine:latest ENV LANG=en_US.UTF-8 ENV JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8 WORKDIR /loklak_server RUN apk update && apk add openjdk8 git bash && \ git clone https://github.com/loklak/loklak_server.git /loklak_server && \ git checkout development && \ ./gradlew build -x test -x checkstyleTest -x checkstyleMain -x jacocoTestReport && \ # Some Configurations and Cleanups CMD ["bin/start.sh", "-Idn"] [SOURCE] This image wouldn’t have any volumes or exposed ports and we are now free to configure them in the configuration files (discussed in a later section). Building and Pushing Docker Image using Travis To automatically build and push on a commit to the master branch, Travis build is used. In the after_success section, a call to push Docker image is made. Travis environment variables hold the username and password for Docker hub and are used for logging in - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD [SOURCE] We needed checks there to ensure that we are on the right branch for the push and we are not handling a pull request - # Build and push Kubernetes Docker image KUBERNETES_BRANCH=loklak/loklak_server:latest-kubernetes-$TRAVIS_BRANCH KUBERNETES_COMMIT=loklak/loklak_server:kubernetes-$TRAVIS_COMMIT if [ "$TRAVIS_BRANCH" == "development" ]; then docker build -t loklak_server_kubernetes kubernetes/images/development docker tag loklak_server_kubernetes $KUBERNETES_BRANCH docker push $KUBERNETES_BRANCH docker tag $KUBERNETES_BRANCH $KUBERNETES_COMMIT docker push $KUBERNETES_COMMIT elif [ "$TRAVIS_BRANCH" == "master" ]; then # Build and push master else echo "Skipping Kubernetes image push for branch $TRAVIS_BRANCH" fi [SOURCE] Kubernetes Configurations for loklak Kubernetes cluster can completely be configured using configurations written in YAML format. The deployment of loklak uses the previously built image. Initially, the image tagged as latest-kubernetes-development is used - apiVersion: apps/v1beta1 kind: Deployment metadata: name: server namespace: web spec: replicas: 1 template: metadata: labels: app: server spec: containers: - name: server image: loklak/loklak_server:latest-kubernetes-development ... [SOURCE] Readiness and Liveness Probes Probes act as the top level tester for the health of a deployment in Kubernetes. The probes are performed periodically to ensure that things are working fine and appropriate steps are taken if they fail. When a new image is updated, the older pod still runs and servers the requests. It is replaced by the new ones only when the probes are…

Continue ReadingDeploying loklak Server on Kubernetes with External Elasticsearch

Scraping Concurrently with Loklak Server

At Present, SearchScraper in Loklak Server uses numerous threads to scrape Twitter website. The data fetched is cleaned and more data is extracted from it. But just scraping Twitter is under-performance. Concurrent scraping of other websites like Quora, Youtube, Github, etc can be added to diversify the application. In this way, single endpoint search.json can serve multiple services. As this Feature is under-refinement, We will discuss only the basic structure of the system with new changes. I tried to implement more abstract way of Scraping by:- 1) Fetching the input data in SearchServlet Instead of selecting the input get-parameters and referencing them to be used, Now complete Map object is referenced, helping to be able to add more functionality based on input get-parameters. The dataArray object (as JSONArray) is fetched from DAO.scrapeLoklak method and is embedded in output with key results // start a scraper inputMap.put("query", query); DAO.log(request.getServletPath() + " scraping with query: " + query + " scraper: " + scraper); dataArray = DAO.scrapeLoklak(inputMap, true, true);   2) Scraping the selected Scrapers concurrently In DAO.java, the useful get parameters of inputMap are fetched and cleaned. They are used to choose the scrapers that shall be scraped, using getScraperObjects() method. Timeline2.Order order= getOrder(inputMap.get("order")); Timeline2 dataSet = new Timeline2(order); List<String> scraperList = Arrays.asList(inputMap.get("scraper").trim().split("\\s*,\\s*"));   Threads are created to fetch data from different scrapers according to size of list of scraper objects fetched. input map is passed as argument to the scrapers for further get parameters related to them and output data according to them. List<BaseScraper> scraperObjList = getScraperObjects(scraperList, inputMap); ExecutorService scraperRunner = Executors.newFixedThreadPool(scraperObjList.size()); try{ for (BaseScraper scraper : scraperObjList) { scraperRunner.execute(() -> { dataSet.mergePost(scraper.getData()); }); } } finally { scraperRunner.shutdown(); try { scraperRunner.awaitTermination(24L, TimeUnit.HOURS); } catch (InterruptedException e) { } }   3) Fetching the selected Scraper Objects in DAO.java Here the variable of abstract class BaseScraper (SuperClass of all search scrapers) is used to create List of scrapers to be scraped. All the scrapers' constructors are fed with input map to be scraped accordingly. List<BaseScraper> scraperObjList = new ArrayList<BaseScraper>(); BaseScraper scraperObj = null; if (scraperList.contains("github") || scraperList.contains("all")) { scraperObj = new GithubProfileScraper(inputMap); scraperObjList.add(scraperObj); } . . .   References: Best practices of Multithreading in Java: https://stackoverflow.com/questions/17018507/java-multithreading-best-practice ExecutorService vs Casual Thread Spawner: https://stackoverflow.com/questions/26938210/executorservice-vs-casual-thread-spawner Basic Data Structures used in Java: https://www.eduonix.com/blog/java-programming-2/learn-to-implement-data-structures-in-java/

Continue ReadingScraping Concurrently with Loklak Server

Caching Elasticsearch Aggregation Results in loklak Server

To provide aggregated data for various classifiers, loklak uses Elasticsearch aggregations. Aggregated data speaks a lot more than a few instances from it can say. But performing aggregations on each request can be very resource consuming. So we needed to come up with a way to reduce this load. In this post, I will be discussing how I came up with a caching model for the aggregated data from the Elasticsearch index. Fields to Consider while Caching At the classifier endpoint, aggregations can be requested based on the following fields - Classifier Name Classifier Classes Countries Start Date End Date But to cache results, we can ignore cases where we just require a few classes or countries and store aggregations for all of them instead. So the fields that will define the cache to look for will be - Classifier Name Start Date End Date Type of Cache The data structure used for caching was Java’s HashMap. It would be used to map a special string key to a special object discussed in a later section. Key The key is built using the fields mentioned previously - private static String getKey(String index, String classifier, String sinceDate, String untilDate) { return index + "::::" + classifier + "::::" + (sinceDate == null ? "" : sinceDate) + "::::" + (untilDate == null ? "" : untilDate); } [SOURCE] In this way, we can handle requests where a user makes a request for every class there is without running the expensive aggregation job every time. This is because the key for such requests will be same as we are not considering country and class for this purpose. Value The object used as key in the HashMap is a wrapper containing the following - json - It is a JSONObject containing the actual data. expiry - It is the expiry of the object in milliseconds. class JSONObjectWrapper { private JSONObject json; private long expiry; ... } Timeout The timeout associated with a cache is defined in the configuration file of the project as “classifierservlet.cache.timeout”. It defaults to 5 minutes and is used to set the eexpiryof a cached JSONObject - class JSONObjectWrapper { ... private static long timeout = DAO.getConfig("classifierservlet.cache.timeout", 300000); JSONObjectWrapper(JSONObject json) { this.json = json; this.expiry = System.currentTimeMillis() + timeout; } ... }   Cache Hit For searching in the cache, the previously mentioned string is composed from the parameters requested by the user. Checking for a cache hit can be done in the following manner - String key = getKey(index, classifier, sinceDate, untilDate); if (cacheMap.keySet().contains(key)) { JSONObjectWrapper jw = cacheMap.get(key); if (!jw.isExpired()) { // Do something with jw } } // Calculate the aggregations ... But since jw here would contain all the data, we would need to filter out the classes and countries which are not needed. Filtering results For filtering out the parts which do not contain the information requested by the user, we can perform a simple pass and exclude the results that are not needed. Since…

Continue ReadingCaching Elasticsearch Aggregation Results in loklak Server

Implementing Tweet Search feature in Loklak Wok Android

Loklak Wok Android is a peer harvester that posts collected tweets to the Loklak Server. Along with that tweets can be searched using the app. This post describes how search API endpoint and TabLayout is used to implement the tweet searching feature. Adding Dependencies to the project This feature uses Retrofit2, Reactive extensions(RxJava2, RxAndroid and Retrofit RxJava adapter) and RetroLambda (for Java lambda support in Android). In app/build.gradle: apply plugin: 'com.android.application' apply plugin: 'me.tatarka.retrolambda' android { ... packagingOptions { exclude 'META-INF/rxjava.properties' } } dependencies { ... compile 'com.google.code.gson:gson:2.8.1' compile 'com.squareup.retrofit2:retrofit:2.3.0' compile 'com.squareup.retrofit2:converter-gson:2.3.0' compile 'com.squareup.retrofit2:adapter-rxjava2:2.3.0' compile 'io.reactivex.rxjava2:rxjava:2.0.5' compile 'io.reactivex.rxjava2:rxandroid:2.0.1' }   In build.gradle project level: dependencies { classpath 'com.android.tools.build:gradle:2.3.3' classpath 'me.tatarka:gradle-retrolambda:3.2.0' }   Implementation The search API endpoint is defined in LoklakApi interface which would provide the tweet search result. public interface LoklakApi { @GET("api/search.json") Observable<Search> getSearchedTweets( @Query("q") String query, @Query("filter") String filter, @Query("count") int count); }   The POJOs (Plain Old Java Objects) for the result of search API endpoint are obtained using jsonschema2pojo, Gson uses POJOs to convert JSON to Java objects and vice-versa. The REST client is created by Retrofit2 and is implemented in RestClient class. The Gson converter and RxJava adapter for retrofit is added in the retrofit builder. create method is called to generate the API methods(retrofit implements LoklakApi Interface). public class RestClient { private RestClient() { } private static void createRestClient() { sRetrofit = new Retrofit.Builder() .baseUrl(BASE_URL) // gson converter .addConverterFactory(GsonConverterFactory.create(gson)) // retrofit adapter for rxjava .addCallAdapterFactory(RxJava2CallAdapterFactory.create()) .build(); } private static Retrofit getRetrofitInstance() { if (sRetrofit == null) { createRestClient(); } return sRetrofit; } public static <T> T createApi(Class<T> apiInterface) { // create method to generate API methods return getRetrofitInstance().create(apiInterface); } }   As search API endpoint provides filter parameter which can be used to filter out tweets containing images and videos. So, the tweets are displayed in three categories i.e. latest, images and videos. The tweets of different category are displayed using a ViewPager. The fragments in ViewPager are inflated by a class that extends FragmentPagerAdapter. SearchFragmentPagerAdapter extends FragmentPagerAdapter, at least two methods getItem and getCount needs to be overridden. Going by the name of methods, getItem provides ith fragment to the  ViewPager and based on the value returned by getCount number of tabs are inflated in TabLayout, a ViewGroup to display fragments in ViewPager in an elegant way. For better UI, the names (here the category of tweets) are displayed, for which we override getPageTitle method. public class SearchFragmentPagerAdapter extends FragmentPagerAdapter { private List<Fragment> mFragmentList = new ArrayList<>(); private List<String> mFragmentNameList = new ArrayList<>(); public SearchFragmentPagerAdapter(FragmentManager fm) { super(fm); } @Override public Fragment getItem(int position) { return mFragmentList.get(position); } @Override public int getCount() { return mFragmentList.size(); } @Override public CharSequence getPageTitle(int position) { return mFragmentNameList.get(position); } public void addFragment(Fragment fragment, String pageTitle) { mFragmentList.add(fragment); mFragmentNameList.add(pageTitle); } }   For easy understanding an analogy with RecyclerView can be made. The TabLayout here functions as a RecyclerView, ViewPager does the work of LayoutManager and FragmentPagerAdapter is analogous to RecyclerView.Adapter. Now, the fragments which contain the…

Continue ReadingImplementing Tweet Search feature in Loklak Wok Android

Creating CountryTweetMap app for Loklak apps site

The CountryTweetMap app is a web application which uses Loklak api and visualises loklak data on a map. The app is presently a part of Loklak apps site and can be used here. In this blog I have discussed in details about how I have developed CountryTweetMap. Before delving right into the development process let us know what the app does? Related issue: https://github.com/fossasia/apps.loklak.org/issues/244 Overview Loklak CoutryTweetMap uses a map to plot data returned by the Loklak api. The user needs to enter a query word in the input field and press search. The app uses Loklak API to determine all the tweets which contain the query words. Finally the various places from where tweets have been made containing the given query word are plotted on the map with markers of  specific color. Color can be red (highest number of tweets), blue (medium number of tweets) and green (less number of tweets). The three categories of markers are added as layers. That is, users can filter the markers. For example, the users can view only the green markers and hide the other markers if they want to know from which countries the number of tweets are low. Once the data is plotted there is also an option to plot distribution. It simply lists the returned data in a tabular form showing country v/s number of tweets. Developing CountryTweetMap app Now we know what CountryTweetMap does. Let us find out how it works. Like other loklak apps present on the app site, this app also uses the loklak API to fetch the data. Next it uses a framework called leaflet.js to plot the acquired data on map. ‘leaflet.js’ provides various APIs to deal with maps and corresponding layers. For date input, the app uses jquery UI, a library based on jquery which provides various ready to use components. Let us iterate through each step involved in creating the app. At first we take input from user and get the data from Loklak server. This is done by a simple ajax call. However before making the ajax call and actually getting the data, we need to ensure certain things. Firstly, we need to make sure that the user has actually entered something in the query field, dates are in order, that is start date is less than end date and count is a number. All these checks are done in the search function itself. $scope.error = ""; if ($scope.tweet === undefined || $scope.tweet === "" || $scope.isLoading === true) { if ($scope.tweet === undefined || $scope.tweet === "") { $scope.error = "Please enter a valid query word." } if ($scope.isLoading === true) { $scope.error = "Previous search not completed. Please wait..."; } $scope.showSnackbar(); return; } var count = $(".count").val(); if (count.length !== 0) { if (/^\d+$/.test(count) === false) { $scope.error = "Count should be a valid number."; $scope.showSnackbar(); return; } } var sinceDate = $(".start-date").val(); var endDate = $(".end-date").val(); if ((sinceDate !== undefined && endDate !== "") && (endDate !== undefined &&…

Continue ReadingCreating CountryTweetMap app for Loklak apps site

Adding Color Options in loklak Media Wall

Color options in loklak media wall gives user the ability to set colors for different elements of the media wall. Taking advantage of Angular two-way data binding property and ngrx/store, we can link up the CSS properties of the elements with concerned state properties which stores the user-selected color. This makes color customization fast and reactive for media walls. In this blog here, I am explaining the unidirectional workflow using ngrx for updating various colors and working of color customization. Flow Chart The flowchart below explains how the color as a string is taken as an input from the user and how actions, reducers and component observables link up to change the current CSS property of the font color. Working Designing Models: It is important at first to design model which must contain every CSS color property that can be customized. A single interface for a particular HTML element of media wall can be added so that color customization for a particular element can take at once with faster rendering. Here we have three interfaces: WallHeader WallBackground WallCard These three interfaces are the models for the three core components of the media wall that can be customized. export interface WallHeader { backgroundColor: string; fontColor: string; } export interface WallBackground { backgroundColor: string; } export interface WallCard { fontColor: string; backgroundColor: string; accentColor: string; }   Creating Actions: Next step is to design actions for customization. Here we need to pass the respective interface model as a payload with updated color properties. These actions when dispatched causes reducer to change the respective state property, and hence, the linked CSS color property. export class WallHeaderPropertiesChangeAction implements Action { type = ActionTypes.WALL_HEADER_PROPERTIES_CHANGE;constructor(public payload: WallHeader) { } } export class WallBackgroundPropertiesChangeAction implements Action { type = ActionTypes.WALL_BACKGROUND_PROPERTIES_CHANGE;constructor(public payload: WallBackground) { } } export class WallCardPropertiesChangeAction implements Action { type = ActionTypes.WALL_CARD_PROPERTIES_CHANGE;constructor(public payload: WallCard) { } }   Creating reducers: Now, we can proceed to create reducer functions so as to change the current state property. Moreover, we need to define an initial state which is the default state for uncustomized media wall. Actions can now be linked to update state property using this reducer when dispatched. These state properties serve two purposes: Updating Query params for Direct URL. Updating Media wall Colors case mediaWallCustomAction.ActionTypes.WALL_HEADER_PROPERTIES_CHANGE: { const wallHeader = action.payload;return Object.assign({}, state, { wallHeader }); }case mediaWallCustomAction.ActionTypes.WALL_BACKGROUND_PROPERTIES_CHANGE: { const wallBackground = action.payload;return Object.assign({}, state, { wallBackground }); }case mediaWallCustomAction.ActionTypes.WALL_CARD_PROPERTIES_CHANGE: { const wallCard = action.payload;return Object.assign({}, state, { wallCard }); }   Extracting Data to the component from the store: In ngrx, the central container for states is the store. Store is itself an observable and returns observable related to state properties. We have already defined various states for media wall color options and now we can use selectors to return state observables from the store. These observables can now easily be linked to the CSS of the elements which changes according to customization. private getDataFromStore(): void { this.wallCustomHeader$ = this.store.select(fromRoot.getMediaWallCustomHeader); this.wallCustomCard$ = this.store.select(fromRoot.getMediaWallCustomCard); this.wallCustomBackground$ =…

Continue ReadingAdding Color Options in loklak Media Wall