Adding Map and RSS Action Type Support to SUSI MagicMirror Module with React

SUSI being an interactive personal assistant, answers questions in a variety of formats. This includes maps, RSS, table, and pie-chart. SUSI MagicMirror Module earlier provided support for only Answer Action Type. So, if you were to ask about a location, it could not show you a map for that location. Support for a variety of formats was added to SUSI Module for MagicMirror so that users can benefit from rich responses by SUSI.AI. One problem that was faced while adding UI components is that in the MagicMirror Module structure, each module needs to supply its DOM by overriding the getDom() method. Therefore, you need to manage all the UI programmatically. Managing UI programmatically in Javascript is a cumbersome task since you need to create DOM nodes, manually apply styling to them, and add them to parent DOM object which is needed to be returned. We need to write UI for each element like below: getDom: function () { .... .... const moduleDiv = document.createElement("div"); const visualizerCanvas = document.createElement("canvas"); moduleDiv.appendChild(visualizerCanvas); const mapDiv = document.createElement("div"); loadMap(mapDiv,lat, long); moduleDiv.appendChild(mapDiv); ... ... } As you can see, manually managing the DOM is neither that easy nor a recommended practice. It can be done in a more efficient way using the React Library by Facebook.  React is an open source UI library by Facebook. It works on the concept of Virtual DOM i.e. the whole DOM object gets created in the memory and only the changed components are reflected on the document. Since the SUSI MagicMirror Module is primarily written in open-source TypeScript Lang (a typed superset of JavaScript), we also need to write React in TypeScript. To add React to a Typescript Project, we need to add some dependencies. They can be added using: $ yarn add react react-dom @types/react @types/react Now, we need to change our Webpack config to build .tsx files for React. TSX like JSX can contain HTML like syntax for representing DOM object in a syntactic sugar form. This can be done by changing resolve extensions and loaders config so that awesome typescript loaded compiles that TSX files. It is needed to be modified like below resolve: { extensions: [".js", ".ts", ".tsx", ".jsx"], }, module: { loaders: [{ test: /\.tsx?$/, loaders: ["awesome-typescript-loader"], }, { test: /\.json$/, loaders: ["json-loader"], }], }, This will allow webpack to build and load both .tsx and .ts files. Now that project is setup properly, we need to add UI for Map and RSS Action Type. The UI for Map is added with the help of React-Leaflet library. React-Leaflet module is a module build on top of Leaflet Map library for loading maps in Browser. We add the React-Leaflet library using $ yarn add react-leaflet Now, we declare a MapView Component in React and render Map in it using the React-Leaflet Library. Custom styling can be applied to it. The render function for MapView React Component is defined as follows. import * as React from "react"; import {Map, Marker, Popup, TileLayer} from "react-leaflet"; interface IMapProps { latitude:…

Continue ReadingAdding Map and RSS Action Type Support to SUSI MagicMirror Module with React

Using NodeJS modules of Loklak Scraper in Android

Loklak Scraper JS implements scrapers of social media websites so that they can be used in other platforms, like Android or in a native Java project. This way there will be only a single source of scraper, as a result it will be easier to update the scrapers in response to the change in websites. This blog explains how Loklak Wok Android, a peer for Loklak Server on Android platform uses the Twitter JS scraper to scrape tweets. LiquidCore is a library available for android that can be used to run standard NodeJS modules. But Twitter scraper can’t be used directly, due to the following problems: 3rd party NodeJS libraries are used to implement the scraper, like cheerio and request-promise-native and LiquidCore doesn’t support 3rd party libraries. The scrapers are written in ES6, as of now LiquidCore uses NodeJS 6.10.2, which doesn’t support ES6 completely. So, if 3rd party NodeJS libraries can be included in our scraper code and ES6 can be converted to ES5, LiquidCore can easily execute Twitter scraper. 3rd party NodeJS libraries can be bundled into Twitter scraper using Webpack and ES6 can be transpiled to ES5 using Babel. The required dependencies can be installed using: $npm install --save-dev webpack $npm install --save-dev babel-core babel-loader babel-preset-es2015 Bundling and Transpiling Webpack does bundling based on the configurations provided in webpack.config.js, present in root directory of project. var fs = require('fs'); function listScrapers() { var src = "./scrapers/" var files = {}; fs.readdirSync(src).forEach(function(data) { var entryName = data.substr(0, data.indexOf(".")); files[entryName] = src+data; }); return files; } module.exports = { entry: listScrapers(), target: "node", module: { loaders: [ { loader: "babel-loader", test: /\.js?$/, query: { presets: ["es2015"], } }, ] }, output: { path: __dirname + '/build', filename: '[name].js', libraryTarget: 'var', library: '[name]', } };   Now let’s break the config file, the function listScrapers returns a JSONObject with key as name of scraper and value as relative location of scraper, ex: {   twitter: "./scrapers/twitter.js",    github: "./scrapers/github.js"    // same goes for other scrapers } The parameters in module.exports as described in the documentation of webpack for multiple inputs and to use the generated output externally: entry: Since a bundle file is required for each scraper we provide the  the JSONObject returned by listScrapers function. The multiple entry points provided generate multiple bundled files. target: As the bundled files are to be used in NodeJS platform,  “node” is set here. module: Using webpack the code can be directly transpiled while bundling, the end users don’t need to run separate commands for transpiling. module contains babel configurations for transpiling. output: options here customize the compilation of webpack path: Location where bundled files are kept after compilation, “__dirname” means the current directory i.e. root directory of the project. filename: Name of bundled file, “[name]“ here refers to the key of JSONObject provided in entry i.e. key of JSONObect returned from listScrapers. Example for Twitter scraper, the filename of bundled file will be “twitter.js”. libraryTarget: by default the functions or methods inside bundled files…

Continue ReadingUsing NodeJS modules of Loklak Scraper in Android

Adding React based World Mood Tracker to loklak Apps

loklak apps is a website that hosts various apps that are built by using loklak API. It uses static pages and angular.js to make API calls and show results from users. As a part of my GSoC project, I had to introduce the World Mood Tracker app using loklak’s mood API. But since I had planned to work on React, I had to go off from the track of typical app development in loklak apps and integrate a React app in apps.loklak.org. In this blog post, I will be discussing how I introduced a React based app to apps.loklak.org and solved the problem of country-wise visualisation of mood related data on a World map. Setting up development environment inside apps.loklak.org After following the steps to create a new app in apps.loklak.org, I needed to add proper tools and libraries for smooth development of the World Mood Tracker app. In this section, I’ll be explaining the basic configuration that made it possible for a React app to be functional in the angular environment. Pre-requisites The most obvious prerequisite for the project was Node.js. I used node v8.0.0 while development of the app. Instead of npm, I decided to go with yarn because of offline caching and Internet speed issues in India. Webpack and Babel To begin with, I initiated yarn in the app directory inside project and added basic dependencies - $ yarn init $ yarn add webpack webpack-dev-server path $ yarn add babel-loader babel-core babel-preset-es2015 babel-preset-react --dev   Next, I configured webpack to set an entry point and output path for the node project in webpack.config.js - module.exports = { entry: './js/index.js', output: { path: path.resolve('.'), filename: 'index_bundle.js' }, ... }; This would signal to look for ./js/index.js as an entry point while bundling. Similarly, I configured babel for es2015 and React presets - { "presets":[ "es2015", "react" ] }   After this, I was in a state to define loaders for module in webpack.config.js. The loaders would check for /\.js$/ and /\.jsx$/ and assign them to babel-loader (with an exclusion of node_modules). React After configuring the basic presets and loaders, I added React to dependencies of the project - $ yarn add react react-dom   The React related files needed to be in ./js/ directory so that the webpack can bundle it. I used the file to create a simple React app - import React from 'react'; import ReactDOM from 'react-dom'; ReactDOM.render( <div>World Mood Tracker</div>, document.getElementById('app') );   After this, I was in a stage where it was possible to use this app as a part of apps.loklak.org app. But to do this, I first needed to compile these files and bundle them so that the external app can use it. Configuring the build target for webpack In apps.loklak.org, we need to have a file by the name of index.html in the app’s root directory. Here, we also needed to place the bundled js properly so it could be included in index.html at app’s root. HTML Webpack Plugin Using…

Continue ReadingAdding React based World Mood Tracker to loklak Apps