Tutorial

Read more about the article Using Activities with Bottom navigation view in Phimpme Android
BottonNavigationView

Using Activities with Bottom navigation view in Phimpme Android

Can we use Bottom Navigation View with activities? In the FOSSASIA Phimpme Project we integrated two big projects such as Open Camera and Leafpic. Both are activity based projects and not developed over fragment. The bottom navigation we use will work with fragment.

In Leafpic as well as in Open Camera, there are a lot of Multi level inheritance which runs the app components. The problem faced to continue to work on activities. For theme management using base activity is necessary. The base activity extend to AppcompatActivity. The leafpic code hierarchy is very complex and depend on various activities. Better way is to shift to activities for using Bottom navigation view and  work with activity similar like fragments.

Image source: https://storage.googleapis.com/material-design/

Solution:

This is possible if we copy the bottom navigation item in each of the activity and setting the clicked state of it as per the menu item ID.

Steps to Implement (How I did in Phimpme)

  • Add a Base Activity which takes care of two thing

    • Taking the layout element and setting in the activity.
    • Set up the correct navigation menu item clicked

Function selectBottomNavigationBarItem(int itemId) will take care of what item is currently active. Called this function in onStart(), getNavigationMenuItemId() function will return the menu id and selectBottomNavigagionBarItem update the bottom navigation view.

public abstract class BaseActivity extends AppCompatActivity implements BottomNavigationView.OnNavigationItemSelectedListener

private void updateNavigationBarState() {
   int actionId = getNavigationMenuItemId();
   selectBottomNavigationBarItem(actionId);
}

void selectBottomNavigationBarItem(int itemId) {
   Menu menu = navigationView.getMenu();
   for (int i = 0, size = menu.size(); i < size; i++) {
       MenuItem item = menu.getItem(i);
       boolean shouldBeChecked = item.getItemId() == itemId;
       if (shouldBeChecked) {
           item.setChecked(true);
           break;
       }
   }
}
  • Add two abstract functions in the BaseActivity for above task
public abstract int getContentViewId();

public abstract int getNavigationMenuItemId();
  • Extend every activity to the Base Activity

For example I created a blank Activity: AccountsActivity and extend to BaseActivity

public class AccountsActivity extends BaseActivity

  • Implement the abstract function in the every activity and set the correct layout and menu item id.

@Override
public int getContentViewId() {
 return R.layout.content_accounts;
}

@Override
public int getNavigationMenuItemId() {
   return R.id.navigation_accounts;
}
  • Remove the setContentView(R.layout.activity_main);

  • Add the onNavigationItemSelected in BaseActivity

@Override
public boolean onNavigationItemSelected(@NonNull final MenuItem item) {
   switch (item.getItemId()) {
       case R.id.navigation_home:
    startActivity(new Intent(this, LFMainActivity.class));
           Break;
       case R.id.navigation_camera:
           startActivity(new Intent(this, CameraActivity.class));
           Break;
       case R.id.navigation_accounts:
           startActivity(new Intent(this, AccountsActivity.class));
           Break;
   }
   finish();
   return true;
}

Output:

The transition is as smooth as we use bottom navigation views with Fragment. The bottom navigation also appear on above of the activity.

Source: https://stackoverflow.com/a/42811144/2902518

Resources:

Continue ReadingUsing Activities with Bottom navigation view in Phimpme Android
Read more about the article Phimpme: Merging several Android Projects into One Project

Phimpme: Merging several Android Projects into One Project

To speed up our development of the version 2 of the Phimpme Android app, we decided to use some existing Open Source libraries and projects such as Open Camera app for Camera features and Leafpic for the photo gallery.

Integrating several big projects into one is a crucial step, and it is not difficult. Here are some points to ponder.

  1. Clone project separately. Build project and note down the features which you want to integrate. Explore the manifest file to know the Launcher, services and other permission app is taking.

I explore the Leafpic app manifest file. Found out its Launcher class, and changed it with the our code. Looked at the permissions app required such as Camera, access fine location, write external storage.

  1. Follow Bottom Up approach while integrating. It makes life easier. By the bottom up approach I mean create a new branch and then start deleting the code which is not required. Always work in branch so that no code lost or messed up.

Not everything is required at the moment. So I first integrate the whole leafpic app and then remove the splash screen. Also noted down the features needs to remove such as drawer, search bar etc.

  1. Remove and Commit, In a big project things are interlinked so removing one feature, actually made you remove most of the code. So my strategy is to focus on one feature and when it removed commit the code. This will help you in a reset hard your code anytime.

Like a lot of times I use reset –hard in my code, because it messed up.

  1. Licensing: One thing which is most important in using open source code is their License. Always follow what is written in their License. This is the way we can give credit to their hard work. There are various types of licenses, three famous are Apache, MIT and GNU General Public License. All have their pros and cons.

Apart of License there are some other condition as well. Like in leafpic which we are using in Phimpme has a condition to inform developers about the project in which we are using leafpic.

  1. Aware of File Duplication, sometimes many files have same name which results into file duplication. So refactor them already in the code.

I refactor the MainActivity in leafpic to LFMainActivity to not to clash with the existing. Resolve package name. This is also one of the tedious task. Android Studio already do a lot of refactoring but some part left.

  • Make sure your manifest file contains correct name.
  • The best way to refactor your package name in xml file is to ctrl+Shift+f in Ubuntu or Edit → Find → Find in path in Android Studio. Then search old package name and replace it with new. You can use alt+ j for multi cursor as well.
  • Clean and rebuild project.

Run app at each step so that you can catch the error at a point. Also use debugger for debugging.

Resources

Continue ReadingPhimpme: Merging several Android Projects into One Project
Read more about the article Test SUSI Web App with Facebook Jest

Test SUSI Web App with Facebook Jest

Jest is used by Facebook to test all Javascript codes specially React code snippets. If you need to setup jest on your react application you can follow up these simple steps. But if your React application is made with “create-react-app”, you do not need to setup jest manually. Because it comes with Jest. You can run test using “react-scripts” node module.

Since SUSI chat is made with “create-react-app” we do not need to install and run Jest directly. We executed our test cases using “npm test” it executes “react-scripts test” command. It executes all “.js” files under “__tests__” folders. And all other files with “.spec.js” and “.test.js” suffixes.

React apps that are made from “create-react-app” come with sample test case (smoke test) and that checks whether whole application is built correctly or not. If it passes the smoke test then it starts to run further test cases.

import React from 'react';
import ReactDOM from 'react-dom';
import ChatApp from '../../components/ChatApp.react';
it('renders without crashing', () => {
 const div = document.createElement('div');
 ReactDOM.render( < ChatApp / > , div);
});

This checks all components which are inside of the “<ChatApp />” component and it check whether all these components integrate correctly or not.

If we need to check only one component in isolated environment. We can use shallow rendering API. we have used shallow rendering API to check each and every component in isolated environment.

We have to install enzyme and test renderer before use it.

npm install --save-dev enzyme react-test-renderer


import React from 'react';
import MessageSection from '../../components/MessageSection.react';
import { shallow } from 'enzyme';
it('render MessageListItem without crashing',()=>{
  shallow(<MessageSection />);
})

This test case tests only the “MessageListItem”, not its child components.

After executing “npm test” you will get the passed and failed number of test cases.

If you need to see the coverage you can see it without installing additional dependencies.

You just need to run this.

npm test -- --coverage

It will show the output like this.

This view shows how many lines, functions, statements, branches your program has and this shows how much of those covered from the test cases you have.

If we are going to write new test cases for susi chat, we have to make separate file in “__tests__” folder and name it with corresponding file name that we are going to test.

it('your test case description',()=>{
 //test what you need 
})

Normally test cases looks like this.in test cases you can use “test” instead of “it” .after test case description, there is a fat arrow function. In side this fat arrow function you can add what you need to test

In below example I have compared returned value of the function with static value.

function funcName(){
 return 1;
}

it('your test case description',()=>{
 expect(funcName()).toBe(1);
})

You have to add your function/variable that need to be tested into “expect()” and value you expect from that function or variable into “toBe()”.  Instead of “toBe()” you can use different functions according to your need.

If you have a long list of test cases you can group them ( using describe()).

describe('my beverage', () => {
  test('is delicious', () => {
    expect(myBeverage.delicious).toBeTruthy();
  });

  test('is not sour', () => {
    expect(myBeverage.sour).toBeFalsy();
  });
});

This post covers only few things of testing . You can learn more about jest testing from official documentation here.

Continue ReadingTest SUSI Web App with Facebook Jest
Read more about the article Step by step guide for Beginners in Web Development for Open Event Frontend

Step by step guide for Beginners in Web Development for Open Event Frontend

Originally the frontend and backend of the Open Event Server project were handled by FLASK with jinja2 being used for rendering templates. As the size of the project grew, it became difficult to keep track of all the modifications made on the frontend side. It also increased the complexity of the code. As a result of this, a new project Open Event Frontend was developed by decoupling the backend and frontend of the Open Event Orga Server. Now the server is being converted fully into functional API and database and the open event frontend project is primarily the frontend for the Open event server API where organisers, speakers and attendees can sign-up and perform various functions.     

The Open Event Frontend project is built on JavaScript web application framework, “Ember.js”. To communicate with the server API Ember.js user Ember data which is a data persistence module via the exposed endpoints. Suppose if you’re coming from the Android background, soon after diving into the web development you can relate that the web ecosystem is much larger than the mobile one and for the first timers it can be difficult to adopt with it because of the reason that in web there are multiple ways to perform a task which can be restricted to very few in the case of Android. For web applications, one can find that much more components are involved in setting up the project while in android one can easily start contributing to project soon after compiling it in Android Studio. One thing which is relatable for both the android and web development is that in the case of android one has to deal with the varying screen sizes and compatibility issue while in the web one has to deal with adding support for different browsers and versions which can be really annoying.

Now let’s see how one can start contributing to the Open event frontend project while having no or a little knowledge of web development. In case if you’ve previous knowledge of JavaScript then you can skip the step 1 and move directly to another step which is learning the framework.

(Here all the steps have been explained in reference if you’re switching from Android  to Web development.)

Step 1. Learning the Language – JavaScript

Now that when you’ve already put your feet into the web development it’s high time to get acquainted with the JavaScript. Essentially in the case of Ember which is easy to comprehend, you can though start with learning the framework itself but the executables file are written in JavaScript so to write them you must have basic knowledge of the concepts in the language. Understanding of JavaScript will help in letting know where the language ends and where the framework starts. It will also help in better understanding of the framework. In my opinion, the basic knowledge of JavaScript like the scope of variables, functions, looping, conditional statements, modifying array and dictionaries, ‘this’ keyword etc. helps in writing and understanding the .js files easily. Also, sometimes in JavaScript, an error might not be thrown as an exception while compiling but it may evaluate the program to undefined, knowledge of the language will help in debugging the code.

Step 2. Learning the Framework – Ember

Ember is a JavaScript Framework which works on Model-View-Controller(MVC) approach. The Ember is a battery included framework which generates all the boilerplate code including components, routes. Templates etc.  required for building an application’s frontend. It is very easy to understand and comprehend. In Ember, we can easily define the data models and relationships and ember will automatically guess the correct API endpoints. Apart from this, the documentation of the ember on its website is very much sufficient to start with. One can easily start developing applications after going through the tutorial mentioned on the ember’s website.  

Step 3. Testing

In the case of Android application development to write test we use android libraries like Mockito and Robolectric. Also, testing is a bit more difficult in Android app development because we have to explicitly write the test but it is a lot easier in the case of web development. In the case of Ember, it provides an ease of testing which no other framework and libraries provide. While generating a component or template ember itself generates the test files for them and all we have to do is to change them according to our requirement. Ember generates unit, acceptance and integration tests by making testing easier. So we don’t have to write the test explicitly we only have to modify the test files generated by ember.    

Step 4. Styling

In Android we have colors.xml, styles.xml, drawables, gradients, shapes etc. for styling our application but in the case of Web, we have Cascading Style Sheets (CSS) for styling our application. Simply using pure CSS make design complicated and difficult to understand, so to make it easier we combine a bunch of design elements with a style file and use Syntactically Awesome Style Sheets (Saas) with mixins to do that which makes creating styles a lot easier and more straightforward. So for styling, our web application one should have the knowledge of HTML as well as CSS.

In conclusion, I can say that learning web development requires learning a few things in parallel which includes learning a language, learning a framework, how to perform testing and different styling skills to make an application beautiful. Due to dynamic nature of the JavaScript and the sheer number of packages and components involved, as opposed to the safe environment that Android Studio provides, it can be sometimes really frustrating.  However, once learned the basics, the knowledge and skills can be easily transferred and applied over and over again.    

Continue ReadingStep by step guide for Beginners in Web Development for Open Event Frontend

How to make SUSI AI Line Bot

In order to integrate SUSI’s API with Line bot you will need to have a line account first so that you can follow below procedure. You can download app from here.

Pre-requisites:

  • Line app
  • Github
  • Heroku

    Steps:
    1. Install Node.js from the link below on your computer if you haven’t installed it already https://nodejs.org/en/.
    2. Create a folder with any name and open shell and change your current directory to the new folder you created.
    3. Type npm init in command line and enter details like name, version and entry point.
    4. Create a file with the same name that you wrote in entry point in above given step. i.e index.js and it should be in same folder you created.
    5. Type following commands in command line  npm install –save @line/bot-sdk. After bot-sdk is installed type npm install –save express after express is installed type npm install –save request when all the modules are installed check your package.json modules will be included within dependencies portion.

      Your package.json file should look like this.

      {
"name": "SUSI-Bot",
"version": "1.0.0",
"description": "SUSI AI LINE bot",
"main": "index.js",
"dependencies": {
   "@line/bot-sdk": "^1.0.0",
   "express": "^4.15.2",
   "request": "^2.81.0"
},
"scripts": {
   "start": "node index.js"
 }
}
    6. Copy following code into file you created i.e index.js 
      'use strict';
const line = require('@line/bot-sdk');
const express = require('express');
var request = require("request");

// create LINE SDK config from env variables

const config = {
   channelAccessToken: process.env.CHANNEL_ACCESS_TOKEN,
   channelSecret: process.env.CHANNEL_SECRET,
};

// create LINE SDK client

const client = new line.Client(config);


// create Express app
// about Express: https://expressjs.com/

const app = express();

// register a webhook handler with middleware

app.post('/webhook', line.middleware(config), (req, res) => {
   Promise
       .all(req.body.events.map(handleEvent))
       .then((result) => res.json(result));
});

// event handler

function handleEvent(event) {
   if (event.type !== 'message' || event.message.type !== 'text') {
       // ignore non-text-message event
       return Promise.resolve(null);
   }

   var options1 = {
       method: 'GET',
       url: 'http://api.asksusi.com/susi/chat.json',
       qs: {
           timezoneOffset: '-330',
           q: event.message.text
       }
   };

   request(options, function(error, response, body) {
       if (error) throw new Error(error);
       // answer fetched from susi
       //console.log(body);
       var ans = (JSON.parse(body)).answers[0].actions[0].expression;
       // create a echoing text message
       const answer = {
           type: 'text',
           text: ans
       };

       // use reply API

       return client.replyMessage(event.replyToken, answer);
   })
}

// listen on port

const port = process.env.PORT || 3000;
app.listen(port, () => {
   console.log(`listening on ${port}`);
});
    7. Now we have to get channel access token and channel secret to get that follow below steps.

    8. If you have Line account then move to next step else sign up for an account and make one.
    9. Create Line account on  Line Business Center with messaging API and follow these steps:
    10. In the Line Business Center, select Messaging API under the Service category at the top of the page.
    11. Select start using messaging API, enter required information and confirm it.
    12. Click LINE@ Manager option, In settings go to bot settings and Enable messaging API
    13. Now we have to configure settings. Allow messages using webhook and select allow for “Use Webhooks”.
    14. Go to Accounts option at top of page and open LINE Developers.
    15. To get Channel access token for accessing API, click ISSUE for the “Channel access token” item.
    16. Click EDIT and set a webhook URL for your Channel. To get webhook url deploy your bot to heroku and see below steps.
    17. Before deploying we have to make a github repository for chatbot to make github repository follow these steps:

      In command line change current directory to folder we created above and  write

      git init
git add .
git commit -m”initial”
git remote add origin <URL for remote repository> 
git remote -v
git push -u origin master

      You will get URL for remote repository by making repository on your github and copying this link of your repository.

    18. To deploy your bot to heroku you need an account on Heroku and after making an account make an app.
    19. Deploy app using github deployment method.


    20. Select Automatic deployment method.


    21. After making app copy this link and paste it in webhook url in Line channel console page from where we got channel access token.

                https://<your_heroku_app_name>.herokuapp.com/webhook
    22. Your SUSI AI Line bot is ready add this account as a friend and start chatting with SUSI.
      Here is the LINE API reference https://devdocs.line.me/en/
Continue ReadingHow to make SUSI AI Line Bot
Read more about the article Deploy Static Web Pages In Six Keystrokes

Deploy Static Web Pages In Six Keystrokes

I added two fairly young projects – Query Server and YayDoc to the projects list on http://labs.fossasia.org/. I pulled the code from GitHub, made the changes and it worked fine. Now to get it reviewed from a co-developer, I needed to host my changes somewhere on the web.

The fossasia-labs repository runs on gh-pages by GitHub. Hence, one way of hosting my changes was to use gh-pages on my fork but I tried this tool instead to deploy my site in six keystrokes.

This is what it took to deploy the static webpage right from my command line. Let’s dive into how this tool is as easy as it gets.

What is surge?
surge is a web-publishing tool aimed at front-end developers to help them get their static web pages up and running easily. It can be used to deploy HTML, CSS and JS with the ease of a single command.

How to use surge?
surge is quite an easy tool to use.  It has been developed as a npm package. Now for folks who don’t know what npm is – npm is the JavaScript package manager (Curious?).

To have surge running, you need to have Node.js installed. Run these in the terminal:

sudo apt-get update 
sudo apt-get install nodejs
sudo apt-get install npm

Now you have Nodejs as well as npm installed. Let’s move on to the main course – installing surge.

npm install --global surge

You have installed surge!
(You may need to preface this command with sudo.)

So let’s go to the directory where we have our files to deploy. Here I have the labs.fossasia.org repository which we’ll try to deploy.

To clone this repo, run this command:

git clone git@github.com:fossasia/labs.fossasia.org.git

After cding into the directory named labs.fossasia.org type

surge

and hit enter.

You’ll be prompted to sign up with your email. Choose a password. After that you’ll see something similar to this.  

Properties of the directory – path and size are listed here. Also, as you can see in the picture, a domain is listed. This is a randomly generated domain by surge. You can stick with it too, or just delete it and type whatever domain you like. surge will deploy your directory to that domain, provided that it is available.

In this example, I thought to escape elfin-education and go with my-labs.surge.sh .

Press enter after typing in the desired domain name and you’ll see surge uploading files to the domain. After it successfully deploys, you’ll get a message :


That’s it. Finally it’s time to check my-labs.surge.sh .

Saving your Domain with CNAME

Next up we take a look at making surge remember the domain.

You’ll be prompted for a domain name, every time you run surge inside the same directory (this is the default behavior). This can be avoided by simply adding a CNAME file to your directory root. Let’s say that you want to stick with ‘my-labs.surge.sh’ in the above example. You can add it to the CNAME file by running this in the terminal.

  echo my-labs.surge.sh > CNAME

surge also offers adding your own custom domain for deployments. To know about this and read further about surge, visit surge.sh .


Additional Resources

Continue ReadingDeploy Static Web Pages In Six Keystrokes
Read more about the article Skills for SUSI

Skills for SUSI

Susi is an open source personal assistant can do a lot of amazing things for you apart from just answering queries in the text. Susi supports many action type such as answer, table, pie chart, RSS, web search, map. Actions contain a list of objects each with type attribute, there can be more than one actions in the object list. For example

curl http://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=Who+are+you

We get a json response similar to

{
"query": "Who are you",
"answers": [{
"data": [],
"metadata": {"count": 0},
"actions": [{
"type": "answer",
"expression": "I was told that I am a Social Universal Super Intelligence. What do you think?"
}]
}],
}

The above query is an example of action type ‘answer’, for developing more skills on action type answer refer to the Fossasia  blog : How to teach Susi skills.  In this blog we will see how to teach a table skill to susi and how susi interprets the skill .So let’s add a skill to display football teams and its players using service Football-Data.org .

For writing rules Open a new etherpad with a desired name <etherpad name> at http://dream.susi.ai/

  1. Next let’s set some query for which we want Susi to answer.

Example queries:

tell me the teams in premier league | Premier league teams

To get answer we define the following rule

!console:
{
"url":"http://api.football-data.org/v1/competitions/398/teams",
"path":"$.teams",
"actions":[{
"type":"table",
"columns":{"name":"Name","code":"Code","shortName":"Short Name","crestUrl":Logo},
"count": -1
}]
}
eol

Expalanation:

The JSON response for above URL look like this:

{
"_links": {
"self": {
"href": "http://api.football-data.org/v1/competitions/398/teams"
},
"competition": {
"href": "http://api.football-data.org/v1/competitions/398"
}
},
"count": 20,
"teams": [{
"_links": {
"self": {
"href": "http://api.football-data.org/v1/teams/66"
},
"fixtures": {
"href": "http://api.football-data.org/v1/teams/66/fixtures"
},
"players": {
"href": "http://api.football-data.org/v1/teams/66/players"
}
},
"name": "Manchester United FC",
"code": "MUFC",
"shortName": "ManU",
"squadMarketValue": null,
"crestUrl": "http://upload.wikimedia.org/wikipedia/de/d/da/Manchester_United_FC.svg"
},
{
"_links": {
"self": {
"href": "http://api.football-data.org/v1/teams/65"
},
"fixtures": {
"href": "http://api.football-data.org/v1/teams/65/fixtures"
},
"players": {
"href": "http://api.football-data.org/v1/teams/65/players"
}
},
"name": "Manchester City FC",
"code": "MCFC",
"shortName": "ManCity",
"squadMarketValue": null,
"crestUrl": "https://upload.wikimedia.org/wikipedia/en/e/eb/Manchester_City_FC_badge.svg"
}]
}

The attribute ‘path’  statuses object contains a list of objects for which we want to show the table. The table is defined with the action type “table” and a columns object which provides a mapping from the column value names to the descriptive names that will be rendered in the client’s output. In our case, there are  4 columns  Name of the team, Team Code, Short Name of the team, and the Team logo’s URL.

The count attribute is used to denote how many rows to populate the table. If count = -1 , then it means as many as possible or displays all the results.It’s easy, isn’t it? We have successfully created table skill for SUSI. Let’s try it out

Go to http://chat.susi.ai/ and type: dream < your dream name>. And type your queries like  “tell me the teams in premier league” in our case  and see how SUSI presents you the results


Next, want to create some more skills. Let’s teach SUSI to play a game Rock, Paper, Scissors, Lizard, Spock. SUSI can give random answers to your queries for example

What is your favorite dish
Potatoes|Vegetables|Fish

SUSI will return any random answer “Potatoes, “Vegetables” or “Fish”. We would use this to create a game.
First form a general query like “I am getting bored let us play something” and the add an answer to it by adding the rules of the game something like

I am getting bored let us play something
Let's play the game Rock, Paper, Scissors, Lizard, Spock. It is an expansion to the game Rock, Paper, Scissors. We both will  pick a variable and reveal. The winner will be  one who defeats the others. It goes like Scissors cuts Paper,Paper covers Rock, Rock crushes Lizard, Lizard poisons Spock, Spock smashes Scissors, Scissors decapitates Lizard, Lizard eats Paper, Paper disproves Spock, Spock vaporizes Rock, (and as it always has) Rock crushes Scissors. Let's begin, choose something

Next since user would choose between any 5 of the Rock, Scissors, Lizard,Paper and spock add answers to each of them based on the rules.

Rock | I choose Rock
Paper, Paper covers Rock I won :) | Sccisors, Rock crushes Scissors , I lost :( | Lizard, Rock crushes Lizard , You won :-) | Spock, Spock vaporises Rock, You lost :) | Rock, Ahh It's a Tie :-D

Paper | I choose Paper
Rock, Paper covers Rock You won :( | Sccisors, Scissors cuts Paper I won :) | Lizard, Lizard eats Paper I won :) | Spock , Paper disproves Spock You won :( | Paper, Ahh It's a Tie :-D

Lizard | I choose Lizard
Rock, Rock crushes Lizard , I won :-) | Spock, Lizard poisons Spock You won :( | Sccisors, Scissors decapitates Lizard I won :) | Paper, Lizard eats Paper You won :( | Lizard, Ahh It's a Tie :-D

Scissors | I choose Scissors
Rock, Rock crushes Scissors , You lost :) | Paper , Scissors cuts Paper You won :( | Spock, Spock smashes Scissors I won :) | Lizard, Scissors decapitates Lizard You won :( | Scissors, Ahh It's a Tie :-D

Spock | I choose Spock
Rock, Spock vaporizes Rock, I lost :( | Lizard, Lizard poisons Spock You lost :) | Scissors, Spock smashes Scissors I lost :(| Paper, Paper disproves Spock I won :) | Spock, Ahh It's a Tie :-D

SUSI will return any random answer to the variable picked go and try to beat SUSI.

Have fun with SUSI and do create some other awesome skills for SUSI, read this tutorial  for more details.

Continue ReadingSkills for SUSI

Hotword Detection with Pocketsphinx for SUSI.AI

Susi has many apps across all the major platforms. Latest addition to them is the Susi Hardware which allows you to setup Susi on a Hardware Device like Raspberry Pi.

Susi Hardware was able to interact with a push of a button, but it is always cool and convenient to call your assistant anytime rather than using a button.

Hotword Detection helps achieve that. Hotword Detection involves running a process in the background that continuously listens for voice. On noticing an utterance, we need to check whether it contains desired word. Hotword Detection and its integration with Susi AI can be explained using the diagram below:

 

What is PocketSphinx?

PocketSphinx is a lightweight speech recognition engine, specifically tuned for handheld and mobile devices, though it works equally well on the desktop.

PocketSphinx is free and open source software. PocketSphinx has various applications but we utilize its power to detect a keyword (say Hotword) in a verbally spoken phrase.

Official Github Repository: https://github.com/cmusphinx/pocketsphinx

Installing PocketSphinx

We shall be using PocketSphinx with Python. Latest version on it can be installed by

pip install pocketsphinx

If you are using a Raspberry Pi or ARM based other board, Python 3.6 , it will install from sources by the above step since author doesn’t provide a Python Wheel. For that, you may need to install swig additionally.

sudo apt install swig

How to detect Hotword with PocketSphinx?

PocketSphinx can be used in various languages. For Susi Hardware, I am using
Python 3.

Steps:

      • Import PyAudio and PocketSphinx 
        from pocketsphinx import *
import pyaudio

         

      • Create a decoder with certain model, we are using en-us model and english us default dictionary. Specify a keyphrase for your application, for Susi AI , we are using “Susi” as Hotword 
        pocketsphinx_dir = os.path.dirname(pocketsphinx.__file__)
model_dir = os.path.join(pocketsphinx_dir, 'model')

config = pocketsphinx.Decoder.default_config()
config.set_string('-hmm', os.path.join(model_dir, 'en-us'))
config.set_string('-keyphrase', 'susi')
config.set_string('-dict', os.path.join(model_dir, dict_name))
config.set_float('-kws_threshold', self.threshold)

         

      • Start a PyAudio Stream from Microphone Input 
        p = pyaudio.PyAudio()

stream = p.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True, frames_per_buffer=20480)
stream.start_stream()

         

      • In a forever running loop, read from stream and process buffer in chunks of 1024 frames if it is not empty. 
        buf = stream.read(1024)
if buf:
    decoder.process_raw(buff)
else:
    break;

         

      • Check for hotword and start speech recognition if hotword detected. After returning from the method, start detection again.

        if decoder.hyp() is not None:
    print("Hotword Detected")
    decoder.end_utt()
    start_speech_recognition()
    decoder.start_utt()

         

      • Run the app, if detection doesn’t seem to work well, adjust kws_threshold in step 2 to give optimal results.

In this way, Hotword Detection can be added to your Python Project. You may also develop some cool hacks with our AI powered Assistant Susi by Voice Control.
Check repository for more info: https://github.com/fossasia/susi_hardware

Continue ReadingHotword Detection with Pocketsphinx for SUSI.AI
Read more about the article Conversion of CSS styles into React styles in SUSI Web Chat App

Conversion of CSS styles into React styles in SUSI Web Chat App

Earlier this week we had an issue where the text in our search box of the SUSI web app was not white even after having all the required styles. After careful inspection it was found that there is a attribute named -webkit-text-fill-color which was set to black.

Now I faced this issue as adding such attribute to our reactJs code will cause lint errors. So after careful searching stackoverflow, i found a way to add css attribute to our react code by using different case. I decided to write a blog on this for future reference and it might come handy to other developers as well.

If you want to write css in javascript, you have to turn dashed-key-words into camelCaseKeys

For example:

background-color => backgroundColor
border-radius => borderRadius
but vendor prefix starts with capital letter (except ms)

-webkit-box-shadow => WebkitBoxShadow (capital W)
-ms-transition => msTransition ('ms' is the only lowercase vendor prefix)

const containerStyle = {
  WebkitBoxShadow: '0 0 0 1000px white inset'
};

So in our case:-

-webkit-text-fill-color became WebkitTextFillColor

The final code of styles looked like: –

const searchstyle = {
      WebkitTextFillColor: 'white',
      color: 'white'
    }

Now, because inline styles gets attached on tags directly instead of using selectors, we have to put this style on the <input> tag itself, not the container.

See the react doc #inline-styles section for more details.

Continue ReadingConversion of CSS styles into React styles in SUSI Web Chat App

Websearch and Link Preview support in SUSI iOS

The SUSI.AI server responds to API calls with answers to the queries made. These answers might contain an action, for example a web search, where the client needs to make a web search request to fetch different web pages based on the query. Thus, we need to add a link preview in the iOS Client for each such page extracting and displaying the title, description and a main image of the webpage.

At first we make the API call adding the query to the query parameter and get the result from it.

API Call:

http://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=amazon

And get the following result:

{

"query": "amazon",

"count": 1,

"client_id": "aG9zdF8xMDguMTYyLjI0Ni43OQ==",

"query_date": "2017-06-02T14:34:15.675Z",

"answers": [

{

"data": [{

"0": "amazon",

"1": "amazon",

"timezoneOffset": "-330"

}],

"metadata": {

"count": 1

},

"actions": [{

"type": "answer",

"expression": "I don't know how to answer this. Here is a web search result:"

},

{

"type": "websearch",

"query": "amazon"

}]

}],

"answer_date": "2017-06-02T14:34:15.773Z",

"answer_time": 98,

"language": "en",

"session": {

"identity": {

"type": "host",

"name": "108.162.246.79",

"anonymous": true

}

}

}

After parsing this response, we first recognise the type of action that needs to be performed, here we get `websearch` which means we need to make a web search for the query. Here, we use `DuckDuckGo’s` API to get the result.

API Call to DuckDuckGo:

http://api.duckduckgo.com/?q=amazon&format=json

I am adding just the first object of the required data since the API response is too long.

Path: $.RelatedTopics[0]

{

 "Result": "<a href=\"https://duckduckgo.com/Amazon.com\">Amazon.com</a>Amazon.com, also called Amazon, is an American electronic commerce and cloud computing company...",

 "Icon": {

"URL": "https://duckduckgo.com/i/d404ba24.png",

"Height": "",

"Width": ""

},

 "FirstURL": "https://duckduckgo.com/Amazon.com",

 "Text": "Amazon.com Amazon.com, also called Amazon, is an American electronic commerce and cloud computing company..."

}

For the link preview, we need an image logo, URL and a description for the same so, here we will use the `Icon.URL` and `Text` key. We have our own class to parse this data into an object.

class WebsearchResult: NSObject {
   
   var image: String = "no-image"
   var info: String = "No data found"
   var url: String = "https://duckduckgo.com/"
   var query: String = ""
   
   init(dictionary: [String:AnyObject]) {
       
       if let relatedTopics = dictionary[Client.WebsearchKeys.RelatedTopics] as? [[String : AnyObject]] {
           
           if let icon = relatedTopics[0][Client.WebsearchKeys.Icon] as? [String : String] {
               if let image = icon[Client.WebsearchKeys.Icon] {
                   self.image = image
               }
           }
           
           if let url = relatedTopics[0][Client.WebsearchKeys.FirstURL] as? String {
               self.url = url
           }
           
           if let info = relatedTopics[0][Client.WebsearchKeys.Text] as? String {
               self.info = info
           }
           
           if let query = dictionary[Client.WebsearchKeys.Heading] as? String {
               let string = query.lowercased().replacingOccurrences(of: " ", with: "+")
               self.query = string
           }   
       }   
   }   
}

We now have the data and only thing left is to display it in the UI.

Within the Chat Bubble, we need to add a container view which will contain the image, and the text description.

 let websearchContentView = UIView()
    
    let searchImageView: UIImageView = {
        let imageView = UIImageView(frame: CGRect(x: 0, y: 0, width: 44, height: 44))
        imageView.contentMode = .scaleAspectFit

       // Placeholder image assigned
        imageView.image = UIImage(named: "no-image")
        return imageView
    }()
    
    let websiteText: UILabel = {
        let label = UILabel()
        label.textColor = .white
        return label
    }()

   func addLinkPreview(_ frame: CGRect) {
        textBubbleView.addSubview(websearchContentView)
        websearchContentView.backgroundColor = .lightGray
        websearchContentView.frame = frame
        
        websearchContentView.addSubview(searchImageView)
        websearchContentView.addSubview(websiteText)




       // Add constraints in UI
        websearchContentView.addConstraintsWithFormat(format: "H:|-4-[v0(44)]-4-[v1]-4-|", views: searchImageView, websiteText)
        websearchContentView.addConstraintsWithFormat(format: "V:|-4-[v0]-4-|", views: searchImageView)
        websearchContentView.addConstraintsWithFormat(format: "V:|-4-[v0(44)]-4-|", views: websiteText)
    }

Next, in the Collection View, while checking other action types, we add checking for `websearch` and then call the API there followed by adding frame size and calling the `addLinkPreview` function.

else if message.responseType == Message.ResponseTypes.websearch {
  let params = [
    Client.WebsearchKeys.Query: message.query!,
    Client.WebsearchKeys.Format: "json"
  ]

  Client.sharedInstance.websearch(params, { (results, success, error) in                      
    if success {
      cell.message?.websearchData = results
      message.websearchData = results
      self.collectionView?.reloadData()
      self.scrollToLast()
    } else {
      print(error)
    }
  })
                    
  cell.messageTextView.frame = CGRect(x: 16, y: 0, width: estimatedFrame.width + 16, height: estimatedFrame.height + 30)
  

 cell.textBubbleView.frame = CGRect(x: 4, y: -4, width: estimatedFrame.width + 16 + 8 + 16, height: estimatedFrame.height + 20 + 6 + 64)
                    
  let frame = CGRect(x: 16, y: estimatedFrame.height + 20, width: estimatedFrame.width + 16 - 4, height: 60 - 8)
  

 cell.addLinkPreview(frame)

}

And set the collection View cell’s size.

else if message.responseType == Message.ResponseTypes.websearch {
  return CGSize(width: view.frame.width, height: estimatedFrame.height + 20 + 64)
}

And we are done 🙂

Here is the final version how this would look like on the device:

 

Continue ReadingWebsearch and Link Preview support in SUSI iOS