Author: jyoth1raditya

This author has written 3 articles
Read more about the article Hotword Detection for SUSI Android with CMUsphinx

Hotword Detection for SUSI Android with CMUsphinx

Being an AI for conversational bots, Hotword detection of SUSI is the top priority to the community. Another requirement was that there should be an option for an offline hotword detection. So, I was searching for an API that has all these capabilities. Sphinx by CMU was the obvious choice. It provides robust mechanism for hotword detection.

What is CMUsphinx?

CMUsphinx is open source and leading speech recognition toolkit. CMUsphinx has different modules for different tasks it needs to perform. Our requirement for SUSI is, that is needs to be lightweight, So we are using Pocketsphinx. Before going into integration let us discuss about basics of speech recognition.

Let us dive into coding and integrating Susi with pocketsphinx.

Building Pocketsphinx .AAR file

Git clone the sphinxbase, pocketsphinx and pocketsphinx-android and put them in the same folder. By following commands below.

git clone http://github.com/cmupshinx/sphinxbase
git clone http://github.com/cmupshinx/pocketsphinx
git clone http://github.com/cmupshinx/pocketsphinx-android

Then import pocketsphinx Android into Android studio. Run the project. .aar files pocketsphinx-android-5prealpha-debug.aar & pocketsphinx-android-5prealpha-release.aar  will be created in the build/outputs/aar.

Integrating Susi with Pocketsphinx

In Android Studio you need to the above generated AAR into your project. Just go to File > New > New module and choose Import .JAR/.AAR Package. After this, We need to change permissions of project. Add the following permissions in AndroidManifest.xml.

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />

Import the following functions into your main activity.

import edu.cmu.pocketsphinx.Assets;
import edu.cmu.pocketsphinx.Hypothesis;
import edu.cmu.pocketsphinx.RecognitionListener;
import edu.cmu.pocketsphinx.SpeechRecognizer;
import edu.cmu.pocketsphinx.SpeechRecognizerSetup;

Next we need to sync the assets we get from .aar file in to our project. Edit app/build.gradle build file to run assets.xml. We do it by adding following code to build.gradle.

ant.importBuild 'assets.xml'
preBuild.dependsOn(list, checksum)
clean.dependsOn(clean_assets)

Now all the import and sync errors of gradle must disappear and you should be good to go. You can start your recognizer by adding this code to your activity.

recognizer = defaultSetup()
        .setAcousticModel(new File(assetsDir, "en-us-ptm"))
        .setDictionary(new File(assetsDir, 
"cmudict-en-us.dict"))
        .getRecognizer();
recognizer.addListener(this);

Decoder model is lengthy process that contains many operations, so it’s recommended to run in inside async task. These are commands for decoder to run. These commands essentially do acoustic and language modelling of speech.

// Create keyword-activation search.
recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);

// Create grammar-based searches.
File menuGrammar = new File(assetsDir, "menu.gram");
recognizer.addGrammarSearch(MENU_SEARCH, menuGrammar);

// Next search for digits
File digitsGrammar = new File(assetsDir, "digits.gram");
recognizer.addGrammarSearch(DIGITS_SEARCH, digitsGrammar);

// Create language model search.
File languageModel = new File(assetsDir, "weather.dmp");
recognizer.addNgramSearch(FORECAST_SEARCH, languageModel);

Speech recognition will end at onEndOfSpeech callback of the recognizer listener.  We can call recognizer.stop or recognizer.cancel(). Cancel will cancel the recognition, stop will cause the final result be passed you in onResult callback. During the recognition, you will get partial results in onPartialResult callback.

Now we have integrated Pocketsphinx with SUSI.AI in Android.

Continue ReadingHotword Detection for SUSI Android with CMUsphinx

Auto Deploying accounts.susi.ai on gh-pages

While migrating web apps from susi server repository to accounts.susi.ai repository. Our team autodeployed accounts.susi.ai on gh-pages branch i.e., each time changes are made to code, it gets auto deployed on gh-pages branch. And changes are automatically live and visible on site.

To do this we have to setup travis in our repository. Travis can be easily set in github repository just by adding .travis.yml file into root directory your repo. Depending on type of repository you are dealing with, the configuration of travis changes. For accounts.susi.ai, It is a written on top of ReactJs framework. So we used the following code.

sudo: required
dist: trusty
language: node_js
node_js:
  - 6
script:
  - npm test
deploy:
  provider: script
  script: "./deploy.sh"
  env:
  global:
  - ENCRYPTION_LABEL: "<.... encryption label from previous step ....>"
  - COMMIT_AUTHOR_EMAIL: "you@example.com"
cache:
  directories:
    - node_modules
branches:
  only:
    - master

 

The above travis configuration file, After every commit checks whether the build is passing or not by running npm test command. After checking the commit, next script deploy.sh is run on the commit. The last line tells us to only execute this script for commits in master branch. deploy.sh can be placed anywhere, we jus need to modify its path in .tavis.yml file. Here is code of deploy.sh :

#!/bin/bash
set -e 

SOURCE_BRANCH="master"
TARGET_BRANCH="gh-pages"

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]; then
    echo "Skipping deploy; just doing a build."
    doCompile
    exit 0
fi
# Save some useful information
REPO=`git config remote.origin.url`
SSH_REPO=${REPO}
SHA=`git rev-parse --verify HEAD`

git clone $REPO out
cd out
git checkout $TARGET_BRANCH || git checkout --orphan $TARGET_BRANCH
cd ..
git config user.name "Travis CI"
git config user.email "travis-ci@github.com"

if git diff --quiet; then
    echo "No changes to the output on this push; exiting."
    exit 0
fi

# Commit the "changes", i.e. the new version.
# The delta will show diffs between new and old versions.
git add -A .
git commit -m "Deploy to GitHub Pages: ${SHA}"

# Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc
ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key"
ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv"
ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR}
ENCRYPTED_IV=${!ENCRYPTED_IV_VAR}
openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in ../deploy_key.enc -out ../deploy_key -d
chmod 600 ../deploy_key
eval `ssh-agent -s`
ssh-add deploy_key

# Now that we're all set up, we can push.
git push $SSH_REPO $TARGET_BRANCH

 

deploy.sh is automatically deploying master on gh pages branch. To perform this task, it needs admin authorisation of github repo. We do this authentication by encrypted keys.
To create encrypted keys we need to generate a new SSH key, SSH keys can be generated by running following command in terminal.

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

 

Then add this key to your project’s git repository at the github repo of project.

https://github.com/<your name>/<your repo>/settings/keys

Now, We can generate encrypted keys by running following command.

travis encrypt-file deploy_key

 

This command generates deploy_key.enc file, which shall be placed in the repo. Its location needs to be updated in the placeholders of .travis.yml and deploy.sh.

Hence, we achieved the task of auto deployment to gh-pages in accounts.susi.ai. Please ensure that account through which encrypted keys are created, always have admin access to repository for continued deployment.

Continue ReadingAuto Deploying accounts.susi.ai on gh-pages
Read more about the article Deploying SUSI.AI with Docker

Deploying SUSI.AI with Docker

Docker is much more efficient than VM in allocating shared resources between various containers as shown in figure. To deploy SUSI we need to create docker container. There are two ways to build it. First way is fork the SUSI project in github. Then you can signup in dockerhub and create autobuild docker container. The second way is to manually build docker file from command prompt of your computer. The following instructions needs to be executed in cloud shell or linux machine.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get -y install docker.io
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo docker build https://github.com/fossasia/susi_server.git


The first three commands install docker software to the machine. Next three lines give required permissions and execution abilities to docker. Final command builds docker container from github. Thus, We have successfully made docker container. We can deploy it on the cloud by using following command.

sudo docker run -d -p 80:80 -p 443:443 susi


Deploying Susi on cloud with kubernetes

We will use Google Cloud Service platform for demonstration. Create your GCS account. Goto dashboard and click on compute engine.

Enable Billing and Create New Project named XYZ. Open the terminal, by clicking the Google cloud shell button on the top.

Please set the compute zone to your nearest zone by running the below command.

gcloud config set compute/zone us-central1-a


Now we need to create cluster, on which we deploy susi app. We do it by this command.

gcloud container clusters create hello-cluster --num-nodes=3


We need to get docker from dockerhub and push it to our project repo.We do it by these commands.

sudo docker pull jyothiraditya/susi_server
gcloud docker -- push <image-id> gcr.io/<project-id>/<name>

We run the docker image on cluster by following commands.

kubectl run susi-server --image=gcr.io/<project-id>/<name> --port=80
kubectl get pods

We expose the container to external traffic, with help of load balancer by the following command.

kubectl expose deployment susi-server --type="LoadBalancer --port=80"


We get the external ip-address to access susi from browser. By entering

kubectl get service susi-server


You can now view the app by going to “EXTERNAL-IP:80”.

References : Docker build , Kubernetes deployment, Google cloud deployment 

 

Continue ReadingDeploying SUSI.AI with Docker