Till now, SUSI smart speaker was working only as an online model like the other speakers in the market. For the first time, we have introduced a feature which allows the speaker to work offline. We deployed the server on the hardware itself and also provide the option of an online server as a fallback.

 

The Offline Support was implemented in the following steps

 

Step 1: Deploying SUSI Server Locally

 

Firstly , configure a bash script to allow automatic deployment of the server along with the initialization of the susi_linux script.

 

echo “Deploying local server” if  [ ! -e “susi-server” ] then    git clone https://github.com/fossasia/susi_server.git fi if [ -e “susi_server” ] then        cd susi_server    git submodule update –recursive –remote    git submodule update –init –recursive    ./gradlew build    bin/start.sh fi

 

The above builds the server and deploys it on ‘localhost:4000’.

 

Then, add the following test on SUSI Linux wrapper to check if the local server is up and running. Using the local server not adds an offline support but also increases the efficiency by around 30%.

def check_local_server():    test_params = {        ‘q’: ‘Hello’,        ‘timezoneOffset’: int(time.timezone / 60)    }    try:        chat_url = ‘http://localhost:4000/susi/chat.json’        if (requests.get(chat_url, test_params)):            print(‘connected to local server’)            global api_endpoint            api_endpoint = ‘http://localhost:4000’    except requests.exceptions.ConnectionError:        print(‘local server is down’) check_local_server()

 

As shown above, this is a test checking for the local server. If the local server is down, the online server is chosen as a fallback

 

Step 2: Adding an Offline STT Service

Now, that we are able to process a query offline. We must have a way in which, we can recognize the user’s voice commands without using the internet. For that, we use the service of PocketSphinx. But first, we check if the internet is available or not

 

def internet_on():        try:            urllib2.urlopen(‘http://216.58.192.142’, timeout=1)  # nosec #pylint-disable type: ignore            return True  # pylint-enable        except urllib2.URLError as err:            print(err)            return False

 

If the internet connection is available, we use the online STT service which is Google STT ( default) and switch over to PocketSphinx in case the internet connection is not available.

 

Step 3: Adding the Offline TTS service

Finally, we’ll need an offline TTS service which will help us turn SUSI’s response to voice commands. We’ll be using a service called flite TTS as our offline TTS.

 

elif payload == ‘ConnectionError’:             self.notify_renderer(‘error’, ‘connection’)                                  self.notify_renderer(‘error’, ‘connection’)             config[‘default_tts’] = ‘flite’             os.system(‘play extras/connect-error.wav’)

 

We check if there is a ConnectionError, and then we switch to flite TTS after play an error query

 

Final Output:

We now get a Smart Speaker which is functional without any internet connection.

 

References

• https://cmusphinx.github.io/wiki/sphinxinaction/
• http://www.festvox.org/flite/
• https://github.com/fossasia/susi_linux 

Tags

 

Fossasia, susi, gsoc, gsoc’18, offline_tts , offline_stt ,flite , pocketsphinx