Modifying Finite State Architecture On SUSI Linux to Process Multiple Queries

During the initial stages of SUSI Linux: As the code base grew, it was getting very difficult to manage code, so we opted to implement a Finite State Architecture in our repo. But, as there were new features implemented in the Repo, we realized that we couldn’t process more than one query at a time which restricted a lot of features. eg. The smart speaker was converted to a simple Bluetooth speaker since no response regarding playing/pausing were accepted.

To solve this issue, we made a slight modification in the architecture.

Brief About SUSI States

SUSI is working as a Finite State Machine and is present in 3 states namely IDLE state, Recognising state and Busy state. The State Machine executes in the following order.

  1. IDLE State:
    When the SUSI state Machine is in this State, SUSI is searching for the hotword “SUSI”, waiting to trigger the complete Machine.
  2. Recognizing State

In this State , the State Machine has started the STT client. After recognition, SUSI sends the query to the Server awaiting the response

  1. Busy State

After the response has been received, the TTS client is triggered and the answer is given out by SUSI

Adding a Second Hotword Recognition Class

Now, to allow SUSI to process the second query, The State machine must be triggered while SUSI is giving out the first response and to trigger the State Machine, we must have hotword recognition while SUSI is speaking the answer to the previous query. Hence, a hotword recognition engine is now initiated every time the State Machine enters the busy state.

We will be using Snowboy as Hotword Detection Engine.

 

import os
TOP_DIR = os.path.dirname(os.path.abspath(__file__))
RESOURCE_FILE = os.path.join(TOP_DIR, “susi.pmdl”)
class StopDetector():
   “””This implements the Stop Detection with Snowboy Hotword Detection Engine.”””
    def __init__(self, detection) -> None:
       super().__init__()
       self.detector = snowboydecoder.HotwordDetector(
           RESOURCE_FILE, sensitivity=0.6)
       self.detection = detection
    def run(self):
       “”” Implementation of run abstract method in HotwordDetector. This method is called when thread is
started for the first time. We start the Snowboy detection and declare detected callback as
       detection_callback method declared ina parent class.
       “””
       self.detector.start(detected_callback=self.detection)

 

Now, this class takes the Callback function as a parameter which is passed when the transition to busy state takes place from the recognition state.

 

Modifying the State Machine Architecture

After declaring a second hotword recognition engine , we must modify how the transitions take place between the States of the SUSI State Machine.

Hence the callback that will be triggered is passed from the busy state.

 

def detection(self):
       “””This callback is fired when a Hotword Detector detects a hotword.
       :return: None
       “””
       if hasattr(self, ‘video_process’):
           self.video_process.send_signal(signal.SIGSTOP)
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.off()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.video_process.send_signal(signal.SIGCONT)
       if hasattr(self, ‘audio_process’):
           self.audio_process.send_signal(signal.SIGSTOP)  
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.wakeup()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.audio_process.send_signal(signal.SIGCONT)

 

As soon as the hotword is detected ,the state machine makes transitions to the Recognition State while pausing the current Music and resumes the Music after the second query has been completed.

 

This is how SUSI processes multiple queries simultaneously while still maintaining finite state archi.

 

Additional Resources

Tags

gsoc, gsoc’18, finite_state_machine, susi_linux, multiple_query, susi.ai, susi

 

Continue ReadingModifying Finite State Architecture On SUSI Linux to Process Multiple Queries