Modifying Finite State Architecture On SUSI Linux to Process Multiple Queries
During the initial stages of SUSI Linux: As the code base grew, it was getting very difficult to manage code, so we opted to implement a Finite State Architecture in our repo. But, as there were new features implemented in the Repo, we realized that we couldn’t process more than one query at a time which restricted a lot of features. eg. The smart speaker was converted to a simple Bluetooth speaker since no response regarding playing/pausing were accepted.
To solve this issue, we made a slight modification in the architecture.
Brief About SUSI States
SUSI is working as a Finite State Machine and is present in 3 states namely IDLE state, Recognising state and Busy state. The State Machine executes in the following order.
- IDLE State:
When the SUSI state Machine is in this State, SUSI is searching for the hotword “SUSI”, waiting to trigger the complete Machine. - Recognizing State
In this State , the State Machine has started the STT client. After recognition, SUSI sends the query to the Server awaiting the response
- Busy State
After the response has been received, the TTS client is triggered and the answer is given out by SUSI
Adding a Second Hotword Recognition Class
Now, to allow SUSI to process the second query, The State machine must be triggered while SUSI is giving out the first response and to trigger the State Machine, we must have hotword recognition while SUSI is speaking the answer to the previous query. Hence, a hotword recognition engine is now initiated every time the State Machine enters the busy state.
We will be using Snowboy as Hotword Detection Engine.
import os TOP_DIR = os.path.dirname(os.path.abspath(__file__)) RESOURCE_FILE = os.path.join(TOP_DIR, “susi.pmdl”) class StopDetector(): “””This implements the Stop Detection with Snowboy Hotword Detection Engine.””” def __init__(self, detection) -> None: super().__init__() self.detector = snowboydecoder.HotwordDetector( RESOURCE_FILE, sensitivity=0.6) self.detection = detection def run(self): “”” Implementation of run abstract method in HotwordDetector. This method is called when thread is started for the first time. We start the Snowboy detection and declare detected callback as detection_callback method declared ina parent class. “”” self.detector.start(detected_callback=self.detection) |
Now, this class takes the Callback function as a parameter which is passed when the transition to busy state takes place from the recognition state.
Modifying the State Machine Architecture
After declaring a second hotword recognition engine , we must modify how the transitions take place between the States of the SUSI State Machine.
Hence the callback that will be triggered is passed from the busy state.
def detection(self): “””This callback is fired when a Hotword Detector detects a hotword. :return: None “”” if hasattr(self, ‘video_process’): self.video_process.send_signal(signal.SIGSTOP) lights.wakeup() subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])]) lights.off() self.transition(self.allowedStateTransitions.get(‘recognizing’)) self.video_process.send_signal(signal.SIGCONT) if hasattr(self, ‘audio_process’): self.audio_process.send_signal(signal.SIGSTOP) lights.wakeup() subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])]) lights.wakeup() self.transition(self.allowedStateTransitions.get(‘recognizing’)) self.audio_process.send_signal(signal.SIGCONT) |
As soon as the hotword is detected ,the state machine makes transitions to the Recognition State while pausing the current Music and resumes the Music after the second query has been completed.
This is how SUSI processes multiple queries simultaneously while still maintaining finite state archi.
Additional Resources
- Info About Finite State Architecture: https://en.wikipedia.org/wiki/Finite-state_machine
- Official Documentation Of Snowboy: https://snowboy.kitt.ai/docs
- Info About SIGSTOP and SIGCONT: https://major.io/2009/06/15/two-great-signals-sigstop-and-sigcont/
Tags
gsoc, gsoc’18, finite_state_machine, susi_linux, multiple_query, susi.ai, susi