Make a helper for AJAX requests using axios

In this blog post, we are going to go through the implementation of the helper function that is created for making AJAX requests using axios. Currently, the AJAX calls are made in a very random manner where the calls are written in the component itself and involves rewriting of headers, etc for the API call. Let us go through the implementation in the blog, which will standardise the way to make API calls in SUSI.AI web clients.

The primary changes are –

  • Making a common helper for AJAX requests with the help of axios.
  • Making a common file containing all the API calls across the project.

Going through the implementation

The API calls within the repository were not being made in an organised way, also a lot of redundant code was present. The aim of creating the helper is that, all the API calls is called via this common function. It takes care of the headers and also sending access_token with the API if the user is already logged in for API calls requiring authentication. The function for a API request now looks this simple –

// API call for signup
export function getSignup(payload) {
 const { email, password } = payload;
 const url = `${API_URL}/${AUTH_API_PREFIX}/signup.json`;
 return ajax.get(url, { signup: email, password });
}
  • In the above snippet, the ajax is the common helper used for making API calls. ajax is an object of functions that returns a promise for  various methods of API requests
  • We have primarily taken into consideration GET & POST requests type.
  • The helper function is as follows –
/* Insert imports here*/
const cookies = new Cookies();
const obj = {};

['get', 'post', 'all'].forEach(function(method) {
 obj[method] = function(url, payload, settings = {}) {
   /* Request will be aborted after 30 seconds */
   settings = {
     timeout: 30000,
     dataType: 'json',
     crossDomain: true,
     ...settings,
   };
   
   // Check if logged in
   if (cookies.get('loggedIn')) {
     payload = {
       access_token: cookies.get('loggedIn'),
       ...payload,
     };
   }

   return new Promise(function(resolve, reject) {
     let methodArgs = [];
     if (method === 'post') {
       if (payload && payload instanceof FormData !== true) {
           // Convert to Form Data
           payload = toFormData(payload);
       }

       settings.headers = {
         'Content-Type': 'application/x-www-form-urlencoded',
         ...settings.headers,
       };
     } else if (method === 'get') {
       if (payload) {
         // Add params to the URL   
         url += `?${Object.keys(payload)
           .map(key => key + '=' + payload[key])
           .join('&')}`;
       }
     }

     const methodsToAxiosMethodsMap = {
       get: 'get',
       post: 'post',
       all: 'all',
     };

     if (method === 'all') {
       methodArgs = [url];
     } else if (method === 'get') {
       methodArgs = [url, settings];
     } else {
       methodArgs = [url, payload, settings];
     }

     axios[methodsToAxiosMethodsMap[method]].apply({}, methodArgs).then(
       function(data = {}, ...restSuccessArgs) {
         const statusCode = _.get(data, 'status');
         /*  Send only api response */
         let responseData = { statusCode, ..._.get(data, 'data') };

         if (method === 'all') {
           responseData = data;
           responseData.statusCode = statusCode;
         }

         if (payload) {
           responseData.requestPayload = payload;
         }
         // Mark the promise resolved and return the payload
         resolve(camelizeKeys(responseData), ...restSuccessArgs);
       },
       function(data = {}, ...restErrorArgs) {
         // If request is canceled by user
         if (axios.isCancel(data)) {
           reject(data);
         }

         const statusCode = _.get(data, 'response.status', -1);
         let responseData = { statusCode, ..._.get(data, 'response.data') };

         if (method === 'all') {
           responseData = data;
           responseData.statusCode = statusCode;
         }

         if (payload) {
           responseData.requestPayload = payload;
         }
         // Mark the promise rejected and return the payload
         reject(camelizeKeys(responseData), ...restErrorArgs);
       },
     );
   });
 };
});

export default obj;
  • The above objects contains 3 functions –
    • ajax.get(url, payload, settings) – GET – The helper adds the query params to the URL by iterating through them and appending it to the URL and joining them with &.
    • ajax.post(url, payload, settings) –  POST – The helper checks, if the POST requests contains a payload which is an instance of form data, it converts to toFormData payload.
    • ajax.all(url, payload, settings) – ALL – It helps to deal with concurrent requests.
  • The url is the complete API endpoint, payload consists of the data/request payload. The settings consists of any headers related info, that needs to be added exclusively to the axios config.
  • There is also no need to pass access token to each API request as a payload. The helper check whether the user is logged-in and adds the access_token to the request payload. The snippet below demonstrates it –
if (cookies.get('loggedIn')) {
     payload = {
       access_token: cookies.get('loggedIn'),
       ...payload,
     };
   }
  • The access token, if present in the cookies is added to the payload, therefore authenticating the API call.
  • The keys of the response are changed to camel case before being sent to the caller function. It is done to maintain variable nomenclature standards across the app and also to follow javascript guidelines.
  • The file containing the API calls is structured as follows –
import ajax from '../helpers/ajax';
import urls from '../utils/urls';

const { API_URL } = urls;
const AUTH_API_PREFIX = 'aaa';
const CHAT_API_PREFIX = 'susi';
const CMS_API_PREFIX = 'cms';

// API without request payload
export function fetchApiKeys() {
 const url = `${API_URL}/${AUTH_API_PREFIX}/getApiKeys.json`;
 return ajax.get(url);
}

// API with request payload
export function getLogin(payload) {
 const { email, password } = payload;
 const url = `${API_URL}/${AUTH_API_PREFIX}/login.json`;
 return ajax.get(url, { login: email, password, type: 'access-token' });
}

The above was the implementation of the helper function that is created for making AJAX requests using axios. I hope the blog provided a detailed insight of it helps in making the process of making API calls more standardised and easier.

References

Continue ReadingMake a helper for AJAX requests using axios

Adding different metrics sections to the start page

In the initial version of the SUSI.AI Skill CMS we simply displayed all the skills present in the system in the form of cards. Once the skill analytics was incorporated into the CMS we got a bunch of skill statistics and thus we enhanced the start page by incorporating horizontally scrollable skill cards as per skill metrics like top rated skills, most used skills, skills which have received the most feedback etc. I worked on adding the skills with most feedback section and the section for the top games. This post will majorly deal with how the metrics sections are implemented on the start page and how any new metrics can be incorporated into the system and thus displayed on the CMS.

About the API

/cms/getSkillMetricsData.json?language=${language}

Sample API call:

https://api.susi.ai/cms/getSkillMetricsData.json?language=en

 

This will return a JSON which contains the skill data for all the metrics.

{
 "accepted": true,
 "model": "general",
 "group": "All",
 "language": "en",
 "metrics": {
        "newest": [...],
     "rating": [...],
      ...
 }
 "message": "Success: Fetched skill data based on metrics",
   "session": {"identity": {
           "type": "host",
          "name": "162.158.23.7_68cefd16",
          "anonymous": true
   }}
}

 

All of the data for several metics comes from the metrics object of the response which in turn contains arrays of skill data for each metric.

CMS Implementation

Once the BrowseSkill component is mounted we make an API call to the server to fetch all the data and save it to the component state, this data is then fed to the ScrollCardList component as props and the scroll component is rendered with appropriate data for different metrics.

loadMetricsSkills = () => {
   let url;
   url =
           urls.API_URL +
           '/cms/getSkillMetricsData.json?language=' +
           this.state.languageValue;
   let self = this;
   $.ajax({
           url: url,
           dataType: 'jsonp',
           jsonp: 'callback',
           crossDomain: true,
           success: function(data) {
                   self.setState({
                           skillsLoaded: true,
                           staffPicksSkills: data.metrics.staffPicks,
                           topRatedSkills: data.metrics.rating,
                           topUsedSkills: data.metrics.usage,
                           latestUpdatedSkills: data.metrics.latest,
                           newestSkills: data.metrics.newest,
                           topFeedbackSkills: data.metrics.feedback,
                           topGames: data.metrics['Games, Trivia and Accessories'],
                   });
           },
           error: function(e) {
                   console.log('Error while fetching skills based on top metrics', e);
                   return self.loadMetricsSkills();
           },
   });
};

 

We are using a single component for skill metrics and skill listing which show up on applying any filter or visiting any category. Thus we think of a condition when the skill metrics are to be displayed and conditionally render the metrics section depending on the condition.

So the metrics section shows up only when we have not visited any category or language page, there’s no search query in the search bar, there’s no rating refine filter applied and no time filter applied.

let metricsHidden =
         this.props.routeType ||
         this.state.searchQuery.length > 0 ||
         this.state.ratingRefine ||
         this.state.timeFilter;

 

Depending on the section you want to display, pass appropriate data as props to the SkillCardScrollList component, say we want to display the section with most feedback

{this.state.topFeedbackSkills.length &&
!metricsHidden ? (
   <div style={metricsContainerStyle}>
           <div
                   style={styles.metricsHeader}
                   className="metrics-header"
           >
                   <h4>
                           {'"SUSI, what are the skills with most feedback?"'}
                   </h4>
           </div>
           {/* Scroll Id must be unique for all instances of SkillCardList*/}
           {!this.props.routeType && (
                   <SkillCardScrollList
                           scrollId="topFeedback"
                           skills={this.state.topFeedbackSkills}
                           modelValue={this.state.modelValue}
                           languageValue={this.state.languageValue}
                           skillUrl={this.state.skillUrl}
                   />
           )}
   </div>
) : null}

 

So if there are skills preset in the topFeedbackSkills array which was saved in the state from the server initially and the condition to hide metrics is false we render the component and pass appropriate props for scrollId, skills data, language and model values and skill url.

In a similar way any metrics section can be implemented in the CMS, if the data is not present in the API, modify the endpoint to enclose the data you need, fetch data data from the server and just render it.

So I hope after reading through this you have a more clearer understanding about how the metrics sections are implemented on the CMS.

Resources

Continue ReadingAdding different metrics sections to the start page

A Workflow of Auto Executing Services on SUSI.AI Smart Speaker

As we plan to create a headless client on RaspberryPi, the requirement was that the SUSI.AI programs should run automatically. To do so, we had to figure out a way to boot up various scripts on startup.

We had the following options to execute the scripts on startup:

  1. Editing Rc.local file
  2. Systemd Rules
  3. Crontab

We decided to proceed with Systemd Rules because using Rc.local and Crontab requires modifying the default system files which in case of any error would make the os functionalities to crash very soon.

We then created the SystemD rules for the following services:

1.factory-daemon.service
2. python-flask.service
3. susi-server.service
4. update-daemon.service
5. susi-linux.service

Now I’ll demonstrate the working and the functionality of each service being implemented.

1. Factory-Daemon Service

This service initiates the factory daemon with the raspberry Pi startup and then keeps it running continuously looking for any input from the GPiO port.

[Unit]
Description=SUSI Linux Factory Daemon
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3 /home/pi/SUSI.AI/susi_linux/factory_reset/factory_reset.py

[Install]
WantedBy=multi-user.target

2. Python-Flask Service

This service starts a python Server to allow handshake between mobile apps and the Smart Speaker which will allow the user to configure SUSI Smart Speaker accordingly.

[Unit]
Description=Python Server for SUSI Linux
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3  /home/pi/SUSI.AI/susi_linux/access_point/server/server.py

[Install]
WantedBy=multi-user.target

3.SUSI-Server Service

This service starts the Local SUSI Server as soon as the Raspberry Pi starts up which in turn allows the SUSI Linux programs to fetch responses of queries very quickly.

[Unit]
Description=Starting SUSI Server for SUSI Linux
After=multi-user.target

[Service]
Type=oneshot
ExecStart=/home/pi/SUSI.AI/susi_linux/susi_server/susi_server/bin/restart.sh

[Install]
WantedBy=multi-user.target

4. Update-Daemon Service

This Service creates a Daemon which starts with the Raspberry Pi and fetches the latest updates from the repository from the upstream branch.

[Unit]
Description=Update Check- SUSI Linux
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/home/pi/SUSI.AI/susi_linux/update_daemon/update_check.sh

[Install]
WantedBy=multi-user.target

5. SUSI-Linux Service

This Service finally runs the main SUSI Linux software after everything has started.

[Unit]
Description=Starting SUSI Linux
Wants=network-online.target
After=network-online.target

[Service]
Type=idle
WorkingDirectory=/home/pi/SUSI.AI/susi_linux/
ExecStart=/usr/bin/python3 -m main

[Install]
WantedBy=multi-user.target

This blog gives a brief workflow of auto-executing services on SUSI Smart Speaker.

Resources

Continue ReadingA Workflow of Auto Executing Services on SUSI.AI Smart Speaker

Configuring LED Lights with SUSI Smart Speaker

To make the SUSI Smart Speaker more interactive and to improve the visual aesthetics, we configured SUSI Smart Speaker’s response with 3 RGB led lights. We have used a new piHat as an external hardware to configure the LEDs.

Now the new hardware specs of the SUSI Smart Speaker are:

  1. Raspberry Pi
  2. ReSpeaker PiHat 2 Mic Array
  3. External Speakers

Using an external PiHat not only added the RGB light functionality but also eliminated the need to use a USB microphone and configured a factory reset button

Configuring the PiHat as the default Audio driver

To Use the PiHat as the default input driver, we use the package called PulseAudio.

And we use the following command in the installation script.

pacmd set-sink-port alsa_output.platform-soc_sound.analog-stereo analog-output-headphones

Configuring PiHat’s GPIO Button with Factory Reset

There is an onboard User Button, which is connected to GPIO17. We use the python library RPi.GPIO to detect the user button. The python script is used in the following way

GPIO.setmode(GPIO.BCM)
GPIO.setup(17,GPIO.IN)
i = 1
while True:
if GPIO.input(17) == 1:
       time.sleep(0.1)
       pass
   elif GPIO.input(17) == 0 :
       start = time.time()
       while GPIO.input(17) == 0 :
           time.sleep(0.1)
       end = time.time()
       total = end – start
       if total >= 7 :
           subprocess.call([‘bash’,‘factory_reset.sh’])  # nosec #pylint-disable type: ignore
       else :
           mixer = alsaaudio.Mixer()
           value = mixer.getvolume()[0]
           if value != 0:
               mixer.setvolume(0)
           else:
               mixer.setvolume(50)
       print(total)
       time.sleep(0.1)

 

This script checks on the button which is configured on GPIO port 17 on the PiHat. If the button is pressed for than 7 secs, the factory reset process takes place, else the device is muted.

Configuring PiHat’s LED with Speaker’s Response

We use a python library called SPIDEV to sync the LED lights with SUSI’s response. SPIDEV is usually used to send a response to the bus devices on the Raspberry Pi.

The first step was installing spidev

sudo pip install spidev

Now we create a class where we store all the methods where we send the signal to the bus port. We treat the LED lights as a circular array and then have a rotation of RGB lights

class LED_COLOR:
    # Constants
   MAX_BRIGHTNESS = 0b11111    LED_START = 0b11100000
    def __init__(self, num_led, global_brightness=MAX_BRIGHTNESS,
                order=‘rgb’, bus=0, device=1, max_speed_hz=8000000):
       self.num_led = num_led
       order = order.lower()
       self.rgb = RGB_MAP.get(order, RGB_MAP[‘rgb’])
       if global_brightness > self.MAX_BRIGHTNESS:
           self.global_brightness = self.MAX_BRIGHTNESS
       else:
           self.global_brightness = global_brightness
        self.leds = [self.LED_START, 0, 0, 0] * self.num_led
       self.spi = spidev.SpiDev()
       self.spi.open(bus, device)
       if max_speed_hz:
           self.spi.max_speed_hz = max_speed_hz
    def clear_strip(self):
        for led in range(self.num_led):
           self.set_pixel(led, 0, 0, 0)
       self.show()
    def set_pixel(self, led_num, red, green, blue, bright_percent=100):
       if led_num < 0:
           return          if led_num >= self.num_led:
           return
       brightness = int(ceil(bright_percent * self.global_brightness / 100.0))
       ledstart = (brightness & 0b00011111) | self.LED_START
        start_index = 4 * led_num
       self.leds[start_index] = ledstart
       self.leds[start_index + self.rgb[0]] = red
       self.leds[start_index + self.rgb[1]] = green
       self.leds[start_index + self.rgb[2]] = blue
    def set_pixel_rgb(self, led_num, rgb_color, bright_percent=100):
       self.set_pixel(led_num, (rgb_color & 0xFF0000) >> 16,
                      (rgb_color & 0x00FF00) >> 8, rgb_color & 0x0000FF, bright_percent)
    def rotate(self, positions=1):
       cutoff = 4 * (positions % self.num_led)
       self.leds = self.leds[cutoff:] + self.leds[:cutoff]
    def show(self):
       data = list(self.leds)
       while data:
           self.spi.xfer2(data[:32])
           data = data[32:]
       self.clock_end_frame()
    def cleanup(self):
       self.spi.close()  # Close SPI port
    def wheel(self, wheel_pos):
       “””Get a color from a color wheel; Green -> Red -> Blue -> Green”””
        if wheel_pos > 255:
           wheel_pos = 255  # Safeguard
       if wheel_pos < 85:  # Green -> Red
           return self.combine_color(wheel_pos * 3, 255 – wheel_pos * 3, 0)
       if wheel_pos < 170:  # Red -> Blue
           wheel_pos -= 85
           return self.combine_color(255 – wheel_pos * 3, 0, wheel_pos * 3)
       wheel_pos -= 170
       return self.combine_color(0, wheel_pos * 3, 255 – wheel_pos * 3)

 

Now we use the threading to create non-blocking code which will allow SUSI to send response as well as change the LED’s simultaneously.

class Lights:
   LIGHTS_N = 3
    def __init__(self):
        self.next = threading.Event()
       self.queue = Queue.Queue()
       self.thread = threading.Thread(target=self._run)
       self.thread.daemon = True
       self.thread.start()
    def wakeup(self, direction=0):
       def f():
           self._wakeup(direction)
        self.next.set()
       self.queue.put(f)
    def listen(self):
       self.next.set()
       self.queue.put(self._listen)
    def think(self):
       self.next.set()
       self.queue.put(self._think)
    def speak(self):
       self.next.set()
       self.queue.put(self._speak)
    def off(self):
       self.next.set()
       self.queue.put(self._off)

This is how LED lights are configured with SUSI’s response

Resources

Additional Resources

Continue ReadingConfiguring LED Lights with SUSI Smart Speaker

Connecting the Smart Speaker with Mobile Clients

The beauty of SUSI Smart Speaker lies in it being customizable according to the user’s needs. And we allow the user to customize it by providing an interface through the mobile clients. To do so, we create a local server on the Raspberry Pi itself. The Raspberry Pi is started in an Access Point mode and the mobile clients hit the endpoints in a specific order and then the configuration is sent to the server and stored according to the user.

 

The following API’s are required to be executed by the mobile clients

1> /speaker_config

2> /wifi_credentials

3> /auth

4> /config

 

The following is the order of API execution

1. /speaker_config

This endpoint only takes the room name as a parameter. And then send send to the server to store the location of the device under the user’s account

def speaker_config():
   room_name = request.args.get(‘room_name’)
   config = json_config.connect(config_json_folder)
   config[‘room_name’] = rogom_name

 

2. /wifi_credentials

This endpoint takes the wifi ssid and wifi password as the parameters and then stores it in the raspberry Pi wifi config file.

 

def wifi_config():
   wifi_ssid = request.args.get(‘wifissid’)
   wifi_password = request.args.get(‘wifipassd’)
   subprocess.call([‘sudo’, ‘bash’, wifi_search_folder + ‘/wifi_search.sh’, wifi_ssid, wifi_password])
   display_message = {“wifi”:“configured”, “wifi_ssid”:wifi_ssid, “wifi_password”: wifi_password}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

Now the script wifi_search is called which stores the wifi credentials in the wifi_config file using the following command

 

cat >> /etc/wpa_supplicant/wpa_supplicant.conf <<EOF
network={
   ssid=“$SSID”
   psk=“$PSK”
}
EOF

 

3. /auth

This endpoint takes the SUSI’s login credentials as parameters, i.e. the registered email id and the corresponding password.

 

def login():
   auth = request.args.get(‘auth’)
   email = request.args.get(’email’)
   password = request.args.get(‘password’)
   subprocess.call([‘sudo’, ‘bash’, access_point_folder + ‘/login.sh’, auth, email, password])
   display_message = {“authentication”:“successful”, “auth”: auth, “email”: email, “password”: password}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

4. /config

Finally, this endpoint takes the stt, tts, hotword detection engine and wake button as the parameters and configures the speaker accordingly.

 

def config():
   stt = request.args.get(‘stt’)
   tts = request.args.get(‘tts’)
   hotword = request.args.get(‘hotword’)
   wake = request.args.get(‘wake’)
   subprocess.Popen([‘sudo’, ‘bash’, access_point_folder + ‘/config.sh ‘, stt, tts, hotword, wake])
   display_message = {“configuration”:“successful”, “stt”: stt, “tts”: tts, “hotword”: hotword, “wake”:wake}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

Now, this function runs a script called config.sh which in turn runs a script called rwap.sh to convert the Raspberry Pi to normal mode and then finally start SUSI on startup.

 

#!/bin/bash

if [ $EUID -ne 0 ]
then echo “Must be root”
exit
fi

cd /etc/hostapd/
sed -i ‘1,14d’ hostapd.conf

cd /etc/
sed -i ‘57,60d’ dhcpcd.conf

cd /etc/network/
sed -i ‘9,17d’ interfaces

echo “Please reboot”
sudo reboot

 

After successfully hitting all the endpoint from the client, your Smart Speaker would restart and would see the following screen on your client.

 

References

Additional Resources

Tags

fossasia, susi, susi.ai, gsoc, gsoc’18, handshake

Continue ReadingConnecting the Smart Speaker with Mobile Clients

Modifying Finite State Architecture On SUSI Linux to Process Multiple Queries

During the initial stages of SUSI Linux: As the code base grew, it was getting very difficult to manage code, so we opted to implement a Finite State Architecture in our repo. But, as there were new features implemented in the Repo, we realized that we couldn’t process more than one query at a time which restricted a lot of features. eg. The smart speaker was converted to a simple Bluetooth speaker since no response regarding playing/pausing were accepted.

To solve this issue, we made a slight modification in the architecture.

Brief About SUSI States

SUSI is working as a Finite State Machine and is present in 3 states namely IDLE state, Recognising state and Busy state. The State Machine executes in the following order.

  1. IDLE State:
    When the SUSI state Machine is in this State, SUSI is searching for the hotword “SUSI”, waiting to trigger the complete Machine.
  2. Recognizing State

In this State , the State Machine has started the STT client. After recognition, SUSI sends the query to the Server awaiting the response

  1. Busy State

After the response has been received, the TTS client is triggered and the answer is given out by SUSI

Adding a Second Hotword Recognition Class

Now, to allow SUSI to process the second query, The State machine must be triggered while SUSI is giving out the first response and to trigger the State Machine, we must have hotword recognition while SUSI is speaking the answer to the previous query. Hence, a hotword recognition engine is now initiated every time the State Machine enters the busy state.

We will be using Snowboy as Hotword Detection Engine.

 

import os
TOP_DIR = os.path.dirname(os.path.abspath(__file__))
RESOURCE_FILE = os.path.join(TOP_DIR, “susi.pmdl”)
class StopDetector():
   “””This implements the Stop Detection with Snowboy Hotword Detection Engine.”””
    def __init__(self, detection) -> None:
       super().__init__()
       self.detector = snowboydecoder.HotwordDetector(
           RESOURCE_FILE, sensitivity=0.6)
       self.detection = detection
    def run(self):
       “”” Implementation of run abstract method in HotwordDetector. This method is called when thread is
started for the first time. We start the Snowboy detection and declare detected callback as
       detection_callback method declared ina parent class.
       “””
       self.detector.start(detected_callback=self.detection)

 

Now, this class takes the Callback function as a parameter which is passed when the transition to busy state takes place from the recognition state.

 

Modifying the State Machine Architecture

After declaring a second hotword recognition engine , we must modify how the transitions take place between the States of the SUSI State Machine.

Hence the callback that will be triggered is passed from the busy state.

 

def detection(self):
       “””This callback is fired when a Hotword Detector detects a hotword.
       :return: None
       “””
       if hasattr(self, ‘video_process’):
           self.video_process.send_signal(signal.SIGSTOP)
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.off()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.video_process.send_signal(signal.SIGCONT)
       if hasattr(self, ‘audio_process’):
           self.audio_process.send_signal(signal.SIGSTOP)  
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.wakeup()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.audio_process.send_signal(signal.SIGCONT)

 

As soon as the hotword is detected ,the state machine makes transitions to the Recognition State while pausing the current Music and resumes the Music after the second query has been completed.

 

This is how SUSI processes multiple queries simultaneously while still maintaining finite state archi.

 

Additional Resources

Tags

gsoc, gsoc’18, finite_state_machine, susi_linux, multiple_query, susi.ai, susi

 

Continue ReadingModifying Finite State Architecture On SUSI Linux to Process Multiple Queries

Creating a Custom Raspbian Image containing SUSI.AI Linux Libraries

Installing Raspbian and SUSI Linux on your Raspberry Pi can be a long process and if your raspberry Pi crashes due to some bug, you have to repeat the process again and again.

It wastes a lot of valuable time. So, we will discuss a method in which we will have to install the SUSI Linux repo only once and can use it again in case of any issues.

First, we’ll go through the requirements for f installing SUSI Linux on our hardware

Hardware Requirements

1> Raspberry Pi

2> Micro SD card (16GB or greater)

3> USB Mic

4> USB Mouse and USB Keyboard

5> HDMI Monitor

6> ReSpeaker Pi Hat 2 Mic Array(optional)

7> 3.5 mm Jack Headphones / Speaker

 

Step 1: Preparing SD for Installation

1> To format your SD card

  • You can use softwares like SDCardformater to do so.

2> To install Raspbian

  • Download raspbian official build from here
  • Now mount the Image using software like Etcher or win32diskimager

 

Step 2: Installing SUSI Linux on your Pi

1> Navigate to the folder `/home/pi` and make a folder called SUSI.AI

 

cd /home/pi
mkdir SUSI.AI
cd SUSI.AI

 

2> Clone the SUSI Linux repo from here and navigate in the repo

 

git clone http://github.com/fossasia/susi_linux

cd susi_linux/

 

3> Run the installation script by using the command `./install.sh`

 

./install.sh

 

4> Run the configuration script by using the following command

`python3 config_generator.py <stt> <tts> <hotword> <wake>`

 

5> Run SUSI linux with the following command

`python3 -m main`

‘log’

If it plays a bell after you say ‘SUSI’ , it means that your software has been successfully installed

Step 3: Creating the image

1> Now that you have successfully installed SUSI Linux on your raspberry Pi , we will make a backup of the current stage of the system and use it for future references

 

2>Turn off the raspberry Pi , and remove the SD card from the Pi and insert it in your system.

 

3> To create the custom Image , use something like win32 imager and follow the steps below

  • In the text box , create a custom where you want your image to exist
  • Click on read button
  • And voila

 

References

https://www.raspberrypi.org/downloads/raspbian/

Tags

Fossasia, gsoc’18, SUSI.AI , susi_linux , gsoc, SUSI HW, installation

Continue ReadingCreating a Custom Raspbian Image containing SUSI.AI Linux Libraries

Implementing Volume Action in SUSI Smart Speaker

We all know that a Smart Speaker to excel above its competitors has to excel in first being a good “Speaker” and a speaker has a basic and essential feature which is “volume control”. But things get better if you can control your volume with your voice.

So, we have implemented a feature that allows the user to control the volume of the audio with his/her voice.

Below are the steps we had to follow to implement this feature

 

Step 1: Creating the Skills

The skills required to implement the ‘volume-action’ is implemented in the SUSI Server repo itself.

The skill is located in

susi_server/conf/system_skills/general/en/en_0001_foundation.txt

 

And below are the skills required

 

set audio volume to *|set audio volume to * percent|set audio volume to * points|set volume to *|set volume to * percent|set volume to * points
!console:Audio volume is now $1$ percent.
{“actions”:[
{“type”:“audio_volume”, “volume”:$1$}
]}
eol

 

We get the following response from the server

 

“actions”: [
     {
       “volume”: “80”,
       “type”: “audio_volume”
     },
     {
       “type”: “answer”,
       “expression”: “Audio volume is now 80 percent.”
     }

 

Step 2: Finding Volume Action in the server response

Now that our Server responds to our queries regarding the voice change action , we must implement it in our Smart Speaker Client.


We first create a custom class in our in the SUSI API Wrapper repo which has only one member

 

class VolumeAction(BaseAction):
   def __init__(self , volume):
       super().__init__()
       self.volume = volume

 

We check through the actions in the server’s response

 

elif isinstance(action, VolumeAction):
           result[‘volume’] = action.volume

 

Step 3: Implementing it in the client

Now to implement the action in our client we use a library called ‘alsaaudio’ to control the master volume of our RaspberryPi

 

              m = alsaaudio.Mixer()
               m.setvolume(int(reply[‘volume’]))
               os.system(‘play {0} &’.format(self.components.config[‘detection_bell_sound’]))  # nosec #pylint-disable type: ignore                m = alsaaudio.Mixer()
               m.setvolume(int(reply[‘volume’]))
               os.system(‘play {0} &’.format(self.components.config[‘detection_bell_sound’]))  # nosec #pylint-disable type: ignore

 

Now the user can easily change the speaker using the voice commands

References

 

Tags

GSoC, GSoC’18, SUSI.AI, SUSI Linux, Smart Speaker , SUSI API Wrapper, SUSI Server, FOSSASIA, Volume Action

Continue ReadingImplementing Volume Action in SUSI Smart Speaker

Displaying SUSI Smart speaker under Devices while logging in

The user should be given an ability to access all his devices on one page(Smart Speaker, IOS Device, WebClient and the Android Device). The user was previously allowed to access his/her web app account, the IOS app, Android App. But not the Smart Speaker. Now, this feature will allow the user to easily manage the Smart Speaker devices without many hassles.

In this post, we will be talking about the API’s that we have used to send the details of the Smart-Speaker to the server.

About the API’s

  1. Below is the API endpoint which will return the list of all devices present under the user’s account

We use the following endpoint

/aaa/ListUserSettings.json?/access_token=access_token

 

Below is sample response :

“devices”: {
“8C-39-45-cc-eb-95”: {
“name”: “Device 1”,
“room”: “Room 1”,
“geolocation”: {
“latitude”: “52.34567”,
“longitude”: “62.34567”
}
 }
}

 

  1. The second endpoint that we will be using is to add a new Device under the devices section

API Endpoint

/aaa/addNewDevice.json?

 

This endpoint has the following parameters

  • macid (Mac address of the device)
  • name (Name of the device)
  • room (Room info of the device)
  • latitude (Latitude info of the device)
  • longitude (Longitude info of the device)

 

After successfully hitting the endpoint , you’ll get the following response

 

{
“accepted”: true,
“message”: “You have successfully added the device!”,
“session”: {“identity”: {
“type”: “email”,
“name”: “sansyrox@gmail.com”,
“anonymous”: false
}}
}

 

Implementing the API’s

  1. First, we check the server for existing devices. This step is implemented primarily to check weather our current Smart Speaker is already configured or not.
get_device_info = api_endpoint + ‘/aaa/listUserSettings.json?’

param1 = {
       ‘access_token’:access_token
   }

   # print(access_token)

   if access_token is not None:
       device_info_response = requests.get(get_device_info,param1)
       device_info = device_info_response.json()

   # print(device_info)

If the current device is not already configured on Server, we proceed to next step.

  1. Now we will configure the device with the server and then post the device settings there.
    We will implement the API in the following way:

 

if device_info is not None:
   device = device_info[‘devices’] # list of existing mac ids
   print(device)
   session = device_info[‘session’] # session info
   identity = session[‘identity’]
   name = identity[‘name’]
   
   params2 = {
   ‘macid’: macid,
   ‘name’: name,
   ‘device’: ‘Smart Speaker’,
   ‘access_token’: access_token
   }

   for dev in device:
       if dev == macid:
           print(‘Device already configured’)
           return
       else :
           adding_device = requests.post(add_device_url, params2)
           print(adding_device.url)

 

To extract the mac address from the speaker and pass it as the params , we use a python library called UUID and this is how SUSI Smart Speaker is displayed on the web client(chat.susi.ai).

Resources

Tags

 

Continue ReadingDisplaying SUSI Smart speaker under Devices while logging in

Creating an Update Daemon for SUSI Smart Speaker

A daemon in reference of operating systems is a computer program that runs as a background process rather than under direct control of the user. Various daemons are being used in SUSI smart speaker.

The following daemons have been created

  • Update Daemon
  • Media Discovery Daemon
  • Factory Reset Daemon 

In this blog, we’ll be discussing the implementation of the Update Daemon in SUSI.AI

Update Daemon

Due to the ever-growing coding community, it is needed to provide regular updates to the smart speaker and keep it in sync with the latest technology. Hence an Update Daemon was required that could fetch updates at a regular interval.

The Updated Daemon was implemented in the following steps

1.Deciding the Update Interval

How frequently should we check for updates was the first question that was tackled while implementing this daemon.
We decided that we should check for Update, every time the Raspberry Pi starts and an internet connection was available.

2. Implementing The Decision

To start the Update script every time the Raspberry Pi starts, we decided to create Systemd rules.

[Unit]
Description=Update Check- SUSI Linux
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/home/pi/SUSI.AI/susi_linux/update_daemon/update_check.sh

[Install]
WantedBy=multi-user.target

The above rule waits for a network connection to be established with the Raspberry Pi and then triggers a bash script that fetches updates

3. Fetching The Updates


Now, a bash script was prepared that would fetch the latest changes from the online repo and merge the latest changes in the local repo

 

#!/bin/sh

UPSTREAM=${1:-‘@{u}’}
LOCAL=$(git rev-parse @)
REMOTE=$(git rev-parse “$UPSTREAM”)
BASE=$(git merge-base @ “$UPSTREAM”)
CHECK=”
if [ $LOCAL = $REMOTE ]
then
   echo “Up-to-date”
   CHECK=’up-to-date
elif [ $LOCAL = $BASE ]
then
   echo “Need to pull”
   CHECK=”Need-to-pull”
else
   echo “Diverged”
fi

if [$CHECK = “Need-to-pull”]
then
   git fetch UPSTREAM
   git merge UPSTREAM/master
fi

 

Resources

Tags

 

susi.ai, gsoc, gsoc’18, fossasia, update, daemon, update_daemon, smart speaker, systemd, hardware

Continue ReadingCreating an Update Daemon for SUSI Smart Speaker