Reducing the YouTube response time by 90%

In this blog post, we are going to cover how the audio from Youtube is being used in SUSI Smart Speaker and how we reduced the response time from ~40 seconds to ~4 seconds for an average music video length.

First Approach

Earlier, we were using MPV player’s inbuilt feature to fetch the YouTube music. However, MPV player was a bulky option and the music server had to be started every time before initiating a music video.

video_process = subprocess.Popen([‘mpv’, ‘–no-video’, ‘https://www.youtube.com/watch?v=’ + video_url[4:], ‘–really-quiet’]) # nosec #pylint-disable type: ignore requests.get(‘http://localhost:7070/song/’ + video_url) self.video_process = video_process stopAction.run() stopAction.detector.terminate()

Making it Efficient

To reduce the response time, we created a custom Music Server based on Flask,python-vlc and python-pafy which accepts requests from the main client and instructs the System to play the music with just 90% more efficiency.

app = Flask(__name__)

Instance = vlc.Instance(‘–no-video’)

player = Instance.media_player_new()

url = @app.route(‘/song’, methods=[‘GET’])

def youtube():

    vid = request.args.get(‘vid’)

    url = ‘https://www.youtube.com/watch?v=’ + vid

    video = pafy.new(url)
    streams = video.audiostreams 

    best = streams[3]

    playurl = best.url

    Media = Instance.media_new(playurl)

    Media.get_mrl()

    player.set_media(Media)

    player.play()

    display_message = {“song”:“started”}

    resp = jsonify(display_message)

    resp.status_code = 200

    return resp

However, shifting to this Server removed the ability to process multiple queries and hence we were unable to pause/play/stop the music until it completed the time duration. We wanted to retain the ability to have ‘play/pause/stop’ actions without implementing multiprocessing or multithreading as it would’ve required extensive testing to successfully implement them without creating deadlocks and would’ve been overkill for a simple feature.

Bringing Back the Lost Functionalities

The first Step we took was to remove the vlc-python module and implement a way to obtain an URL that we use in another asynchronous music player.

@app.route(‘/song’, methods=[‘GET’])
def youtube():

    vid = request.args.get(‘vid’)

    streams = video.audiostreams

    best = streams[3]

    playurl = best.url 

    display_message = {“song”: “started”, “url”: playurl}

    resp = jsonify(display_message)

    resp.status_code = 200

    return resp

The next issue was to actually find a way to run the Music Player asynchronously. We used the `subprocess. Popen` method and cvlc to play the songs asynchronously.

try:

    x = requests.get(‘http://localhost:7070/song?vid=’ + video_url[4:])

    data = x.json()

    url = data[‘url’]

    video_process = subprocess.Popen([‘cvlc’, ‘http’ + url[5:], ‘–no-video’])

    self.video_process = video_process

except Exception as e:

    logger.error(e);

And this is how we were able to increase the efficiency of the music player while maintaining the functionalities.

References

Continue ReadingReducing the YouTube response time by 90%

Adding 3D Home Screen Quick Actions to SUSI iOS App

Home screen quick actions are a convenient way to perform useful, app-specific actions right from the Home screen, using 3D Touch. Apply a little pressure to an app icon with your finger—more than you use for tap and hold—to see a list of available quick actions. Tap one to activate it. Quick actions can be static or dynamic.

We have added some 3D home screen quick action to our SUSI iOS app. In this post, we will see how they are implemented and how they work.

The following 3D home screen quick actions are added to SUSI iOS:

  • Open SUSI Skills – user can directly go to SUSI skills without opening a chat screen.
  • Customize Settings – user can customize their setting directly by using this quick action.
  • Setup A Device – when the user quickly wants to configure his/her device for SUSI Smart Speaker, this is quick action is very helpful in that.
  • Change SUSI’s Voice – user can change SUSI message reading language accents directly from this quick action.

Each Home screen quick action includes a title, an icon on the left or right (depending on your app’s position on the home screen), and an optional subtitle. The title and subtitle are always left-aligned in left-to-right languages.

Step 1 – Adding the Shortcut Items

We add static home screen quick actions using the UIApplicationShortcutItems array in the app Info.plist file. Each entry in the array is a dictionary containing items matching properties of the UIApplicationShortcutItem class. As seen in screenshot below, we have 4 shortcut items and each item have three properties UIApplicationShortcutItemIconType/UIApplicationShortcutItemIconFile, UIApplicationShortcutItemTitle, and UIApplicationShortcutItemType.

  • UIApplicationShortcutItemIconType and UIApplicationShortcutItemIconFile is the string for setting icon for quick action. For the system icons, we use UIApplicationShortcutItemIconType property and for the custom icons, we use UIApplicationShortcutItemIconFile.
  • UIApplicationShortcutItemTitle is a required string that is displayed to the user.
  • UIApplicationShortcutItemType is a required app specific string used to identify the quick action.

Step 2 – Handling the Shortcut

AppDelegate is the place where we handle all the home screen quick actions. We define these variables:

var shortcutHandled: Bool!
var shortcutIdentifier: String?

When a user chooses one of the quick actions the launch of the system or resumes the app and calls the performActionForShortcutItem method in app delegate:

func application(_ application: UIApplication,
                     performActionFor shortcutItem: UIApplicationShortcutItem,
                     completionHandler: @escaping (Bool) -> Void) {
        shortcutIdentifier = shortcutItem.type
        shortcutHandled = true
        completionHandler(shortcutHandled)
    }

Whenever the application becomes active, applicationDidBecomeActive function is called:

func applicationDidBecomeActive(_ application: UIApplication) {
        // Handel Home Screen Quick Actions
        handelHomeActions()
    }

Inside the applicationDidBecomeActive function we call the handleHomeAction() method which handles the home screen quick action.

func handelHomeActions() {
       if shortcutHandled == true {
           shortcutHandled = false
           if shortcutIdentifier == ControllerConstants.HomeActions.openSkillAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.customizeSettingsAction {
               // Handle action accordingly
           } else if shortcutIdentifier == ControllerConstants.HomeActions.setupDeviceAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.changeVoiceAction {
               // Handle action accordingly
               }
           }
       }
   }

Final Output:

Resources –

  1. Home Screen Quick Actions – Human Interface Guidelines by Apple
  2. Adding 3D Touch Quick Actions by Use Your Leaf
  3. Apple’s documentation on performActionFor:completionHandler
Continue ReadingAdding 3D Home Screen Quick Actions to SUSI iOS App

Adding report skills feature in SUSI iOS

SUSI.AI is having a various type of Skills that improving the user experience. Skills are powering the SUSI.AI personal chat assistant. SUSI skills have a nice feedback system. We have three different feedback way for SUSI skills, 5-star rating system, posting feedback, and reporting skills.

5-Star Rating – rate skills from 1 (lowest) to 5 (highest) star

Posting Feedback – user can post feedback about particular skill

Report Skill – user can report skill if he/she found it inappropriate

In this post, we will see how reporting skills feature work in SUSI iOS and how it is implemented. You can learn about 5-star rating here and posting feedback feature here.

Adding report skill button –

let reportSkillButton: UIButton = {
        let button = UIButton(type: .system)
        button.contentHorizontalAlignment = .left
        button.setTitle("Report Skill", for: .normal)
        button.setTitleColor(UIColor.iOSGray(), for: .normal)
        button.titleLabel?.font = UIFont.systemFont(ofSize: 16)
        button.translatesAutoresizingMaskIntoConstraints = false
        return button
    }()

In above, we have set translatesAutoresizingMaskIntoConstraints property for false. By default, the property is set to true for any view you programmatically create. If you add views in Interface Builder, the system automatically sets this property to false. If this property’s value is true, the system creates a set of constraints that duplicate the behavior specified by the view’s autoresizing mask.

Setting up report skill button –

We are setting constraints programmatically as we created button programmatically and set translatesAutoresizingMaskIntoConstraints to false. Also, setting a target to the button.

if let delegate = UIApplication.shared.delegate as? AppDelegate, let _ = delegate.currentUser {
            view.addSubview(reportSkillButton)
            reportSkillButton.widthAnchor.constraint(equalToConstant: 140).isActive = true
            reportSkillButton.heightAnchor.constraint(equalToConstant: 32).isActive = true
            reportSkillButton.leftAnchor.constraint(equalTo: contentType.leftAnchor).isActive = true
            reportSkillButton.topAnchor.constraint(equalTo: contentType.bottomAnchor, constant: 8).isActive = true

            reportSkillButton.addTarget(self, action: #selector(reportSkillAction), for: .touchUpInside)
        }

In the above method, we can see that we are only showing button if user is logged-in. Only a logged-in user can report the skill. To check if user is logged in or not, we are using the AppDelegate shared instance where we save the logged-in user globally when the user signs in.

When user clicks the Report Skill button, a popup is open up with a text field for feedback message like below:

This is how UI look like!

When user clicks the Report action after typing feedback message, we are using the following endpoint:

https://api.susi.ai/cms/reportSkill.json

With the following parameters –

  • Model
  • Group
  • Skill
  • Language
  • Access Token
  • Feedback

Here is how we are handling the API call within our app –

func reportSkill(feedbackMessage: String) {
        if let delegate = UIApplication.shared.delegate as? AppDelegate, let user = delegate.currentUser {

            let params = [
                Client.SkillListing.model: skill?.model as AnyObject,
                Client.SkillListing.group: skill?.group as AnyObject,
                Client.SkillListing.skill: skill?.skillKeyName as AnyObject,
                Client.SkillListing.language: Locale.current.languageCode as AnyObject,
                Client.SkillListing.accessToken: user.accessToken as AnyObject,
                Client.SkillListing.feedback: feedbackMessage as AnyObject
            ]

            Client.sharedInstance.reportSkill(params) { (success, error) in
                DispatchQueue.main.async {
                    if success {
                        self.view.makeToast("Skill reported successfully")
                    } else if let error = error {
                        self.view.makeToast(error)
                    }
                }
            }
        }
    }

On successfully reported skill, we show a toast with ‘Skill reported successfully’ message and if there is error reporting the skills, we present the toast with error as a message.

Resources –

  1. SUSI Skills: https://skills.susi.ai/
  2. Apple’s documentations on translatesAutoresizingMaskIntoConstraints
  3. Allowing user to submit ratings for skills in SUSI iOS
  4. Displaying Skills Feedback on SUSI iOS
Continue ReadingAdding report skills feature in SUSI iOS

A Workflow of Auto Executing Services on SUSI.AI Smart Speaker

As we plan to create a headless client on RaspberryPi, the requirement was that the SUSI.AI programs should run automatically. To do so, we had to figure out a way to boot up various scripts on startup.

We had the following options to execute the scripts on startup:

  1. Editing Rc.local file
  2. Systemd Rules
  3. Crontab

We decided to proceed with Systemd Rules because using Rc.local and Crontab requires modifying the default system files which in case of any error would make the os functionalities to crash very soon.

We then created the SystemD rules for the following services:

1.factory-daemon.service
2. python-flask.service
3. susi-server.service
4. update-daemon.service
5. susi-linux.service

Now I’ll demonstrate the working and the functionality of each service being implemented.

1. Factory-Daemon Service

This service initiates the factory daemon with the raspberry Pi startup and then keeps it running continuously looking for any input from the GPiO port.

[Unit]
Description=SUSI Linux Factory Daemon
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3 /home/pi/SUSI.AI/susi_linux/factory_reset/factory_reset.py

[Install]
WantedBy=multi-user.target

2. Python-Flask Service

This service starts a python Server to allow handshake between mobile apps and the Smart Speaker which will allow the user to configure SUSI Smart Speaker accordingly.

[Unit]
Description=Python Server for SUSI Linux
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3  /home/pi/SUSI.AI/susi_linux/access_point/server/server.py

[Install]
WantedBy=multi-user.target

3.SUSI-Server Service

This service starts the Local SUSI Server as soon as the Raspberry Pi starts up which in turn allows the SUSI Linux programs to fetch responses of queries very quickly.

[Unit]
Description=Starting SUSI Server for SUSI Linux
After=multi-user.target

[Service]
Type=oneshot
ExecStart=/home/pi/SUSI.AI/susi_linux/susi_server/susi_server/bin/restart.sh

[Install]
WantedBy=multi-user.target

4. Update-Daemon Service

This Service creates a Daemon which starts with the Raspberry Pi and fetches the latest updates from the repository from the upstream branch.

[Unit]
Description=Update Check- SUSI Linux
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/home/pi/SUSI.AI/susi_linux/update_daemon/update_check.sh

[Install]
WantedBy=multi-user.target

5. SUSI-Linux Service

This Service finally runs the main SUSI Linux software after everything has started.

[Unit]
Description=Starting SUSI Linux
Wants=network-online.target
After=network-online.target

[Service]
Type=idle
WorkingDirectory=/home/pi/SUSI.AI/susi_linux/
ExecStart=/usr/bin/python3 -m main

[Install]
WantedBy=multi-user.target

This blog gives a brief workflow of auto-executing services on SUSI Smart Speaker.

Resources

Continue ReadingA Workflow of Auto Executing Services on SUSI.AI Smart Speaker

Connecting the Smart Speaker with Mobile Clients

The beauty of SUSI Smart Speaker lies in it being customizable according to the user’s needs. And we allow the user to customize it by providing an interface through the mobile clients. To do so, we create a local server on the Raspberry Pi itself. The Raspberry Pi is started in an Access Point mode and the mobile clients hit the endpoints in a specific order and then the configuration is sent to the server and stored according to the user.

 

The following API’s are required to be executed by the mobile clients

1> /speaker_config

2> /wifi_credentials

3> /auth

4> /config

 

The following is the order of API execution

1. /speaker_config

This endpoint only takes the room name as a parameter. And then send send to the server to store the location of the device under the user’s account

def speaker_config():
   room_name = request.args.get(‘room_name’)
   config = json_config.connect(config_json_folder)
   config[‘room_name’] = rogom_name

 

2. /wifi_credentials

This endpoint takes the wifi ssid and wifi password as the parameters and then stores it in the raspberry Pi wifi config file.

 

def wifi_config():
   wifi_ssid = request.args.get(‘wifissid’)
   wifi_password = request.args.get(‘wifipassd’)
   subprocess.call([‘sudo’, ‘bash’, wifi_search_folder + ‘/wifi_search.sh’, wifi_ssid, wifi_password])
   display_message = {“wifi”:“configured”, “wifi_ssid”:wifi_ssid, “wifi_password”: wifi_password}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

Now the script wifi_search is called which stores the wifi credentials in the wifi_config file using the following command

 

cat >> /etc/wpa_supplicant/wpa_supplicant.conf <<EOF
network={
   ssid=“$SSID”
   psk=“$PSK”
}
EOF

 

3. /auth

This endpoint takes the SUSI’s login credentials as parameters, i.e. the registered email id and the corresponding password.

 

def login():
   auth = request.args.get(‘auth’)
   email = request.args.get(’email’)
   password = request.args.get(‘password’)
   subprocess.call([‘sudo’, ‘bash’, access_point_folder + ‘/login.sh’, auth, email, password])
   display_message = {“authentication”:“successful”, “auth”: auth, “email”: email, “password”: password}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

4. /config

Finally, this endpoint takes the stt, tts, hotword detection engine and wake button as the parameters and configures the speaker accordingly.

 

def config():
   stt = request.args.get(‘stt’)
   tts = request.args.get(‘tts’)
   hotword = request.args.get(‘hotword’)
   wake = request.args.get(‘wake’)
   subprocess.Popen([‘sudo’, ‘bash’, access_point_folder + ‘/config.sh ‘, stt, tts, hotword, wake])
   display_message = {“configuration”:“successful”, “stt”: stt, “tts”: tts, “hotword”: hotword, “wake”:wake}
   resp = jsonify(display_message)
   resp.status_code = 200
   return resp

 

Now, this function runs a script called config.sh which in turn runs a script called rwap.sh to convert the Raspberry Pi to normal mode and then finally start SUSI on startup.

 

#!/bin/bash

if [ $EUID -ne 0 ]
then echo “Must be root”
exit
fi

cd /etc/hostapd/
sed -i ‘1,14d’ hostapd.conf

cd /etc/
sed -i ‘57,60d’ dhcpcd.conf

cd /etc/network/
sed -i ‘9,17d’ interfaces

echo “Please reboot”
sudo reboot

 

After successfully hitting all the endpoint from the client, your Smart Speaker would restart and would see the following screen on your client.

 

References

Additional Resources

Tags

fossasia, susi, susi.ai, gsoc, gsoc’18, handshake

Continue ReadingConnecting the Smart Speaker with Mobile Clients

Modifying Finite State Architecture On SUSI Linux to Process Multiple Queries

During the initial stages of SUSI Linux: As the code base grew, it was getting very difficult to manage code, so we opted to implement a Finite State Architecture in our repo. But, as there were new features implemented in the Repo, we realized that we couldn’t process more than one query at a time which restricted a lot of features. eg. The smart speaker was converted to a simple Bluetooth speaker since no response regarding playing/pausing were accepted.

To solve this issue, we made a slight modification in the architecture.

Brief About SUSI States

SUSI is working as a Finite State Machine and is present in 3 states namely IDLE state, Recognising state and Busy state. The State Machine executes in the following order.

  1. IDLE State:
    When the SUSI state Machine is in this State, SUSI is searching for the hotword “SUSI”, waiting to trigger the complete Machine.
  2. Recognizing State

In this State , the State Machine has started the STT client. After recognition, SUSI sends the query to the Server awaiting the response

  1. Busy State

After the response has been received, the TTS client is triggered and the answer is given out by SUSI

Adding a Second Hotword Recognition Class

Now, to allow SUSI to process the second query, The State machine must be triggered while SUSI is giving out the first response and to trigger the State Machine, we must have hotword recognition while SUSI is speaking the answer to the previous query. Hence, a hotword recognition engine is now initiated every time the State Machine enters the busy state.

We will be using Snowboy as Hotword Detection Engine.

 

import os
TOP_DIR = os.path.dirname(os.path.abspath(__file__))
RESOURCE_FILE = os.path.join(TOP_DIR, “susi.pmdl”)
class StopDetector():
   “””This implements the Stop Detection with Snowboy Hotword Detection Engine.”””
    def __init__(self, detection) -> None:
       super().__init__()
       self.detector = snowboydecoder.HotwordDetector(
           RESOURCE_FILE, sensitivity=0.6)
       self.detection = detection
    def run(self):
       “”” Implementation of run abstract method in HotwordDetector. This method is called when thread is
started for the first time. We start the Snowboy detection and declare detected callback as
       detection_callback method declared ina parent class.
       “””
       self.detector.start(detected_callback=self.detection)

 

Now, this class takes the Callback function as a parameter which is passed when the transition to busy state takes place from the recognition state.

 

Modifying the State Machine Architecture

After declaring a second hotword recognition engine , we must modify how the transitions take place between the States of the SUSI State Machine.

Hence the callback that will be triggered is passed from the busy state.

 

def detection(self):
       “””This callback is fired when a Hotword Detector detects a hotword.
       :return: None
       “””
       if hasattr(self, ‘video_process’):
           self.video_process.send_signal(signal.SIGSTOP)
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.off()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.video_process.send_signal(signal.SIGCONT)
       if hasattr(self, ‘audio_process’):
           self.audio_process.send_signal(signal.SIGSTOP)  
           lights.wakeup()
           subprocess.Popen([‘play’, str(self.components.config[‘detection_bell_sound’])])
           lights.wakeup()
           self.transition(self.allowedStateTransitions.get(‘recognizing’))
           self.audio_process.send_signal(signal.SIGCONT)

 

As soon as the hotword is detected ,the state machine makes transitions to the Recognition State while pausing the current Music and resumes the Music after the second query has been completed.

 

This is how SUSI processes multiple queries simultaneously while still maintaining finite state archi.

 

Additional Resources

Tags

gsoc, gsoc’18, finite_state_machine, susi_linux, multiple_query, susi.ai, susi

 

Continue ReadingModifying Finite State Architecture On SUSI Linux to Process Multiple Queries

Creating a Factory Reset Daemon for SUSI.AI Smart Speaker

In our constantly evolving SUSI.AI Smart Speaker project, we require regular updates for our devices. And imagine a scenario that during a crucial update, there is a crash or an internet disconnection which stops the SUSI.AI Linux program from booting up. We’ll require a reset method for that. So, we have added a button in SUSI smart speaker that works as a factory reset switch. This daemon was accomplished by using python scripting, bash scripting, and Raspbian’s systemd rules.

Approach followed

We have created a python script that detects the button presses on GPIO port 17. The script is run as soon as the Raspberry Pi is booted using the systemd rules and checks for the device inputs. And if the button press is for more than 7 seconds, the factory_reset.sh script is run which deletes all the contents of the repo and clones it again.

 

#! /bin/bash
# To be executed using a physical button

SCRIPT_PATH=$(realpath $0)
DIR_PATH=$(dirname $SCRIPT_PATH)

cd $DIR_PATH/../..
pwd
mv susi_linux/ susi_temp
git clone https://github.com/fossasia/susi_linux #while testing change to personal repo
pwd
ls
cd susi_linux

rm -rf ../susi_temp

./install.sh

 

Detecting the Button Press

We have Used the library RPi.GPIO to detect button click on raspberry Pi.

while True:
       if GPIO.input(17) == 1:
           pass
       elif GPIO.input(17) == 0 :
           start = time.time()
          while GPIO.input(17) == 0 :
               print(“on”)
               time.sleep(0.1)
          end = time.time()
           total = end – start
           if total >= 7 :
              subprocess.call([‘bash’,‘factory_reset.sh’])
          else :
               mixer = alsaaudio.Mixer()
               value = mixer.getvolume()[0]
              if value != 0:
                  mixer.setvolume(0)
               else:
                   mixer.setvolume(50)
           print(total)
           time.sleep(0.1)

 

If the button press is greater than 7 seconds, factory reset process will start and if the press is less than 7 seconds, the button will function as mute button

 

Auto Booting The program

 

For the script to autorun everytime the raspberry pi started. We create systemd file which will allow the program to start as soon as the device has started

 

[Unit]
Description=SUSI Linux Factory Daemon
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3 /home/pi/SUSI.AI/susi_linux/factory_reset/factory_reset.py

[Install]
WantedBy=multi-user.target

 

This runs the factory reset script to boot up as soon as the Raspberry Pi starts

References

Tags

susi, factory_daemon, factory_reset, gsoc, gsoc’18,susi_linux , fossasia

Continue ReadingCreating a Factory Reset Daemon for SUSI.AI Smart Speaker

Using a Flask Server to Connect to SUSI smart speaker

A smart speaker becomes significantly smarter when it is connected to a Smart-Phone.

So, we added a way to connect the Smart-Phone to the Smart Speaker and initiate the first way towards a Smart Home.

Use a simple HTTP connection protocol and deploy a light-weight server on the Raspberry Pi to allow connection from a mobile phone.

Step 1: Setting Up the server

Use flask to deploy a light-weight server on the raspberry pi. We’ll install flask using raspbian repos.

 

1>Install Flask by using the following command

sudo apt-get install python3-flask

 

2> Setting up the boilerplate code.

Open the terminal and type the following commands

`

mkdir server_app
cd server_app

touch app.py

`

 

Add the following code to your app.py file. This create a server at localhost:5000

 

from flask import Flask

app = Flask(__name__)

@app.route(‘/’)
def index():
   return ‘Hello world’

if __name__ == ‘__main__’:
   app.run(debug=False, host=‘0.0.0.0’)  #This will allow the server to be accessible on all devices

 

Step 2: Adding Endpoints

Now , add endpoints which will trigger the scripts during initialisation of the raspberry Pi. This will trigger the respective endpoints

@app.route(‘/auth/<auth>/<email>/<passwd>’)
def login(auth, email, passwd):
os.system(‘sudo ./login.sh {} {} {}’.format(auth, email,passwd)) #nosec #pylint-disable type: ignore
return ‘Authenticated’ # pylint-enable@app.route(‘/wifi_credentials/<wifissid>/<wifipassd>’)
def wifi_config(wifissid,wifipassd):
wifi_ssid = wifissid
wifi_password = wifipassd
os.system(‘sudo ./home/pi/SUSI.AI/susi_linux/access_point/wifi_search.sh {} {}’.format(wifi_ssid,wifi_password))  #nosec #pylint-disable type: ignore
return ‘Wifi Configured’ # pylint-enable

 

Step 3: Connecting to the endpoints

Now, try and hit the API endpoints to get the response.

eg.As shown in the above example, you will be getting a single line response and will execute a bash script behind the scenes

Now you can access the other endpoints and configure the clients with the SUSI Smart Speaker

References

 

Tags

fossasia,GSoC,Python, Flask , raspberryPi, SUSI,smart-speaker,FOSSASIA

Continue ReadingUsing a Flask Server to Connect to SUSI smart speaker

Create a Wireless Access Point Using a Raspberry Pi to Connect with SUSI Smart Speaker

To use the pi as a wifi bridge, a local network or just as a wifi range extender.We at FOSSASIA are using it as a network to connect between our SUSI.AI smart speaker and the Android and IOS devices. Or maybe because you can !! :’)

Requirements:

  1. Raspberry Pi Model 3(since we will be using an internal wifi)
  2. Power supply for the Pi.
  3. Monitor (optional)
  4. Keyboard (optional)
  5. Mouse (optional)

Steps:

1.Install and upgrade raspbian

 

Sudo apt-get update && sudo apt-get install

 

2. Install hostapd and dnsmasq .
This will allow us to use our raspberry pi as a wireless access point

 

apt-get remove –purge hostapd -yqq
apt-get update -yqq
apt-get upgrade -yqq
apt-get install hostapd dnsmasq -yqq

 

3. Now we will add broadcasting IP and DNS address in the dnsmasq configuration file

To access the configuration file use:

sudo nano /etc/dnsmasq.co

 

And to the bottom of the file, add the following commands

 

interface=wlan0
dhcp-range=10.0.0.2,10.0.0.5,255.255.255.0,12h

 

  1. Now to select the SSID and the PASSWORD for the access point, we’ll need to change the configurations of hostapd package
sudo nano /etc/hostapd/hostapd.conf

 

Then, use the following commands :

 

interface=wlan0
hw_mode=g
channel=10
auth_algs=1
wpa=2
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP
rsn_pairwise=CCMP
wpa_passphrase=“your_broadcasting_password”
ssid=“your_broadcasting_ssid”
ieee80211n=1
wmm_enabled=1
ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40]

 

  1. To finally sum up the configuration, we’ll have to create a  custom network interface that combines all the settings that we have made.
sudo nano /etc/network/interfaces

 

And add the following lines it the EOF

allow-hotplug wlan0
iface wlan0 inet static
address 10.0.0.1
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255

 

Now, we just have to have to disable default interfaces so that they do not interfere with the custom interfaces that we have made.

To do so

 

sudo nano /etc/dhcpcd.conf

 

Add the following line at the end of the file

denyinterfaces wlan0

 

  1. Now just restart the services

 

systemctl enable hostapd && systemctl enable dnsmasq

sudo service hostapd start && sudo service dnsmasq start

sudo reboot

 

Now, you will be able to enjoy a self-made access point which is used as a basic mode of connection in SUSI Smart Speaker and can also be used in various other access point methods.

 

References

 

Tags

GSOC’18 , FOSSASIA, ACCESS_POINT, SUSI.AI, GSOC, SUSI , SMART_SPEAKER

Continue ReadingCreate a Wireless Access Point Using a Raspberry Pi to Connect with SUSI Smart Speaker

Adding Audio Streaming from Youtube in SUSI Linux

In this blog post we will describe how the youtube streaming works in the

SUSI smart speaker and how audio is streamed directly from youtube videos.

To achieve this process, we have used an amazing Open-Source project called MPV music Player along with python libraries like Subprocess.

1.Processing a Query to the server

Firstly , the user asks the smart speaker to play the youtube audio by simply adding a ‘play’ word before his/her favorite song. eg. I’ll say ‘play despacito’ and then the command is recognized and a query is sent to the server which sends the following response as a JSON object.

“actions”: [
     {
       “type”: “answer”,
       “expression”: “Playing Luis Fonsi – Despacito ft. Daddy Yankee”
     },
     {
       “identifier”: “kJQP7kiw5Fk”,
       “identifier_type”: “youtube”,
       “type”: “video_play”
     }]

2.Parsing the response

Then the speaker parses the response in the following way.

The Speaker traverses through all the actions returned in the response and checks for all the “identifier” by assigning a custom class to it.

class VideoAction(BaseAction):
   def __init__(self, identifier , identifier_type):
       super().__init__()
       self.identifier = identifier
       self.identifier_type = identifier_type

Now we check whether the query is the type of a custom class VideoAction and then the client processes the query as the response.

      elif isinstance(action, VideoAction):
          result[‘identifier’] = action.identifier
           audio_url = result[‘identifier’]  

3.Implementing the Actions

Now that we have identified that the response contains a Video Action, we can finally implement a way to play the audio from the URL.
We use a music player called MPV Music Player and the library Subprocess to make it run asynchronously.

if ‘identifier’ in reply.keys():
   classifier = reply[‘identifier’]
   if classifier[:3] == ‘ytd’:
       video_url = reply[‘identifier’]
       video_pid = subprocess.Popen(‘mpv –no-video https://www.youtube.com/watch?v={} –really-quiet &’.format(video_url[4:]), shell=True)  # nosec #pylint-disable type: ignore
       self.video_pid = video_pid.pid


This is how audio is streamed from youtube videos in SUSI Smart Speaker.

Resources

  1. https://github.com/mpv-player/mpv
  2. https://docs.python.org/2/library/subprocess.html
  3. https://github.com/fossasia/susi_linux
  4. https://github.com/fossasia/susi_api_wrapper

Tags

fossasia, gsoc’18, susi, susi.ai, youtube, music, mp3 , mpv, audio stream

 

Continue ReadingAdding Audio Streaming from Youtube in SUSI Linux