Previously whenever a sound had to be played via the smart speaker, the subprocess python module was used to call the CVLC process and play audio via it. This puts forward a number of challenges while implementing various music features such as queuing songs, shuffling songs or handle the volume of the music. Thus, the audio structure was remade in the SUSI Smart Speaker project. The audio playing structure resides mainly in the susi_installer and the susi_linux repository. The above flow chart describes how audio is handled now in the project.
Location of related files:
Busy State: susi_linux/main/states/busy_state.py
Player: susi_linux/main/player
Sound Server: susi_installer/raspi/soundserver
VLC Player: susi_installer/pythonmods/vlcplayer
HW mixer: susi_installer/pythonmods/hwmixer
In the new structure instead of using CVLC and a youtube URL server, we use Python VLC with a sound server. The new structure as given in the flowchart is illustrated step by step below-
Busy State
The busy state class sends and accepts responses by the server through the susi_python wrapper. Here is the code which is responsible for handling play audio from a USB thumb drive or youtube.
| if ‘identifier’ in reply.keys(): url = reply[‘identifier’] if url[:3] == ‘ytd’: player.playytb(url[4:]) else: player.play(url) self.transition(self.allowedStateTransitions.get(‘idle’)) |
For example “SUSI, play audio” will have a response like :
| {‘identifier’: ‘file:///media/usb0/example.mp3’, ‘answer’: ‘Playing audio from your usb device’} |
For the above response the play method of the player class will be called.
Player
The Player class checks if the soundserver is running and sends the request to it. If the soundserver is not running then the VLC Player class is used directly.
Play method:
| def play(self, mrl, mode = None): self._executeArg(‘play’, ‘mrl’, mrl, mode) |
| def _executeArg(self, method, key, arg, mode = None): if (mode == ‘server’) or ((mode is None) and (self.mode == ‘server’)): send_request(method + ‘?’ + key + ‘=’ + arg) else: getattr(vlcplayer, method)(arg) |
Sound Server
The soundserver provides various methods of the VLC Player as endpoints. These endpoints are then used by the busy state directly or via the remote access webpage or an external application. An external application such as an android/ios app can use these endpoints to control the music playback on the device.
| @app.route(‘/play’, methods=[‘POST’, ‘PUT’]) def play_route(): if ‘ytb’ in request.args: vlcplayer.playytb(request.args.get(‘ytb’)) return do_return(‘Ok’, 200) elif ‘mrl’ in request.args: vlcplayer.play(request.args.get(‘mrl’)) return do_return(‘Ok’, 200) else: return do_return(‘Unknown play mode’, 400) |
VLC Player
The VLC Player class is actually responsible for playing and handling the music. This uses python VLC module for handling audio playback and various other functionalities. We use the Media List Player class found in the VLC player module to play music, using Media list player over media player gives us the advantage of queuing the files and essentially making a playlist.
For more info on MediaListPlayer class visit – https://www.olivieraubert.net/vlc/python-ctypes/doc/vlc.MediaListPlayer-class.html
| class VlcPlayer(): def __init__(self): self.saved_softvolume = -1 self.saved_hardvolume = -1 self.instance = vlc.Instance(“–no-video”) self.player = self.instance.media_player_new() self.sayplayer = self.instance.media_player_new() self.list_player = self.instance.media_list_player_new() self.list_player.set_media_player(self.player) |
The play method in VLC player
| def play(self, mrl_string): self.mrl = mrl_string.split(“;”) media_list = self.instance.media_list_new(self.mrl) self.list_player.set_media_list(media_list) self.list_player.play() self.softvolume(100, self.player) |
The play method receives a single MRL or multiple MRLs. If multiple MRLs are sent, they are separated via a semicolon ‘;’. The list player method of VLC Player class takes a list of MRLs as an input so if the received string has more than one MRLs it is broken down into a list of MRLs via the python’s inbuilt split method, which is then added to the list player.
At last play method of the mediaListPlayer class is used to play the music/audio.
Resources
Python VLC library – https://pypi.org/project/python-vlc/
VLC python bindings – https://wiki.videolan.org/Python_bindings
Tags
SUSI Smart Speaker, SUSI.AI, FOSSASIA, GSoC19