You are currently viewing Adding Support for Playing Audio in SUSI iOS App

Adding Support for Playing Audio in SUSI iOS App

SUSI.AI supports various actions like the answer, map, table, video play and many more. You can play youtube videos in the chat screen. It also supports for playing audio in the chat screen. In this post, we will see that how playing audio feature implemented in SUSI iOS.

Getting audio action from server side –

In the chat screen, when we ask SUSI to play audio, we get the audio source from the server side. For example, if we ask SUSI “open the pod bay door”, we get the following action object:

"actions": [
"type": "audio_play",
"identifier_type": "youtube",
"identifier": "7qnd-hdmgfk"
"language": "en",
"type": "answer",
"expression": "I'm sorry, Dave. I'm afraid I can't do that."

In the above action object, we can see that we get two actions, audio_play and answer. In audio_play action, we are getting an identifier type which tells us about the source of audio. Identifier type can be youtube or local or any other source. When the identifier is youtube, we play audio from youtube stream. In identifier, we get the audio file path. In case of youtube identifier type, we get youtube video ID and play from youtube stream. In answer action type, we get the expression which we display in chat screen after thumbnail.

Implementing Audio Support in App –

We use Google’s youtube Iframe API to stream audio from youtube videos. We have a VideoPlayerView that handle all the iFrame API methods and player events with help of YTPlayer HTML file.

Presenting the YouTubePlayerCell –

If the action type is audio_play, we are presenting the cell in chat screen using cellForItemAt method of UICollectionView.

if message.actionType == ActionType.audio_play.rawValue {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.youtubePlayerCell, for: indexPath) as? YouTubePlayerCell {
cell.message = message
cell.delegate = self
return cell

Setting size for cell –

Using sizeForItemAt method of UICollectionView to set the size.

if message.actionType == ActionType.audio_play.rawValue {
return CGSize(width: view.frame.width, height: 158)

In YouTubePlayerCell, we fetch thumbnail and display in the cell with a play button. On clicking the play button, we open the player and stream music.

Final Output –

Resources –

  1. Apple’s Documentations on sizeForItemAt
  2. SUSI API Sample for Audio Play Action
  3. YouTube iFrame API for iOS
  4. Apple’s Documentations on cellForItemAt

Jogendra Kumar

Jogendra is pre-final year student at Indian Institute of Technology (BHU) Varanasi. He is an iOS Developer and an active Open Source contributor, also have experience in developing Android, Web and Windows Apps. Currently, he is focusing on Machine Learning and Blockchain Technology. Apart from Coding, he loves coffee, going hackathons, attending conferences, and traveling. You can find him on twitter here

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.