Adding Support for Playing Youtube Videos in SUSI iOS App

SUSI supports very exciting features in chat screen, from simple answer type to complex map, RSS, table etc type responses. Even user can ask SUSI for the image of anything and SUSI response with the image in the chat screen. What if we can play the youtube video from SUSI, we ask SUSI for playing videos and it can play youtube videos, isn’t it be exciting? Yes, SUSI can play youtube videos too. All the SUSI clients (iOS, Android, and Web) support playing youtube videos in chat.

Google provides a Youtube iFrame Player API that can be used to play videos inside the app only instead of passing an intent and playing the videos in the youtube app. iFrame API provide support for playing youtube videos in iOS applications.

In this post, we will see how playing youtube video features implemented in SUSI iOS.

Getting response from server side –

When we ask SUSI for playing any video, in response, we get youtube Video ID in video_play action type. SUSI iOS make use of Video ID to play youtube video. In response below, you can see that we are getting answer action type and in the expression of answer action type, we get the title of the video.

actions:
[
{
type: "answer",
expression: "Playing Kygo - Firestone (Official Video) ft. Conrad Sewell"
},
{
identifier: "9Sc-ir2UwGU",
identifier_type: "youtube",
type: "video_play"
}
]

Integrating youtube player in the app –

We have a VideoPlayerView that handle all the iFrame API methods and player events with help of YTPlayer HTML file.

When SUSI respond with video_play action, the first step is to register the YouTubePlayerCell and present the cell in collectionView of chat screen.

Registering the Cell –

register(_:forCellWithReuseIdentifier:) method registers a class for use in creating new collection view cells.

collectionView?.register(YouTubePlayerCell.self, forCellWithReuseIdentifier: ControllerConstants.youtubePlayerCell)

 

Presenting the YouTubePlayerCell –

Here we are presenting the cell in chat screen using cellForItemAt method of UICollectionView.

if message.actionType == ActionType.video_play.rawValue {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.youtubePlayerCell, for: indexPath) as? YouTubePlayerCell {
cell.message = message
cell.delegate = self
return cell
}
}

 

Setting size for cell –

Using sizeForItemAt method of UICollectionView to set the size.

if message.actionType == ActionType.video_play.rawValue {
return CGSize(width: view.frame.width, height: 158)
}

In YouTubePlayerCell, we are displaying the thumbnail of youtube video using UIImageView. Following method is using to get the thumbnail of particular video by using Video ID –

  1. Getting thumbnail image from URL
  2. Setting image to imageView
func downloadThumbnail() {
if let videoID = message?.videoData?.identifier {
let thumbnailURLString = "https://img.youtube.com/vi/\(videoID)/default.jpg"
let thumbnailURL = URL(string: thumbnailURLString)
thumbnailView.kf.setImage(with: thumbnailURL, placeholder: ControllerConstants.Images.placeholder, options: nil, progressBlock: nil, completionHandler: nil)
}
}

We are adding a play button in the center of thumbnail view so that when the user clicks play button, we can present player.

On clicking the Play button, we are presenting the PlayerViewController, which hold all the player setups, by overFullScreen type of modalPresentationStyle.

@objc func playVideo() {
if let videoID = message?.videoData?.identifier {
let playerVC = PlayerViewController(videoID: videoID)
playerVC.modalPresentationStyle = .overFullScreen
delegate?.loadNewScreen(controller: playerVC)
}
}

The methods above present the youtube player with giving Video ID. We are using YouTubePlayerDelegate method to autoplay the video.

func playerReady(_ videoPlayer: YouTubePlayerView) {
videoPlayer.play()
}

The player can be dismissed by tapping on the light black background.

Final Output –

Resources –

  1. Youtube iOS Player API
  2. SUSI API Sample Response for Playing Video
  3. SUSI iOS Link
Continue ReadingAdding Support for Playing Youtube Videos in SUSI iOS App

Youtube search as a Console Service Endpoint in SUSI.AI

SUSI.AI now has the ability to search and play any song or video directly in the webclient and the speaker. When a user asks for a query regarding playing a song, the clients sends a search request to the server. In this post I will discuss about the server side implementation of the Youtube search. Every time a request is made by any client, the client sends a query request to the server in the form of a json object. For more on the working on the webclient side can be seen here.

The endpoint for youtube search is  http://api.susi.ai/susi/console.json

Continue ReadingYoutube search as a Console Service Endpoint in SUSI.AI

How selected autoplay works and messages are created in webclient

In this post we will discuss the message model of SUSI.AI webclient and how youtube autoplay features selectively autoplays only the last played song from youtube. The answer or response to any query to susi is send by server in json format, which is just a raw form of data which is then used by webclient to create the message in desired form in different action types. Every response to a query is given in an array of action types with each element in the array is a snippet of HTML codes. This array is called the messageListItem. This messageListItem array is further, a part of a bigger array which contains all the codes for the query, answer and date elements. This array is called the messageListItems. This contains the final code block of all the messages displayed on the web client. All this is handled by two files MessageListItem.js and MessageSection.js and we will discuss their working below in detail.

Continue ReadingHow selected autoplay works and messages are created in webclient

Implementing YouTube Search API with WebClient

SUSI.AI is an assistant which enables us to create a lot of skills. These skills offer various functionalities which performs different actions. Therefore SUSI has implemented various action types. Some of these are:

  • Answer
  • Table
  • Maps
  • Rss
  • audio_play
  • video_play

When a user answers a query the server sends response in form of actions type. The client then scans the response JSON object and checks the type of action and performs the desired operations. This action types are verified by

Continue ReadingImplementing YouTube Search API with WebClient