Connecting SUSI iOS App to SUSI Smart Speaker

SUSI Smart Speaker is an Open Source speaker with many exciting features. The user needs an Android or iOS device to set up the speaker. You can refer this post for initial connection to SUSI Smart Speaker. In this post, we will see how a user can connect SUSI Smart Speaker to iOS devices (iPhone/iPad).

Implementation –

The first step is to detect whether an iOS device connects to SUSI.AI hotspot or not. For this, we match the currently connected wifi SSID with SUSI.AI hotspot SSID. If it matches, we show the connected device in Device Activity to proceed further with setups.

Choosing Room –

Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them.

When the user clicks on Wi-Fi displayed cell, it starts the initial setups. We are using didSelectRowAt method of UITableViewDelegate to get which cell is selected. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field.

override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
if indexPath.row == 0, let speakerSSID = fetchSSIDInfo(), speakerSSID == ControllerConstants.DeviceActivity.susiSSID {
// Open a popup to select Rooms
presentRoomsPopup()
}
}

When the user clicks the Next button, we send the speaker room location to the local server of the speaker by the following API endpoint with room name as a parameter:

http://10.0.0.1:5000/speaker_config/

Refer this post for getting more detail about how choosing room work and how it is implemented in SUSI iOS.

Sharing Wi-Fi Credentials –

On successfully choosing the room, we present a popup that asks the user to enter the Wi-Fi credentials of previously connected Wi-Fi so that we can connect our Smart Speaker to the wifi which can provide internet connection to play music and set commands over the speaker.

We present a popup with a text field for entering wifi password.

When the user clicks the Next button, we share the wifi credentials to wifi by the following API endpoint:

http://10.0.0.1:5000/wifi_credentials/

With the following params-

  1. Wifissid – Connected Wi-Fi SSID
  2. Wifipassd – Connected Wi-Fi password

In this API endpoint, we are sharing wifi SSID and wifi password with Smart Speaker. If the credentials successfully accepted by speaker than we present a popup for user SUSI account password, otherwise we again present Enter Wifi Credentials popup.

Client.sharedInstance.sendWifiCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.presentUserPasswordPopup()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentWifiCredentialsPopup()
})
}
}
}

 

Sharing SUSI Account Credentials –

In the method above we have seen that when SUSI Smart Speaker accept the wifi credentials, we proceed further with SUSI account credentials. We open a popup to Enter user’s SUSI account password:

When the user clicks the Next button, we use following API endpoint to share user’s SUSI account credentials to SUSI Smart Speaker:

http://10.0.0.1:5000/auth/

With the following params-

  1. email
  2. password

User email is already saved in the device so the user doesn’t have to type it again. If the user credentials successfully accepted by speaker then we proceed with configuration process otherwise we open up Enter Password popup again.

Client.sharedInstance.sendAuthCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.setConfiguration()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentUserPasswordPopup()
})
}
}
}

 

Setting Configuration –

After successfully sharing SUSI account credentials, following API endpoint is using for setting configuration.

http://10.0.0.1:5000/config/

With the following params-

  1. sst
  2. tts
  3. hotword
  4. wake

The success of this API call makes successfully connection between user iOS Device and SUSI Smart Speaker.

Client.sharedInstance.setConfiguration(params) { (success, message) in
DispatchQueue.main.async {
if success {
// Successfully Configured
self.isSetupDone = true
self.view.makeToast(ControllerConstants.DeviceActivity.doneSetupDetailText)
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
})
}
}
}

After successful connection-

 

Resources –

  1. Apple’s Documentation of tableView(_:didSelectRowAt:) API
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. SUSI Linux Link: https://github.com/fossasia/susi_linux
  4. Adding Option to Choose Room for SUSI Smart Speaker in iOS App
Continue ReadingConnecting SUSI iOS App to SUSI Smart Speaker
Read more about the article Adding Option to Choose Room for SUSI Smart Speaker in iOS App
SAMSUNG CAMERA PICTURES

Adding Option to Choose Room for SUSI Smart Speaker in iOS App

SUSI Smart Speaker is an open source smart speaker that supports many features. The user can use Android or iOS to connect their device with SUSI Smart Speaker. During initial installation, it is asking from use to enter the Room name. Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them. You can find useful instructions for the initial connection between the iOS device and SUSI Smart Speaker here. It this post, we will see how the adding rooms feature implemented for SUSI iOS.

When the user enters into the Device Activity screen, we check if the iOS device is connected to SUSI.AI Wi-Fi hotspot or not. If the device is connected to SUSI Smart Speaker, it shows the Wi-Fi displayed SSID in Device Activity Screen. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field. The user can enter Room location and by clicking the Next button, proceed further with the setup.

In the popup, there is also an option for choosing rooms, where the list of most common room names is displayed and the user can choose room name from the list.

Presenting Room Picker View Controller –

func presentRoomsPicker() {
let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let roomVC = storyboard.instantiateViewController(withIdentifier: "RoomPickerVC") as? RoomPickerController {
roomVC.deviceActivityVC = self
let roomNVC = AppNavigationController(rootViewController: roomVC)
self.present(roomNVC, animated: true)
}
}

Room Picker View Controller is UITableViewController that display the rooms names in table cells. Some of the most common rooms names displayed are:

let rooms: [String] = ["Bedroom", "Kitchen", "Family Room", "Entryway", "Living Room", "Front Yard", "Guest Room", "Dining Room", "Computer Room", "Downstairs", "Front Porch", "Garage", "Hallway", "Driveway"]

 

Presenting Room Cell –

We are using cellForRowAt method to present the cell.

override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCell(withIdentifier: "roomsCell", for: indexPath)
cell.textLabel?.text = rooms[indexPath.row]
cell.imageView?.image = ControllerConstants.Images.roomsIcon
return cell
}

 

Selecting the room from cell –

When the user clicks on the cell, first willSelectRowAt method use to display the right icon in the accessory view that shows which cell is selected.

if let oldIndex = tableView.indexPathForSelectedRow {
tableView.cellForRow(at: oldIndex)?.accessoryType = .none
}
tableView.cellForRow(at: indexPath)?.accessoryType = .checkmark

We are storing the selected room in the following variable and selecting it by using didSelectRowAt method of UITableView.

var selectedRoom: String?
override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
selectedRoom = rooms[indexPath.row]
}

In Room Picker Screen, the user has two option, Cancel and Done. If the user clicks the Cancel, we dismiss the Room Picker screen and display the popup with the empty room location text field and with Choose Room option. If the user clicks the Done button, we store the picked room in UserDefaults shared instance and dismiss Room Picker screen with a different popup which has already filled room location in the text field and without choose room option in the popup as in the image below. By clicking the next, the user proceeds with the further setup.

Resources –

  1. Apple’s Documentations on UserDefaults.
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. Apple’s Documentations on tableView(_:cellForRowAt:)
  4. Apple’s Documentations on tableView(_:willSelectRowAt:)
  5. Apple’s Documentations on tableView(_:didSelectRowAt:)
Continue ReadingAdding Option to Choose Room for SUSI Smart Speaker in iOS App

Using SUSI.AI Accounting Model to store Device information on Server

Just like with Google Home devices and Amazon Alexa devices, SUSI.AI Users should have a way to add and store the information of their devices (Smart Speakers) on SUSI Server, so that it could be displayed to them on various clients. Hence, we needed a Servlet which could add and store the User data on the Server.

Implementation of Servlets in SUSI.AI

All servlets in SUSI extend AbstractAPIHandler class and implement APIHandler. All servlets have 4 methods, which we overwrite depending on what we want the servlet to do. They are as follows :

@Override
    public String getAPIPath() {
        return null;
    }

@Override
    public BaseUserRole getMinimalBaseUserRole() {
        return null;
    }

@Override
    public JSONObject getDefaultPermissions(BaseUserRole baseUserRole) {
        return null;
    }

@Override
    public ServiceResponse serviceImpl(Query post, HttpServletResponse response, Authorization rights, JsonObjectWithDefault permissions) throws APIException {
        Return null;
    }

 

How these 4 methods work together?

  1. First method is getAPIPath(). It returns the endpoint of the servlet.
  2. The second method is getMinimalBaseUserRole(). It returns the minimum privilege level required to access the endpoint.
  3. The third method is getDefaultPermissions(). It gets the Default Permissions of a UserRole in SUSI Server. Different UserRoles have different permissions defined in SUSI Server.
  4. Whenever the endpoint defined in the getAPIPath() method is called properly, it responds with whatever is defined in the fourth method, which is serviceImpl().

How is User data stored on SUSI Server?

Before we move on to the actual implementation of the API required to store the device information, we should get a brief idea of how exactly User data is stored on SUSI Server.

Every SUSI User has an Accounting Object. It is a JSONObject which stores the Settings of the particular User on the Server. For example, every time you change some setting on the Web Client https://chat.susi.ai, an API call is made to aaa/changeUserSettings.json with appropriate parameters with information of the changed setting, and the User settings are stored to the Server in the Accounting Object of the User.

Implementation of a Servlet to store Device information on Server

The task of this servlet is to store the information of a new device (Smart speaker) whenever it is initially set up using the Android/iOS app.

This is the implementation of the 4 methods of a servlet which is used to store information of connected devices:

    @Override
    public String getAPIPath() {
        return "/aaa/addNewDevice.json";
    }

    @Override
    public UserRole getMinimalUserRole() {
        return UserRole.USER;
    }

    @Override
    public JSONObject getDefaultPermissions(UserRole baseUserRole) {
        return null;
    }

    @Override
   public ServiceResponse serviceImpl(Query query, HttpServletResponse response, Authorization authorization, JsonObjectWithDefault permissions) throws APIException {

           JSONObject value = new JSONObject();
           
           String key = query.get("macid", null);
           String name = query.get("name", null);
           String device = query.get("device", null);
               
           if (key == null || name == null || device == null) {
               throw new APIException(400, "Bad service call, missing arguments");
           } else {
               value.put(name, device);
           }
           
           if (authorization.getIdentity() == null) {
               throw new APIException(400, "Specified user data not found, ensure you are logged in");
           } 
           else {
                Accounting accounting = DAO.getAccounting(authorization.getIdentity());
                if (accounting.getJSON().has("devices")) {
                       accounting.getJSON().getJSONObject("devices").put(key, value);
                   } 
                else {
                       JSONObject jsonObject = new JSONObject();
                       jsonObject.put(key, value);
                       accounting.getJSON().put("devices", jsonObject);
                   }
               
               JSONObject result = new JSONObject(true);
               result.put("accepted", true);
               result.put("message", "You have successfully added the device!");
               return new ServiceResponse(result);
           }
    }

 

As it can be seen from the above code, the endpoint for this servlet is /aaa/addNewDevice.json and it accepts 3 parameters –

  • macid : Mac Address of the device
  • name : Name of the device
  • device : Additional information which you want to send about the device (subject to changes)

As the main task of this servlet is user specific, and should only be accessible to the particular user, hence we returned UserRole as USER in the getMinimalUserRole() method.

In the serviceImpl() method, first we extract the value of the URL query parameters and store them in variables. If any of the parameters is missing, we display an error response code of 400 with error message “Bad service call, missing arguments”. If query parameters are fine, we store the name and device values in a new JSONObject, value in this case.

An if-else statement then checks for whether the User is logged in or not, using the authorization.getIdentity(), which is a function which returns the identity of the User. The implementation of this function is in Accounting.java file, and is as follows :

    public ClientIdentity getIdentity() {
        return identity;
    }

 

If the User is logged in, getAccounting() function of DAO.java file is called which returns an Accounting object according to the following implementation of the function :

    public static Accounting getAccounting(@Nonnull ClientIdentity identity) {
        return new Accounting(identity, accounting);
    }

 

Our device information is then stored in this Accounting object in the following format :

{
  "lastLoginIP": "162.158.166.19",
  "accepted": true,
  "message": "Success: Showing user data",
  "session": {"identity": {
    "type": "email",
    "name": "myemail@gmail.com",
    "anonymous": false
  }},
  "settings": {
    "customThemeValue": "4285f4,f5f4f6,fff,f5f4f6,fff,4285f4,",
    "theme": "light"
  },
  "devices": {
    "MacID2": {"MyDevice": "SmartSpeaker"},
    "MacID1": {"Name of device": "SmartSpeaker"}
  }
}

 

Resources

Continue ReadingUsing SUSI.AI Accounting Model to store Device information on Server