Adding 3D Home Screen Quick Actions to SUSI iOS App

Home screen quick actions are a convenient way to perform useful, app-specific actions right from the Home screen, using 3D Touch. Apply a little pressure to an app icon with your finger—more than you use for tap and hold—to see a list of available quick actions. Tap one to activate it. Quick actions can be static or dynamic.

We have added some 3D home screen quick action to our SUSI iOS app. In this post, we will see how they are implemented and how they work.

The following 3D home screen quick actions are added to SUSI iOS:

  • Open SUSI Skills – user can directly go to SUSI skills without opening a chat screen.
  • Customize Settings – user can customize their setting directly by using this quick action.
  • Setup A Device – when the user quickly wants to configure his/her device for SUSI Smart Speaker, this is quick action is very helpful in that.
  • Change SUSI’s Voice – user can change SUSI message reading language accents directly from this quick action.

Each Home screen quick action includes a title, an icon on the left or right (depending on your app’s position on the home screen), and an optional subtitle. The title and subtitle are always left-aligned in left-to-right languages.

Step 1 – Adding the Shortcut Items

We add static home screen quick actions using the UIApplicationShortcutItems array in the app Info.plist file. Each entry in the array is a dictionary containing items matching properties of the UIApplicationShortcutItem class. As seen in screenshot below, we have 4 shortcut items and each item have three properties UIApplicationShortcutItemIconType/UIApplicationShortcutItemIconFile, UIApplicationShortcutItemTitle, and UIApplicationShortcutItemType.

  • UIApplicationShortcutItemIconType and UIApplicationShortcutItemIconFile is the string for setting icon for quick action. For the system icons, we use UIApplicationShortcutItemIconType property and for the custom icons, we use UIApplicationShortcutItemIconFile.
  • UIApplicationShortcutItemTitle is a required string that is displayed to the user.
  • UIApplicationShortcutItemType is a required app specific string used to identify the quick action.

Step 2 – Handling the Shortcut

AppDelegate is the place where we handle all the home screen quick actions. We define these variables:

var shortcutHandled: Bool!
var shortcutIdentifier: String?

When a user chooses one of the quick actions the launch of the system or resumes the app and calls the performActionForShortcutItem method in app delegate:

func application(_ application: UIApplication,
                     performActionFor shortcutItem: UIApplicationShortcutItem,
                     completionHandler: @escaping (Bool) -> Void) {
        shortcutIdentifier = shortcutItem.type
        shortcutHandled = true
        completionHandler(shortcutHandled)
    }

Whenever the application becomes active, applicationDidBecomeActive function is called:

func applicationDidBecomeActive(_ application: UIApplication) {
        // Handel Home Screen Quick Actions
        handelHomeActions()
    }

Inside the applicationDidBecomeActive function we call the handleHomeAction() method which handles the home screen quick action.

func handelHomeActions() {
       if shortcutHandled == true {
           shortcutHandled = false
           if shortcutIdentifier == ControllerConstants.HomeActions.openSkillAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.customizeSettingsAction {
               // Handle action accordingly
           } else if shortcutIdentifier == ControllerConstants.HomeActions.setupDeviceAction {
               // Handle action accordingly
               }
           } else if shortcutIdentifier == ControllerConstants.HomeActions.changeVoiceAction {
               // Handle action accordingly
               }
           }
       }
   }

Final Output:

Resources –

  1. Home Screen Quick Actions – Human Interface Guidelines by Apple
  2. Adding 3D Touch Quick Actions by Use Your Leaf
  3. Apple’s documentation on performActionFor:completionHandler
Continue ReadingAdding 3D Home Screen Quick Actions to SUSI iOS App

Connecting SUSI iOS App to SUSI Smart Speaker

SUSI Smart Speaker is an Open Source speaker with many exciting features. The user needs an Android or iOS device to set up the speaker. You can refer this post for initial connection to SUSI Smart Speaker. In this post, we will see how a user can connect SUSI Smart Speaker to iOS devices (iPhone/iPad).

Implementation –

The first step is to detect whether an iOS device connects to SUSI.AI hotspot or not. For this, we match the currently connected wifi SSID with SUSI.AI hotspot SSID. If it matches, we show the connected device in Device Activity to proceed further with setups.

Choosing Room –

Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them.

When the user clicks on Wi-Fi displayed cell, it starts the initial setups. We are using didSelectRowAt method of UITableViewDelegate to get which cell is selected. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field.

override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
if indexPath.row == 0, let speakerSSID = fetchSSIDInfo(), speakerSSID == ControllerConstants.DeviceActivity.susiSSID {
// Open a popup to select Rooms
presentRoomsPopup()
}
}

When the user clicks the Next button, we send the speaker room location to the local server of the speaker by the following API endpoint with room name as a parameter:

http://10.0.0.1:5000/speaker_config/

Refer this post for getting more detail about how choosing room work and how it is implemented in SUSI iOS.

Sharing Wi-Fi Credentials –

On successfully choosing the room, we present a popup that asks the user to enter the Wi-Fi credentials of previously connected Wi-Fi so that we can connect our Smart Speaker to the wifi which can provide internet connection to play music and set commands over the speaker.

We present a popup with a text field for entering wifi password.

When the user clicks the Next button, we share the wifi credentials to wifi by the following API endpoint:

http://10.0.0.1:5000/wifi_credentials/

With the following params-

  1. Wifissid – Connected Wi-Fi SSID
  2. Wifipassd – Connected Wi-Fi password

In this API endpoint, we are sharing wifi SSID and wifi password with Smart Speaker. If the credentials successfully accepted by speaker than we present a popup for user SUSI account password, otherwise we again present Enter Wifi Credentials popup.

Client.sharedInstance.sendWifiCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.presentUserPasswordPopup()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentWifiCredentialsPopup()
})
}
}
}

 

Sharing SUSI Account Credentials –

In the method above we have seen that when SUSI Smart Speaker accept the wifi credentials, we proceed further with SUSI account credentials. We open a popup to Enter user’s SUSI account password:

When the user clicks the Next button, we use following API endpoint to share user’s SUSI account credentials to SUSI Smart Speaker:

http://10.0.0.1:5000/auth/

With the following params-

  1. email
  2. password

User email is already saved in the device so the user doesn’t have to type it again. If the user credentials successfully accepted by speaker then we proceed with configuration process otherwise we open up Enter Password popup again.

Client.sharedInstance.sendAuthCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.setConfiguration()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentUserPasswordPopup()
})
}
}
}

 

Setting Configuration –

After successfully sharing SUSI account credentials, following API endpoint is using for setting configuration.

http://10.0.0.1:5000/config/

With the following params-

  1. sst
  2. tts
  3. hotword
  4. wake

The success of this API call makes successfully connection between user iOS Device and SUSI Smart Speaker.

Client.sharedInstance.setConfiguration(params) { (success, message) in
DispatchQueue.main.async {
if success {
// Successfully Configured
self.isSetupDone = true
self.view.makeToast(ControllerConstants.DeviceActivity.doneSetupDetailText)
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
})
}
}
}

After successful connection-

 

Resources –

  1. Apple’s Documentation of tableView(_:didSelectRowAt:) API
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. SUSI Linux Link: https://github.com/fossasia/susi_linux
  4. Adding Option to Choose Room for SUSI Smart Speaker in iOS App
Continue ReadingConnecting SUSI iOS App to SUSI Smart Speaker

Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad

You may have experienced Apple HomPad, Google Home, Alexa etc or read about smart speakers that offer interactive action over voice commands. The smart speaker uses the hot word for activation. They utilize Wi-Fi, Bluetooth, and other wireless protocols.

SUSI.AI is also coming with Open Source smart speaker that can do various actions like playing music etc over voice commands. To use SUSI Smart Speaker, you have to connect it to the SUSI iOS or Android App. You can manage your connected devices in SUSI iOS, Android and Web clients. Here we will see initial setups for connecting SUSI Smart Speaker with iPhone/iPad (iOS Devices).

You may aware that iOS does not allow connecting to wifi within the app. To connect to a particular Wi-Fi, you have to go to phone settings, from there you can connect to Wi-Fi. SUSI Smart Speaker create a temporary Hotspot for initial setups. Follow the instruction below to connect to SUSI Smart Speaker hotspot –

  1. Tap to Home button, and go to your iPhone Settings > Wi-Fi
  2. Connect to the Wi-Fi hotspot for the device that you are setting up. It will have name “susi.ai”, like in the image below
  3. Come back to the SUSI app to proceed with setup.

These instruction is also available within the app when you are not connected to SUSI Smart Speaker hotspot and click `Setup a Device` or plus icon on Device Activity screen navigation bar.

Devices Activity and getting current Wi-Fi SSID:

Devices section in Settings screen shows the currently connected device. In Devices Activity screen, the user can manage the connected device. Only a logged-in user can access Devices Activity. When the user clicks on Device Accessories in setting, if the user is not logged-in, an alert is prompted with Login option. By clicking Login option, user directed to Login screen where the user can log in and come back to device section to proceed further.

If the user is already logged-in, Device Activity screen is presented. We use following method to scan if iPhone/iPad is connected to SUSI Smart Speaker:

func fetchSSIDInfo() -> String? {
var ssid: String?
if let interfaces = CNCopySupportedInterfaces() as? [String] {
for interface in interfaces {
if let interfaceInfo = CNCopyCurrentNetworkInfo(interface as CFString) as NSDictionary? {
ssid = interfaceInfo[kCNNetworkInfoKeySSID as String] as? String
break
}
}
}
return ssid
}

Apple’s SystemConfiguration API is used to get current Wi-Fi SSID. SystemConfiguration Allow applications to access a device’s network configuration settings. Determine the reachability of the device, such as whether Wi-Fi or cell connectivity is active.

import SystemConfiguration.CaptiveNetwork

The method above return the SSID of your device current Wi-Fi. SSID is simply the technical term for a network name. When you set up a wireless home network, you give it a name to distinguish it from other networks in your neighborhood. You’ll see this name when you connect your device to your wireless network.

If current Wi-Fi match with SUSI Smart Speaker hotspot, we display device in TableView, if not we display “No device connected yet”.

if let speakerSSID = fetchSSIDInfo(), speakerSSID == "susi.ai" {
cell.accessoryType = .disclosureIndicator
cell.textLabel?.text = speakerSSID
} else {
cell.accessoryType = .none
cell.textLabel?.text = "No device connected yet"
}

SUSI Smart Speaker is coming with very exciting features. Stay tuned.

Resources –

  1. SUSI iOS Link: https://github.com/fossasia/susi_iOS
  2. Apple’s SystemConfiguration Framework Documentation
  3. Bell’s article on What Do SSID and WPA2 mean
Continue ReadingInitial Setups for Connecting SUSI Smart Speaker with iPhone/iPad