“STOP” action in SUSI Android App

Generally whenever there was a long query asked from SUSI through speech, it would respond to the user with a speech output, similarly the output is given through speech whenever the user clicks on the “Try it” button on the skill details activity.

The long answers for e.g. asking SUSI “How to cook biryani ?” gives a very long response which when conveyed through the speech output takes a long amount of time. Also most of the time users don’t want to listen to such long answers, but at the same time there is no option or feature to stop SUSI. The user either needs to switch over to another activity or has to close the app.

So, to solve this problem such that neither the user has to shut down the app nor the user has to switch over to another activity, the “STOP” action was added in SUSI.

The “STOP” action was integrated in the server and is of following type :

 

“actions”: [{“type”: “stop”}],

How to define STOP response ?

To integrate this action type in the app, a separate response type was added and checked for. In the file ParseSusiResponseHandler.kt the action type stop was added as :

Constant.STOP -> try {
  stop = susiResponse.answers[0].actions[1].type
} catch (e: Exception) {

}

So, now whenever the query of type “stop” was entered by the user it would be caught by this block and further processing could be done. The stop variable takes the value from the JSON response fetched and the value extracted is the one stored against the type key in the second field of the actions JSON in the first field of the answers JSON.

What to be done when executing STOP?

After the STOP action was caught the task to do was to define the behaviour of the app when this response was caught. So, first thing that should happen when STOP is received is that the TextToSpeech Engine should close so that SUSI no longer can speak the sentence but defining top is more than just closing the TTS engine. STOP signifies that any activity that is being continued right now should be stopped.

So, Using the android lifecycle methods I added a function in the IChatView interface to define what actions should take place in the stopping process.

override fun stopMic() {
  onPause()
  registerReceiver(networkStateReceiver, IntentFilter(ConnectivityManager.CONNECTIVITY_ACTION))

  window.setSoftInputMode(
          WindowManager.LayoutParams.SOFT_INPUT_STATE_ALWAYS_HIDDEN
  )

  if (recordingThread != null)
      chatPresenter.startHotwordDetection()

  if (etMessage.text.toString().isNotEmpty()) {
      btnSpeak.setImageResource(R.drawable.ic_send_fab)
      etMessage.setText(“”)
      chatPresenter.micCheck(false)
  }

  chatPresenter.checkPreferences()
}

The above function sends the activity to the onPause() lifecycle method to put the activity in a pausing state so that all that is taking place right now in the activity stops and this definitely serves our purpose. But after pausing the activity, there was a further need to perform some functionality that had to done on the start of the activity and therefore the code for that was also added in this function.

How to catch STOP?

The below code was added in ChatPresenter.kt file in which if the actionType from the psh object of type ParseSusiResponseHelper is “STOP” then the view function stopMic() is called which was defined above.

val psh = ParseSusiResponseHelper()
psh.parseSusiResponse(susiResponse, i, utilModel.getString(R.string.error_occurred_try_again))

var setMessage = psh.answer
if (psh.actionType == Constant.ANSWER && (PrefManager.checkSpeechOutputPref() && check || PrefManager.checkSpeechAlwaysPref())) {
  setMessage = psh.answer

  var speechReply = setMessage
  if (psh.isHavingLink) {
      speechReply = setMessage.substring(0, setMessage.indexOf(“http”))
  }
  chatView?.voiceReply(speechReply, susiResponse.answers[0].actions[i].language)
} else if (psh.actionType == Constant.STOP) {
  setMessage = psh.stop
  chatView?.stopMic()
}

Final Output

References

  1. Stop  json response from susi server : https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=susi+stop
  2. Android life cycle methods – Google: https://developer.android.com/guide/components/activities/activity-lifecycle
  3. Interaction between view and presenters : https://medium.com/@cervonefrancesco/model-view-presenter-android-guidelines-94970b430ddf
Continue Reading

Displaying Skills Feedback on SUSI iOS

SUSI allows the user to rate the SUSI Skills with the five-star rating system. SUSI offer a good feedback system where the user can post feedback to any skill by using iOS, Android, and Web clients. In Skill Detail, there is a skill feedback text field where the user can write feedback about SUSI Skill. We display the users posted feedbacks on Skill Detail screen. In this post, we will see how the displaying skills feedback feature implemented on SUSI iOS.

Implementation –

We are displaying three feedback on Skill Detail screen, to see all feedback, there is a “See All Review” option, by clicking user is directed to a new screen where he/she can see all feedback related to particular skill.

We use the endpoint below for getting skill feedback from server side –

https://api.susi.ai/cms/getSkillFeedback.json

With the following params:

  • Model
  • Group
  • Language
  • Skill Name

The API endpoint above return the all the feedback array related to particular susi skill. We store feedbacks in an array of Feedback object, which holds three value:

    • Feedback String – Feedback string posted by the user
    • Email – Email address of feedback poster user
    • Time Stamp – Time of posting feedback
class Feedback: NSObject {
var feedbackString: String = ""
var email: String = ""
var timeStamp: String = ""
...
}

To display feedbacks, we are using UITableView with two prototype cells, one for feedbacks and one for “See All Review” option.

There can be different cases eg. when the total number of feedback for skill is less than three or three. When the feedback count is three or less than three, there is no need to show “See All Review” option. Also, tableView height is different for different feedback count. For varying tableView height, we have created an outlet for tableView height constraints and vary accordingly.

@IBOutlet weak var feedbackTableHeighConstraint: NSLayoutConstraint!

Now, let’s see how the number of cells, height for cells and different cells presented according to feedback count with UITableViewDelegate and UITableViewDataSource methods.

Handling number of tableView rows –

func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
guard let feedbacks = feedbacks, feedbacks.count > 0 else {
feedbackTableHeighConstraint.constant = 0.0
return 0
}
if feedbacks.count < 4 {
feedbackTableHeighConstraint.constant = CGFloat(72 * feedbacks.count)
return feedbacks.count
} else {
feedbackTableHeighConstraint.constant = 260.0
return 4
}
}

Where feedbacks is the array of Feedback object which holds the feedbacks we are getting from the server side for a skill.

var feedbacks: [Feedback]?

In the above method, we see that how we are handling the number of cells case. Now let’s see how to handle which cells to be present on basis of the number of cells case –

func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
guard let feedbacks = feedbacks, feedbacks.count > 0 else {
feedbackTableHeighConstraint.constant = 0.0
return UITableViewCell()
}
if feedbacks.count < 4 {
if let cell = tableView.dequeueReusableCell(withIdentifier: "feedbackDisplayCell", for: indexPath) as? FeedbackDisplayCell {
cell.feedback = feedbacks[indexPath.row - Int(indexPath.row/2)]
return cell
}
} else if feedbacks.count > 3 {
if indexPath.row == 3 {
let cell = tableView.dequeueReusableCell(withIdentifier: "allFeedbackCell", for: indexPath)
return cell
} else {
if let cell = tableView.dequeueReusableCell(withIdentifier: "feedbackDisplayCell", for: indexPath) as? FeedbackDisplayCell {
cell.feedback = feedbacks[indexPath.row]
return cell
}
}
}
return UITableViewCell()
}

If the number of feedbacks is greater than three than we provide “See All Review” option to the user to see all the feedback related to skill. We are displaying all feedbacks using UITableViewController. When the user clicks the “See All Review” option, we pass the feedbacks (Array of all the feedback) to new UITableViewController. By passing feedbacks, we are reducing one network call.

let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let allFeedbackVC = storyboard.instantiateViewController(withIdentifier: "AllFeedbackController") as? AllFeedbackViewController {
allFeedbackVC.feedbacks = self.feedbacks
let nvc = AppNavigationController(rootViewController: allFeedbackVC)
self.present(nvc, animated: true, completion: nil)
}

On all skill feedback screen, we are displaying the full review. For the different size of text, we are setting the different size of cell size by using the method below:

if let feedbacks = feedbacks {
let estimatedLabelHeight = UILabel().heightForLabel(text: feedbacks[indexPath.row].feedbackString, font: UIFont.systemFont(ofSize: 14.0), width: 250.0)
return 64 + estimatedLabelHeight
} else {
return 80
}

 

Final Output –

Resources –

  1. SUSI API Link: https://api.susi.ai/
  2. SUSI iOS Link: https://github.com/fossasia/susi_iOS
  3. Apple’s Documentation on UITableViewDelegate: https://developer.apple.com/documentation/uikit/uitableviewdelegate?changes=_6
  4. Apple’s Documentation on UITableViewDataSource: https://developer.apple.com/documentation/uikit/uitableviewdatasource
Continue Reading

Change Text-to-Speech Voice Language of SUSI in SUSI iOS

SUSI iOS app now enables the user to change the text-to-speech voice language within the app. Now, the user can select any language of their choice from the list of 37 languages list. To change the text-to-speech voice language, go to, Settings > Change SUSI’s Voice, choose the language of your choice. Let see here how this feature implemented.

Apple’s AVFoundation API is used to implement the text-to-speech feature in SUSI iOS. AVFoundation API offers 37 voice languages which can be used for text-to-speech voice accent. AVFoundation’s AVSpeechSynthesisVoice API can be used to select a voice appropriate to the language of the text to be spoken or to select a voice exhibiting a particular local variant of that language (such as Australian or South African English).

To print the list of all languages offered by AVFoundation:

import AVFoundation

print(AVSpeechSynthesisVoice.speechVoices())

Or the complete list of supported languages can be found at Languages Supported by VoiceOver.

When the user clicks Change SUSI’s voice in settings, a screen is presented with the list of available languages with the language code.

Dictionary holds the list of available languages with language name and language code and used as Data Source for tableView.

var voiceLanguagesList: [Dictionary<String, String>] = []

When user choose the language and click on done, we store language chosen by user in UserDefaults:

UserDefaults.standard.set(voiceLanguagesList[selectedVoiceLanguage][ControllerConstants.ChooseLanguage.languageCode], forKey: ControllerConstants.UserDefaultsKeys.languageCode)
UserDefaults.standard.set(voiceLanguagesList[selectedVoiceLanguage][ControllerConstants.ChooseLanguage.languageName], forKey: ControllerConstants.UserDefaultsKeys.languageName)

Language name with language code chosen by user displayed in settings so the user can know which language is currently being used for text-to-speech voice.

To select a voice for use in speech, we obtain an AVSpeechSynthesisVoice instance using one of the methods in Finding Voices and then set it as the value of the voice property on an AVSpeechUtterance instance containing text to be spoken.

Earlier stored language code in UserDefaults shared instance used for setting the text-to-speech language for AVSpeechSynthesisVoice.

if let selectedLanguage = UserDefaults.standard.object(forKey: ControllerConstants.UserDefaultsKeys.languageCode) as? String {
speechUtterance.voice = AVSpeechSynthesisVoice(language: selectedLanguage)
}

AVSpeechUtterance is responsible for a chunk of text to be spoken, along with parameters that affect its speech.

Resources –

  1. UserDefaults: https://developer.apple.com/documentation/foundation/userdefaults
  2. AVSpeechSynthesisVoice: https://developer.apple.com/documentation/avfoundation/avspeechsynthesisvoice
  3. AVFoundation: https://developer.apple.com/av-foundation/
  4. SUSI iOS Link: https://github.com/fossasia/susi_iOS
Continue Reading

Displaying skill rating for each skill on skill page of SUSI SKILL CMS

SUSI exhibits several skills which are managed by the SUSI Skill CMS, it essentially is a client which allows users to create/update skills conveniently since for each skill it is important to have the functionality of rating system so developers can get to know which skills are performing better than the rest and consequently improve them, thus a skill rating system which allows the users to give positive or negative feedback for each skill is implemented on the server.

Fetching skill_rating from the server

  1. Fetch skill data for which ratings are to be displayed through ajax calls
    API Endpoint –

    /cms/getSkillMetadata.json?
    

  2. Parse the received metadata object to get positive and negative ratings for that skill
  3. if(skillData.skill_rating) {
           	let positive_rating = skillData.skill_rating.positive;
            	let negative_rating = skillData.skill_rating.negative;
    }
    

    Sample API response

    {
      "skill_metadata": {
        "model": "general",
        "group": "Knowledge",
        "language": "en",
        "developer_privacy_policy": null,
        "descriptions": "Want to know about fossasia, just ask susi to tell that, Susi tells about the SUSI.AI creators",
        "image": "images/creator_info.png",
        "author": "madhav rathi",
        "author_url": "https://github.com/madhavrathi",
        "author_email": null,
        "skill_name": "Creator Info",
        "terms_of_use": null,
        "dynamic_content": false,
        "examples": [
          "Who created you?",
          "what is fossasia?"
        ],
        "skill_rating": {
          "negative": "0",
          "positive": "0",
          "stars": {
            "one_star": 0,
            "four_star": 0,
            "five_star": 0,
            "total_star": 0,
            "three_star": 0,
            "avg_star": 0,
            "two_star": 0
          },
          "feedback_count": 0
        },
        "creationTime": "2018-03-17T16:38:29Z",
        "lastAccessTime": "2018-06-15T15:51:50Z",
        "lastModifiedTime": "2018-03-17T16:38:29Z"
      },
      "accepted": true,
      "message": "Success: Fetched Skill's Metadata",
      "session": {"identity": {
        "type": "host",
        "name": "162.158.166.37_d80fb5c9",
        "anonymous": true
      }}
    }
    

  4. Set the react state of the component to store positive and negative rating.
  5. this.setState({
      positive_rating,
      negative_rating
    })
    

  6. Use react-icons to fetch like and dislike icon components from font-awesome.
  7. npm i -S react-icons
    

  8. Import the corresponding icons in the SkillPage component
  9. import { FaThumbsOUp, FaThumbsODown } from 'react-icons/lib/fa/'
    

  10. Display the rating count along with their icons
  11. <div className="rating">
        <div className="positive">
             <FaThumbsOUp />
             {this.state.positive_rating}
         </div>
           <div className="negative">
                 <FaThumbsODown />
                 {this.state.negative_rating}
             </div>
    </div>
    

Example

References

Continue Reading

Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad

You may have experienced Apple HomPad, Google Home, Alexa etc or read about smart speakers that offer interactive action over voice commands. The smart speaker uses the hot word for activation. They utilize Wi-Fi, Bluetooth, and other wireless protocols.

SUSI.AI is also coming with Open Source smart speaker that can do various actions like playing music etc over voice commands. To use SUSI Smart Speaker, you have to connect it to the SUSI iOS or Android App. You can manage your connected devices in SUSI iOS, Android and Web clients. Here we will see initial setups for connecting SUSI Smart Speaker with iPhone/iPad (iOS Devices).

You may aware that iOS does not allow connecting to wifi within the app. To connect to a particular Wi-Fi, you have to go to phone settings, from there you can connect to Wi-Fi. SUSI Smart Speaker create a temporary Hotspot for initial setups. Follow the instruction below to connect to SUSI Smart Speaker hotspot –

  1. Tap to Home button, and go to your iPhone Settings > Wi-Fi
  2. Connect to the Wi-Fi hotspot for the device that you are setting up. It will have name “susi.ai”, like in the image below
  3. Come back to the SUSI app to proceed with setup.

These instruction is also available within the app when you are not connected to SUSI Smart Speaker hotspot and click `Setup a Device` or plus icon on Device Activity screen navigation bar.

Devices Activity and getting current Wi-Fi SSID:

Devices section in Settings screen shows the currently connected device. In Devices Activity screen, the user can manage the connected device. Only a logged-in user can access Devices Activity. When the user clicks on Device Accessories in setting, if the user is not logged-in, an alert is prompted with Login option. By clicking Login option, user directed to Login screen where the user can log in and come back to device section to proceed further.

If the user is already logged-in, Device Activity screen is presented. We use following method to scan if iPhone/iPad is connected to SUSI Smart Speaker:

func fetchSSIDInfo() -> String? {
var ssid: String?
if let interfaces = CNCopySupportedInterfaces() as? [String] {
for interface in interfaces {
if let interfaceInfo = CNCopyCurrentNetworkInfo(interface as CFString) as NSDictionary? {
ssid = interfaceInfo[kCNNetworkInfoKeySSID as String] as? String
break
}
}
}
return ssid
}

Apple’s SystemConfiguration API is used to get current Wi-Fi SSID. SystemConfiguration Allow applications to access a device’s network configuration settings. Determine the reachability of the device, such as whether Wi-Fi or cell connectivity is active.

import SystemConfiguration.CaptiveNetwork

The method above return the SSID of your device current Wi-Fi. SSID is simply the technical term for a network name. When you set up a wireless home network, you give it a name to distinguish it from other networks in your neighborhood. You’ll see this name when you connect your device to your wireless network.

If current Wi-Fi match with SUSI Smart Speaker hotspot, we display device in TableView, if not we display “No device connected yet”.

if let speakerSSID = fetchSSIDInfo(), speakerSSID == "susi.ai" {
cell.accessoryType = .disclosureIndicator
cell.textLabel?.text = speakerSSID
} else {
cell.accessoryType = .none
cell.textLabel?.text = "No device connected yet"
}

SUSI Smart Speaker is coming with very exciting features. Stay tuned.

Resources –

  1. SUSI iOS Link: https://github.com/fossasia/susi_iOS
  2. Apple’s SystemConfiguration Framework Documentation
  3. Bell’s article on What Do SSID and WPA2 mean
Continue Reading

Post feedback for SUSI Skills in SUSI iOS

SUSI iOS, web and Android clients allow the user to rate the SUSI Skills in a 5-star rating system. Users can write about how much particular skill is helpful for them or if improvements are needed. Users can rate skills from one to five star as well. Here we will see how to submit feedback for SUSI skills and how it is implemented on SUSI iOS.

How to submit feedback –

  1. Go to Skill Listing Screen < Skill Detail Screen
  2. Scroll to the feedback section
  3. Write feedback about SUSI skill
  4. Click on POST button to post the skill feedback

An anonymous user can not submit skill feedback. You must have to logged-in in order to post skill feedback. If you are not logged-in and click POST button to post skill feedback, an alert is presented with Login option, by clicking Login, the user is directed to Login screen where the user can log in and later can post skill feedback.

Implementation of posting skill feedback –

Google’s Material textfield is used for skill feedback text field. We have assigned TextField class from Material target to skill feedback text field to make it very interactive and give better user experience.

Skill feedback text field in the normal state –

Skill feedback text field in the active state –

When the user clicks POST after writing skill feedback, we check if the user is logged-in or not.

if let delegate = UIApplication.shared.delegate as? AppDelegate, let user = delegate.currentUser {
...
}

We have saved the logged-in user globally using AppDelegate shared method during login and using it here. The AppDelegate is sort of like the entry point for the application. It implements UIApplicationDelegate and contains methods that are called when application launches, when is going to the background (i.e. when the user hit the home key), when it’s opened back up, and more. The AppDelegate object is stored as a property on the UIApplication class and is accessible from anywhere in swift classes.

Case 1: If the user is not logged-in, we show a popup to the user with the login option

By clicking Login, the user is directed to Login screen where the user can log in and later can post skill feedback.

Case 2: If the user is already logged-in, we use the endpoint below for posting skill feedback –

http://api.susi.ai/cms/feedbackSkill.json

ModelWith following parameters –

  • Group
  • Skill
  • Feedback
  • Access token
Client.sharedInstance.postSkillFeedback(postFeedbackParam) { (feedback, success, responseMessage) in
DispatchQueue.main.async {
if success {
self.skillFeedbackTextField.text = ""
self.skillFeedbackTextField.resignFirstResponder()
self.view.makeToast(responseMessage)
} else {
self.view.makeToast(responseMessage)
}
}
}

In return response, we get feedback posted by the user –

{
feedback: "Helpful",
session:
{
...
},
accepted: true,
message: "Skill feedback updated"
}

 

Resources –

  1. Material Design Guidelines for iOS
  2. Apple’s documentation on UIApplicationDelegate API
  3. Apple’s documentation on UIApplication API
  4. ChrisRisner’s article on Singletons and AppDelegate
Continue Reading

Using a Git repo as a Storage & Managing skills through susi_skill_cms

In this post, I’ll be talking about SUSI’s skill management and the workflow of creating new skills

The SUSI skills are maintained in a separate github repository susi_skill_data which provides the features of version controlling and the ability to rollback to a previous version implemented in SUSI Server.

The workflow is as explained in the featured image of this blog, SUSI CMS provides the user with a GUI through which user can talk to the SUSI Server and using it’s api calls, it can manipulate the susi skills present/stored on the susi_skill_data repository.

When the user opts to create a new skill, a new createSkill component is loaded with an editor to define rules of the skill. Once the form is submitted, an AJAX POST request is made to the server which actually commits the skill data to the repository and thus it is visible in the CMS from that point on.

Grab the skill details within the editor and put them in a form which is to be sent via the POST request.

let form = new FormData();
form.append('model', 'general');
form.append('group', this.state.groupValue);
form.append('language', this.state.languageValue);
form.append('skill', this.state.expertValue.trim().replace(/\s/g,'_'));
form.append('image', this.state.file);
form.append('content', this.state.code);
form.append('image_name', this.state.imageUrl.replace('images/',''));
form.append('access_token', cookies.get('loggedIn'));


Configure POST request settings object

let settings = {
   'async': true,
   'crossDomain': true,
   'url': urls.API_URL + '/cms/createSkill.json',
   'method': 'POST',
   'processData': false,
   'contentType': false,
   'mimeType': 'multipart/form-data',
   'data': form
};


Make an AJAX request using the settings above to upload the skill to the server and send a notification when the request is successful.

$.ajax(settings)
   .done(function (response) {
   self.setState({
          loading:false
   });
notification.open({
    message: 'Accepted',
    description: 'Your Skill has been uploaded to the server',
    icon: <Icon type='check-circle' style={{ color: '#00C853' }}       />,
});


Parse the received response as JSON and if the accept key in the response is true, we push the new skill data to the history API and set relevant states.

let data = JSON.parse(response);
if(data.accepted===true){
  self.props.history.push({
	pathname: '/' + self.state.groupValue  +
  	'/' + self.state.expertValue.trim().replace(/\s/g,'_') +
  	'/' + self.state.languageValue,
	state: {
  	from_upload: true,
  	expertValue:  self.state.expertValue,
  	groupValue: self.state.groupValue ,
  	languageValue: self.state.languageValue,
}});


If the accepted key of the server response is not true, display a notification.

else{
	self.setState({
  		loading:false
	});
	notification.open({
	  	message: 'Error Processing your Request',
	  	description: String(data.message),
	  	icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
	});
}})


Handle cases when AJAX request fails and send a corresponding notification

.fail(function (jqXHR, textStatus) {
 ...
  notification.open({
    message: 'Error Processing your Request',
    description: String(textStatus),
    icon: <Icon type='close-circle' style={{ color: '#f44336' }} />,
  });
});


I hope after reading this post, the objectives of susi_skill_data are more clear and you understood how CMS handles the creation of skills.

Resources

1.AJAX Jquery – AJAX request using Jquery
2. React State – Read about React states and lifecycle hooks.

Continue Reading

Comparison between SUSI AI with Mycroft AI and Amazon Alexa

Now is the era of Voice User Interface (VUI) devices and they play a very important role as personal assistants. Here we compare the SUSI AI, Mycroft AI and Amazon Alexa based on the number of skills, their availability, easiness to add and edit skills and the provision of the user to modify the skill and add more to it if needed, etc.

Issue: https://github.com/fossasia/labs.fossasia.org/issues/215

The Comparison:

  1. Starting with the number of skills, here Amazon Alexa supports way more number of skills as compared to both Mycroft AI and SUSI AI.
  2. Availability: Mycroft AI and SUSI AI are available everywhere and can set up anywhere regardless of the country whereas Alexa is available in U.S., U.K., Germany,  India but they are aggressively expanding.
  3. Adding and editing skills: Mycroft and SUSI are open source and their skills can be added and edited and viewed by the open source community. Issues can be made to enhance the functionality of the skills whereas Alexa skills are not open source and certification and publishing of the skill is done by the Amazon team. Mycroft and SUSI skills can be customized by the user but this fails with Alexa as users have to create that same skill from scratch if they have to customize them.
  4. Platforms supported: Mycroft, SUSI and Alexa all support Linux. Mycroft lacks support for Windows and Mac but supports Raspberry Pi and Android, Alexa provides support for Windows and Mac and Raspberry Pi. SUSI also provides support for Android and iOS and can be integrated with speakers, vehicles, Pi, etc.
  5. Dedicated devices: As of now SUSI AI lacks such device. Mycroft has Mark 1 and Alexa has Echo. These devices are portable and are good candidates for home automation.
  6. Languages used for skill development: Mycroft mostly uses python. Alexa uses python, NodeJS, C#, etc for development of applications. SUSI uses its own language but language like javascript can be included in it. It’s easier to specify patterns using wildcards and variables in SUSI.

Due to different languages used, Mycroft AI skills can’t be directly used in SUSI AI. We need to convert Mycroft skills to SUSI skills if Mycroft skills are to be used for SUSI.

Some suggestions for making a dedicated device for SUSI:

  1. We can use a Raspberry Pi, USB headphones and a microphone to make a basic platform.
  2. We can install Jasper to enable the voice input on the Pi. Jasper is a open source application that enables us to make voice controlled applications.
  3. We can use SUSI server to interact with the device and the home appliances like lights. SUSI server can process the states of the the appliance (lights in this case) and return it as JSON objects to Raspberry Pi and then it may change the state as per user input.

Make a simple Hello World skill with SUSI:

  1. Visit https://github.com/fossasia/susi_skill_cms/blob/master/docs/Skill_Tutorial.md for a basic introduction to SUSI skills syntax and how does it work.
  2. Go to http://dream.susi.ai .
  3. Enter the skill name, say “hello”.
  4. You will be greeted by a welcome message – “roses are red…..”. Delete it and replace it with the following snippet.
::name <Skill_name> #<— Enter skill name. for example hello

::author <author_name>

::author_url <author_url> #<— You can leave this empty as of now.

::description <description> #<— skill description

::dynamic_content No

::developer_privacy_policy <link> #<— you can leave this as of now.

::image <image_url> #<— You can leave this as of now.

::term_of_use <link>

#Intent. Comments are written with a #

hi|hello|what’s up #<— This is what the user says

Hi|I am good|Hello #<— This is what the skill answers

6. Now go to http://susi.ai/chat for the testing.

7. In the SUSI chat dialog box (present at the bottom of the page) enter dream <test application name> where “test application name” is the name you enter when you first visit http://dream.susi.ai. In this case “dream hello”.

8. You can input “what’s up” in the dialog box and it will give you the desired output which you mentioned in the application.

Conclusion:

SUSI has its own good points but it lacks in some department like the number and type of skills. Like Mycroft we can start making various skills and try to make a basic prototype of a dedicated SUSI personal assistant device.

Resources

  1. Jasper
  2. Skill addition to SUSI
  3. Mycroft hello world skill

 

Continue Reading
Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners
SUSI Desktop

Setting up SUSI Desktop Locally for Development and Using Webview Tag and Adding Event Listeners

SUSI Desktop is a cross platform desktop application based on electron which presently uses chat.susi.ai as a submodule and allows the users to interact with susi right from their desktop.

Any electron app essentially comprises of the following components

    • Main Process (Managing windows and other interactions with the operating system)
    • Renderer Process (Manage the view inside the BrowserWindow)

Steps to setup development environment

      • Clone the repo locally.
$ git clone https://github.com/fossasia/susi_desktop.git
$ cd susi_desktop
      • Install the dependencies listed in package.json file.
$ npm install
      • Start the app using the start script.
$ npm start

Structure of the project

The project was restructured to ensure that the working environment of the Main and Renderer processes are separate which makes the codebase easier to read and debug, this is how the current project is structured.

The root directory of the project contains another directory ‘app’ which contains our electron application. Then we have a package.json which contains the information about the project and the modules required for building the project and then there are other github helper files.

Inside the app directory-

  • Main – Files for managing the main process of the app
  • Renderer – Files for managing the renderer process of the app
  • Resources – Icons for the app and the tray/media files
  • Webview Tag

    Display external web content in an isolated frame and process, this is used to load chat.susi.ai in a BrowserWindow as

    <webview src="https://chat.susi.ai/"></webview>
    

    Adding event listeners to the app

    Various electron APIs were used to give a native feel to the application.

  • Send focus to the window WebContents on focussing the app window.
  • win.on('focus', () => {
    	win.webContents.send('focus');
    });
    
  • Display the window only once the DOM has completely loaded.
  • const page = mainWindow.webContents;
    ...
    page.on('dom-ready', () => {
    	mainWindow.show();
    });
    
  • Display the window on ‘ready-to-show’ event
  • win.once('ready-to-show', () => {
    	win.show();
    });
    

    Resources

    1. A quick article to understand electron’s main and renderer process by Cameron Nokes at Medium link
    2. Official documentation about the webview tag at https://electron.atom.io/docs/api/webview-tag/
    3. Read more about electron processes at https://electronjs.org/docs/glossary#process
    4. SUSI Desktop repository at https://github.com/fossasia/susi_desktop.

    Continue Reading
    Close Menu
    %d bloggers like this: