Handling Planned Actions for the SUSI Smart Speaker 

Planned action is the latest feature added to the SUSI Smart Speaker, The user now has the option to set timed actions such as-  settings alarms, countdown timer etc. So the user now can say “SUSI, Set an alarm in one minute” and after one minute the smart speaker will notify you.

The following flowchart represents the workflow for the working of a planned action:

Planned Action Response

The SUSI Server accepts the planned action query and sends a multi-action response which looks like this: 

Here we can see that we have two actions in the server response. The first action is of the type “answer” and is executed by the SUSI Linux client immediately, the other response has the `plan_date` and `plan_delay` keys which tells the BUSY State of the SUSI Linux client that this is a planned action and is then sent to the scheduler.

Parsing Planned Action Response From The Server 

The SUSI python wrapper is responsible for parsing the response from the server and making it understandable to the SUSI Linux client. In SUSI Python we have classes which represent the different types of actions possible. The SUSI Python takes all the actions sent by the server and parses them into objects of different action types. To enable handling planned actions we add two more attributes to the base action class – `planned_date` and `planned_delay`.

Here we can see, All the action types which can be planned actions call the base class’ constructor to set the value for planned_delay and planned_date attributes. 

The next step is to parse the different action type object and generate the final result

Here if the action object has a non none value for the planned attributes, the action object’s values are added to a planned actions list.

Listening to Planned Actions in the SUSI Linux Client

In the busy state, we see if the payload coming from the IDLE state is a query or a planned response coming from the scheduler. If the payload is a query, the query is sent to the server, otherwise the payload is executed directly

If the payload was a query and the server replies with a planned action response, then 

The server response is sent to the scheduler.

The scheduler then schedules the event and send the payload to the IDLE state with the required delay. To trigger planned actions we implemented an event based observer using RxPy. The listener resides in the idle state of the SUSI State Machine. 

The observer in the IDLE state on receiving an event sends the payload to the busy state where it is processed. This is done by the transition_busy method which uses the allowedStateTransitions method.

Resources

Understanding the SUSI State Machine – https://blog.fossasia.org/implementing-susi-linux-app-as-a-finite-state-machine/

Audio Structure of SUSI Smart Speaker – https://blog.fossasia.org/audio-structure-of-susi-smart-speaker/

Reactive Python Documentation – https://rxpy.readthedocs.io/en/latest/

Tags

SUSI Smart Speaker, SUSI.AI, FOSSASIA, GSoC19

The implementation of event invoice tables is already explained in the blog post Implementation of Event Invoice view using Ember Tables. This blog post is an extension which will showcase the implementation of paid route for event invoice in Open Event Frontend. Event invoices can be explained as the monthly fee given by the organizer to the platform for hosting their event.

We begin by defining a route for displaying the paid event invoice. We chose event-invoice/paid route in router.js

this.route('event-invoice', function() {
   ...
   this.route('paid', { path: '/:invoice_identifier/paid' });
 });

Now, we need to finalize the paid route. The route is divided in three sections – 

• titleToken :  To display title in the tab of the browser.
• model :  To fetch event-invoice with the given identifier.
• afterModel : To check if the status of the fetched event -invoice is paid or due. If it is due, it is redirected to review page otherwise it is redirected to paid page.

import Route from '@ember/routing/route';
 
export default class extends Route {
 
 titleToken(model) {
   return this.l10n.tVar(`Paid Event Invoice - ${model.get('identifier')}`);
 }
 
 model(params) {
   return this.store.findRecord('event-invoice', params.invoice_identifier, 
     {
         include : 'event,user',
         reload  : true
     }
   );
 }
 
 afterModel(model) {
   if (model.get('status') === 'due') {
     this.transitionTo('event-invoice.review', model.get('identifier'));
   } else if (model.get('status') === 'paid') {
     this.transitionTo('event-invoice.paid', model.get('identifier'));
   }
 }
}

Now, we need to design the template for paid route. It includes various elements –

1. Invoice Summary : 

<div class="ten wide column print">      
    {{event-invoice/invoice-summary data=model                             
        event=model.event                               
        eventCurrency=model.event.paymentCurrency}} 
</div>

The invoice summary contains some of the details of the event invoice. It contains the event name, date of issue, date of completion and amount payable of the event invoice. The snippet for invoice summary can be viewed here

      2.  Event Info :

<div class="mobile hidden six wide column">
  {{event-invoice/event-info event=model.event}}
</div>

The event info section of the paid page contains the description of the event to which the invoice is associated. It contains event location, start date and end date of the event. The snippet for event info can be viewed here

      3.  Billing Info :

<div class="ten wide column">
  {{event-invoice/billing-info user=model.user}}
</div>

The billing info section of the event invoice paid page contains the billing info of the user to which the event is associated and that of the admin. The billing info includes name, email, phone, zip code and country of the user and the admin. The snippet for billing info can be viewed here

      4.  Payee Info :

<div class="mobile hidden row">
   {{event-invoice/payee-info data=model payer=model.user}}
</div>

The payee information displays the name and email of the user who pays for the invoice and also the method of the payment along with relevant information. The snippet for payee info can be viewed here

We can download the invoice of the payment made for the event invoice. This is triggered when the Print Invoice button is clicked. 

Code snippet to trigger the download of the invoice –

 <div class="row">
   <div class="column right aligned">
     <button {{action 'downloadEventInvoice' model.event.name model.identifier }} class="ui labeled icon blue {{if isLoadingInvoice 'loading'}} button">
        <i class="print alternate icon"></i>
        {{t 'Print Invoice'}}
     </button>
   </div>
 </div>

The function downloadEventInvoice is called when the button Print Invoice is clicked. The event name and order ID is passed as parameters to the function. When the invoice pdf generation is successful, message Here is your Event Invoice is displayed on the screen whereas if there is an error, the message Unexpected error occurred is displayed.

@action
 async downloadEventInvoice(eventName, orderId) {
   this.set('isLoading', true);
   try {
     const result = this.loader.downloadFile(`/events/invoices/${this.orderId}`);
     const anchor = document.createElement('a');
     anchor.style.display = 'none';
     anchor.href = URL.createObjectURL(new Blob([result], { type: 'application/pdf' }));
     anchor.download = `${eventName} - EventInvoice-${orderId}.pdf`;
     document.body.appendChild(anchor);
     anchor.click();
     this.notify.success(this.l10n.t('Here is your Event Invoice'));
     document.body.removeChild(anchor);
   } catch (e) {
     console.warn(e);
     this.notify.error(this.l10n.t('Unexpected error occurred.'));
   }
   this.set('isLoading', false);
 }

Resources:

• Specifying a routes model in Ember JS
• Actions in Ember JS

Related work and code repo:

• Front-End Repository
• API Server Repository
• Implementation of paid route for event invoices

In the previous versions of the firmware for the smart speaker, we had a separate flask server for serving the configuration page and another flask server for serving the control page. This puts forward inconsistencies in the frontend. To make the frontend and the user experience the same across all platforms, the SUSI.AI Web Client is now integrated into the smart speaker.

Now whenever the device is in Access Point mode (Setup mode), and the user accesses the web client running on the smart speaker, the device configuration page is shown.

If the device is not in access point mode, the normal Web Client is shown with a green bar on top saying “You are currently accessing the local version of your SUSI.AI running on your smart device. Configure now”
Clicking on the Configure now link redirects the user to the control page, where the user can change the settings for their smart speaker and control media playback.

To integrate both the control web page and the configure web page in the web client we needed to combine the flask server for the control and the configure page. This is done by adding all the endpoints of the configure server to the sound server and then renaming the sound server to control server – Merge Servers

Serving the WebClient through the control server

The next step is to add the Web Client static code to the control server and make it available on the root URL of the server.
To do this, first, we have to clone the web client during the installation and add the required symbolic links for the server.

The next step is to add the route for the Web Client in the control flask server.

Here the index.html file is of the Web Client which we linked to the templates folder in the previous step.

Connecting the web client to the locally running SUSI Server

The Web Client by default connects to the SUSI Server at https://api.susi.ai but however the smart speaker has its own server running. To connect to the local server we do the following

– Before building the local version from the source of web client we need to set the process.env.REACT_APP_LOCAL_ENV environment variable to true.

– While building the javascript function check() checks if the above-mentioned environment variable is set to true and if yes, sets the API_URL to http://<host_IP_address>:4000

Resources

Javascript window.location – https://developer.mozilla.org/en-US/docs/Web/API/Window/location

Symlinks Linux – https://www.shellhacks.com/symlink-create-symbolic-link-linux/
SUSI.AI Web Client documentation – https://github.com/fossasia/susi.ai/blob/master/README.md

Tags

SUSI Smart Speaker, SUSI.AI, FOSSASIA, GSoC19