Registering The SUSI Smart Speaker With your SUSI.AI account

When the SUSI Smart Speaker is set up for the first time it needs to be configured. After successful configuration, the smart speaker is registered with the associated account so that the user can see their smart speaker device information from the settings of their susi.ai account. There are two ways to configure  the smart speaker:

  • Through the android app
  • Through the Web Configuration Page

Both these processes are shown in detail here – https://github.com/fossasia/susi_installer/blob/development/docs/configure_guide.md


After the configuration setup is done, the Smart Speaker reboots and connects to your WiFi and registers the device with the given account using the login information provided during the setup.

 

Figure: Device Details are shown in the susi.ai account settings after successful configuration.

Working

The Auth Endpoint

Whenever the speaker is configured via the android app or manually via the web interface it uses various endpoints (access-point-server). For storing login information /auth endpoint is used. The /auth endpoint writes the login details to config.json file in /home/pi/SUSI.AI/config.json

The ss-susi-register service is then enabled i.e. the service will run in the next startup which will register the device online after the device is connected to the WiFi.

@app.route(‘/auth’, methods=[‘GET’])
def login():
    auth = request.args.get(‘auth’)
    email = request.args.get(’email’)
    password = request.args.get(‘password’)
    subprocess.call([‘sudo’, ‘-u’, ‘pi’, susiconfig, ‘set’, “susi.mode=”+auth, “susi.user=”+email, “susi.pass=”+password])
    display_message = {“authentication”:”successful”, “auth”: auth, “email”: email, “password”: password}
    if auth == ‘authenticated’ and email != “”:
        os.system(‘sudo systemctl enable ss-susi-register.service’)
    resp = jsonify(display_message)
    resp.status_code = 200
    return resp # pylint-enable


The SYSTEMD Registration Service

ss-susi-register.service – https://github.com/fossasia/susi_installer/blob/development/raspi/systemd/ss-susi-register.service

This is the service which registers the device on bootup after the configuration phase. The service waits for the network services to run such that the registration script is run only after when it is connected to a network. This service uses register.py to register the device online.

[Unit]
Description=Register the smart speaker online
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
WorkingDirectory=/home/pi/SUSI.AI
ExecStart=/usr/bin/python3 susi_installer/raspi/access_point/register.py

[Install]
WantedBy=multi-user.target


The Registration Script 

Register.py – https://github.com/fossasia/susi_installer/blob/development/raspi/access_point/register.py

This script is responsible for the following tasks

  • Get configuration information from config.json
config = json_config.connect(‘/home/pi/SUSI.AI/config.json’)
user = config[‘login_credentials’][’email’]
password = config[‘login_credentials’][‘password’]
room = config[‘room_name’]
  • Use the login information from config.json to get the authorization token for the respective account.
def get_token(login,password):
    url = ‘http://api.susi.ai/aaa/login.json?type=access-token’
    PARAMS = {
        ‘login’:login,
        ‘password’:password,
    }
    r1 = requests.get(url, params=PARAMS).json()
    return r1[‘access_token’]
  • Use the authorization token and other information from config.json and register the smart speaker online.
def device_register(access_token,room):
    g = geocoder.ip(‘me’)
    mac=’:’.join(re.findall(‘..’, ‘%012x’ % uuid.getnode()))
    url=’https://api.susi.ai/aaa/addNewDevice.json?&name=SmartSpeaker’
    PARAMS = {
        ‘room’:room,
        ‘latitude’:str(g.lat),
        ‘longitude’:str(g.lng),
        ‘macid’:mac,
        ‘access_token’:access_token
    }
    r1 = requests.get(url, params=PARAMS).json()
    return r1

  • If the registration fails put back the smart speaker in the access point(configuration) mode and reset the account information in config.json
try:
        access_token=get_token(user,password)
        out=device_register(access_token,room)
        logger.debug(str(out))
        break
    except:
        if i != 2:
            time.sleep(5)
            logger.warning(“Failed to register the device, retrying.”)
        else:
            logger.warning(“Resetting the device to hotspot mode”)
            config[‘usage_mode’]=”anonymous”
            config[‘login_credentials’][’email’]=””
            config[‘login_credentials’][‘password’]=””
            subprocess.Popen([‘sudo’,’bash’, ‘susi_installer/raspi/access_point/wap.sh’])

  • Disable the systemd service
    The script should run only once i.e. only after the configuration process, so the ss-susi-register.service needs to be disabled.
os.system(‘sudo systemctl disable ss-susi-register.service’)

Resources

Creating a Linux service with systemd – https://medium.com/@benmorel/creating-a-linux-service-with-systemd-611b5c8b91d6

Running shell commands in python – https://cmdlinetips.com/2014/03/how-to-run-a-shell-command-from-python-and-get-the-output/

Tags

SUSI Smart Speaker, SUSI.AI, FOSSASIA, GSoC19

Continue Reading Registering The SUSI Smart Speaker With your SUSI.AI account

Refactoring Order Status in Open Event

This blog post will showcase the introduction of new Initializing status for orders in Open Event Frontend. So, now we have a total of six status. Let’s take a closer look and understand what exactly these order status means:

StatusDescriptionColor Code
InitializingWhen a user selects tickets and clicks on Order Now button on public event page, the user will get 15 minutes to fill up the order form. The status for order till the form is submitted is – initializingYellow
PlacedIf only offline paid tickets are present in order i.e. paymentMode belongs to one of the following – bank, cheque, onsite; then the status of order is placedBlue
PendingIf the order contains online paid tickets, the status for such order is pending. User gets 30 minutes to complete payment for such pending orders.         
If user completes the payment in this timespan of 30 minutes, the status of order is updated to completed.However if user fails to complete payment in 30 minutes, the status of the order is updated to expired.
Orange
CompletedThere are two cases when the status of order is completed –
1. If the ordered tickets are free tickets, the status of order is completed.
2. If the online payment for pending tickets is completed in timespan of 30 minutes, the status is updated to completed. 
Green
ExpiredThere are two cases when status of order is updated to expired.
1. If the user fails to fill up the order form in the 15 minutes allotted to the user, the status changes from initializing to expired.
2. If the user fails to complete the payment for online paid orders in timeframe of 30 minutes allotted, the status is updated from pending to expired. 
Red
CancelledWhen an organizer cancels an order, the order is given status of cancelled.Grey
  Placed Order
Completed Order

Pending Order
Expired Order

So, basically the status of code is set based on the value of paymentMode attribute. 

If the paymentMode is free, the status is set to completed.
If the paymentMode is bank or cheque or onsite, the status is set to placed.
Otherwise, the status is set to pending.

if (paymentMode === 'free') {
    order.set('status', 'completed');
} else if (paymentMode === 'bank' || paymentMode ===  'cheque' || paymentMode === 'onsite') {
    order.set('status', 'placed');
} else {
    order.set('status', 'pending');
}

We render the status of order at many places in the frontend, so we introduced a new helper order-color which returns the color code depending on the status of the order.

import { helper } from '@ember/component/helper';

export function orderColor(params) {
 switch (params[0]) {
   case 'completed':
     return 'green';
   case 'placed':
     return 'blue';
   case 'initializing':
     return 'yellow';
   case 'pending':
     return 'orange';
   case 'expired':
     return 'red';
   default:
     return 'grey';
 }
}

export default helper(orderColor);

This refactor was followed up on server also to accommodate changes:

  • Ensuring that the default status is always initializing. For this, we place a condition in before_post hook to mark the status as initializing.
  • Till now, the email and notification were sent out only for completed orders but as we now use placed status for offline paid orders so we send out email and notification for placed orders too. For this, I updated the condition in after_create_object hook
class OrdersListPost(ResourceList):
    ...
    def before_post(self, args, kwargs, data=None):
        ...
        if not has_access('is_coorganizer', event_id=data['event']):
           data['status'] = 'initializing'

    def after_create_object(self, order, data, view_kwargs):
       ...
       # send e-mail and notifications if the order status is completed
       if order.status == 'completed' or order.status ==  'placed':
           # fetch tickets attachment
           order_identifier = order.identifier
       ...
  • To ensure that orders with status as initializing and pending are updatable only, we introduced a check in before_update_object hook.
class OrderDetail(ResourceDetail):
    ...
    def before_update_object(self, order, data, view_kwargs):
               ...
        elif current_user.id == order.user_id:
           if order.status != 'initializing' and order.status != 'pending':
               raise ForbiddenException({'pointer': ''},  "You cannot update a non-initialized or non-pending order")
  • To allow a new status initializing for the orders, we needed to include it as a valid choice for status in order schema. 
class OrderSchema(SoftDeletionSchema):
     ...
     status = fields.Str(
       validate=validate.OneOf(
           choices=["initializing", "pending", "cancelled",
                    "completed", "placed", "expired"]
     ))

Resources:

Related work and code repo:

Continue Reading Refactoring Order Status in Open Event

Implementation of Role Invites in Open Event Organizer Android App

Open Event Organizer Android App consists of various features which can be used by event organizers to manage their events. Also, they can invite other people for various roles. After acceptance of the role invite, the particular user would have access to features like the event settings and functionalities like scanning of tickets and editing of event details, depending on the access level of the role.

There can be various roles which can be assigned to a user: Organizer, Co-Organizer, Track Organizer, Moderator, Attendee, Registrar.

Here we will go through the process of implementing the feature to invite a person for a particular role for an event using that person’s email address.

The ‘Add Role’ screen has an email field to enter the invitee’s email address and select the desired role for the person. Upon clicking the ‘Send Invite’ button, the person would be sent a mail containing a link to accept the role invite.

The Role class is used for the different types of available roles.

@Data
@Builder
@Type("role")
@AllArgsConstructor
@NoArgsConstructor
@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class Role {

    @Id(LongIdHandler.class)
    public Long id;

    public String name;
    public String titleName;
}

The RoleInvite class:

@Data
@Builder
@Type("role-invite")
@AllArgsConstructor
@NoArgsConstructor
@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class RoleInvite {

    @Id(LongIdHandler.class)
    public Long id;

    @Relationship("event")
    public Event event;

    @Relationship("role")
    public Role role;

    public String email;
    public String createdAt;
    public String status;
    public String roleName;
}

A POST request is required for sending the role invite using the email address of the recipient as well as the role name.

@POST("role-invites")
Observable<RoleInvite> postRoleInvite(@Body RoleInvite roleInvite);

On clicking the ‘Send Invite’ button, the email address would be validated and if it is valid, the invite would be sent.

binding.btnSubmit.setOnClickListener(v -> {
        if (!validateEmail(binding.email.getText().toString())){            
            showError(getString(R.string.email_validation_error));
            return;
        }
        roleId = binding.selectRole.getSelectedItemPosition() + 1;
        roleInviteViewModel.createRoleInvite(roleId);
});

createRoleInvite() method in RoleInviteViewModel:

public void createRoleInvite(long roleId) {

    long eventId = ContextManager.getSelectedEvent().getId();
    Event event = new Event();
    event.setId(eventId);
    roleInvite.setEvent(event);
    role.setId(roleId);
    roleInvite.setRole(role);

    compositeDisposable.add(
        roleRepository
            .sendRoleInvite(roleInvite)
            .doOnSubscribe(disposable -> progress.setValue(true))
            .doFinally(() -> progress.setValue(false))
            .subscribe(sentRoleInvite -> {
                success.setValue("Role Invite Sent");
            }, throwable -> error.setValue(ErrorUtils.getMessage(throwable).toString())));
}

It takes roleId as an argument which is used to set the desired role before sending the POST request.

We can notice the use of sendRoleInvite() method of RoleRepository. Let’s have a look at that:

@Override
public Observable<RoleInvite> sendRoleInvite(RoleInvite roleInvite) {
    if (!repository.isConnected()) {
        return Observable.error(new Throwable(Constants.NO_NETWORK));
    }

    return roleApi
        .postRoleInvite(roleInvite)
        .doOnNext(inviteSent -> Timber.d(String.valueOf(inviteSent)))
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread());
}

Resources:

API Documentation: Roles, Role Invites

Pull Request: feat: Implement system of role invites

Open Event Organizer App: Project repo, Play Store, F-Droid

Continue Reading Implementation of Role Invites in Open Event Organizer Android App

Implementation of Pagination in Open Event Organizer Android App

Pagination (Endless Scrolling or Infinite Scrolling) breaks down a list of content into smaller parts, loaded one at a time. It is important when the quantity of data to be loaded is huge and loading all the data at once can result in timeout.

Here, we will discuss about the implementation of pagination in the list of attendees in the Open Event Organizer App (Eventyay Organizer App).

It is an Android app used by event organizers to create and manage events on the Eventyay platform. Features include event creation, ticket management, attendee list with ticket details, scanning of participants etc.

In the Open Event Organizer App, the loading of attendees would result in timeout when the number of attendees would be large. The solution for fixing this was the implementation of pagination in the Attendees fragment.

First, the API call needs to be modified to include the page size as well as the addition of page number as a Query.

@GET("events/{id}/attendees?include=order,ticket,event&fields[event]=id&fields[ticket]=id&page[size]=20")
Observable<List<Attendee>> getAttendeesPageWise(@Path("id") long id, @Query("page[number]") long pageNumber);

Now, we need to modify the logic of fetching the list of attendees to include the page number. Whenever one page ends, the next page should be fetched automatically and added to the list.

The page number needs to be passed as an argument in the loadAttendeesPageWise() method in AttendeesViewModel.

public void loadAttendeesPageWise(long pageNumber, boolean forceReload) {

    showScanButtonLiveData.setValue(false);

    compositeDisposable.add(
        getAttendeeSourcePageWise(pageNumber, forceReload)
            .doOnSubscribe(disposable -> progress.setValue(true))
            .doFinally(() -> progress.setValue(false))
            .toSortedList()
            .subscribe(attendees -> {
                attendeeList.addAll(attendees);
                attendeesLiveData.setValue(attendees);
                showScanButtonLiveData.setValue(!attendeeList.isEmpty());
            }, throwable -> error.setValue(ErrorUtils.getMessage(throwable).toString())));
}

Also in the getAttendeeSourcePageWise() method:

private Observable<Attendee> getAttendeeSourcePageWise(long pageNumber, boolean forceReload) {
    if (!forceReload && !attendeeList.isEmpty())
        return Observable.fromIterable(attendeeList);
    else
        return attendeeRepository.getAttendeesPageWise(eventId, pageNumber, forceReload);
}

Now, in the AttendeesFragment, a check is needed to increase the current page number and load attendees for the next page when the user reaches the end of the list. 

if (!recyclerView.canScrollVertically(1)) {

    if (recyclerView.getAdapter().getItemCount() > currentPage * ITEMS_PER_PAGE) {
        currentPage++;
    } else {
        currentPage++;                       
        attendeesViewModel.loadAttendeesPageWise(currentPage, true);
    }
}

When a new page is fetched, we need to update the existing list and add the elements from the new page.

@Override
public void showResults(List<Attendee> attendees) {
    attendeeList.addAll(attendees);
    fastItemAdapter.setNewList(attendeeList);
    binding.setVariable(BR.attendees, attendeeList);
    binding.executePendingBindings();
}

Now, list of attendees would be fetched pagewise, thus improving the performance and preventing timeouts.

Resources:

Further reading:

Open Event Organizer App: Project repo, Play Store, F-Droid

Continue Reading Implementation of Pagination in Open Event Organizer Android App

Deleting a user’s own account in Open Event

This blog post will showcase an option using which a user can delete his/her account in Open Event Frontend. In Open Event we allow a user who is not associated with any event and/or orders to delete his/her own account. User can create a new account with same email later if they want. 

It is a 2-step process just to ensure that user doesn’t deletes the account accidentally.

The user needs to get to the Account section where he/she is required to select Danger Zone tab. If user is not associated with any event and/or order, he/she will get an option to delete his/her account along with the following text :

All user data will be deleted. Your user data will be entirely erased and any data that will stay in the system for accounting purposes will be anonymized and there will be no link to any of your personal information. Once you delete this account, you will no longer have access to the system.
Option to delete account in Open Event Frontend

If the user is associated with any event and/or order, the option to delete the account is disabled along with the following text :

Your account currently cannot be deleted as active events and/or orders are associated with it. Before you can delete your account you must transfer the ownership of your event(s) to another organizer or cancel your event(s). If you have tickets orders stored in the system, please cancel your orders first too.
Disabled option to delete account in Open Event Frontend

For above toggle we need to check if a user is deletable or not. For that we must check if a user is associated with any event and/or order. The code snippet which checks this is given below :

     isUserDeletable: computed('data.events', 
       'data.orders', function() {
         if (this.get('data.events').length || this.get('data.orders').length) {
             return false;
         }
         return true;
     })

When a user clicks on the option to delete his/her account, a modal pops up asking the user to confirm his/her email. Once user fills in correct email ID the Proceed button becomes active.

Modal to confirm email ID

The code snippet which triggers the action to open the modal deleteUserModal is given below:

<button {{action 'openDeleteUserModal' data.user.id data.user.email}} class='ui red button'>     
    {{t 'Delete Your Account'}} 
</button>
   openDeleteUserModal(id, email) {     
     this.setProperties({
       'isUserDeleteModalOpen' : true,
       'confirmEmail'          : '',
       'userEmail'             : email,
       'userId'                : id
     });
   }

The code snippet which deals with the email confirmation:

isEmailDifferent : computed('confirmEmail', function() {
   return this.userEmail ? this.confirmEmail !== this.userEmail : true;
 })

When user confirms his/her email and hits Proceed button, a new modal appears which asks the user to confirm his/her action to delete account.

Final confirmation to delete account

The code snippet which triggers the action to open the modal confirmDeleteUserModal is given below: 

<button {{action openConfirmDeleteUserModal}} class="ui red button {{if isEmailDifferent 'disabled'}}">
   {{t 'Proceed'}}
</button>
   openConfirmDeleteUserModal() {
     this.setProperties({
       'isUserDeleteModalOpen'        : false,
       'confirmEmail'                 : '',
       'isConfirmUserDeleteModalOpen' : true,
       'checked'                      : false
     });
   }

When user clicks the Delete button, it triggers deleteUser function, which finally deletes the user’s account and redirect him/her to index page. 

The code snippet for deleteUser function:

  deleteUser(user) {
     this.set('isLoading', true);
     user.destroyRecord()
       .then(() => {
         this.authManager.logout();
         this.routing.transitionTo('index');
         this.notify.success(this.l10n.t('Your account has been deleted successfully.'));
       })
       .catch(() => {
         this.notify.error(this.l10n.t('An unexpected error has occurred.'));
       })
       .finally(() => {
         this.setProperties({
           'isLoading'                    : false,
           'isConfirmUserDeleteModalOpen' : false,
           'checked'                      : false
         });
       });
   }

To ensure that a user cannot delete his/her account via API call, if he/she is associated with event and/or orders, there is a check on server. 

The code snippet for this check:

if data.get('deleted_at') != user.deleted_at:
    if has_access('is_user_itself', user_id=user.id) or  has_access('is_admin'):
        if data.get('deleted_at'):
             if len(user.events) != 0:
                 raise ForbiddenException({'source': ''},  "Users associated with events cannot  be deleted")
             elif len(user.orders) != 0:
                 raise ForbiddenException({'source': ''}, "Users associated with orders cannot be deleted")
             else:
                 modify_email_for_user_to_be_deleted(user)
        else:
             modify_email_for_user_to_be_restored(user)
             data['email'] = user.email
        user.deleted_at = data.get('deleted_at')
else:
     raise ForbiddenException({'source': ''}, "You are not authorized to update this information.")

The helpers modify_email_for_user_to_be_deleted and modify_email_for_user_to_be_restored used in the above code snippet are responsible to update the email of user before deleting him/her or before restoring him/her respectively.

The helper modify_email_for_user_to_be_deleted updates the email ID of user which is to be deleted. It adds ‘.deleted’ substring to email of a user to be deleted.

The helper modify_email_for_user_to_be_restored updates the email ID of user to be restored. It removes ‘.deleted’ substring from a user to be restored. If the email ID obtained after removing ‘.deleted’ from the email ID, we get an email ID which already exists in the system then an error is raised – This email is already registered! Manually edit and then try restoring

We understand that in today’s era, everyone is a bit sceptical about their data and so we now provide freedom to our user to delete their account whenever they want. 

Resources:

Related work and code repo:

Continue Reading Deleting a user’s own account in Open Event

Implementing Complex Custom Forms in Open Event

Several modules of Open Event Frontend involve the use of custom forms, which currently are not truly custom in the sense that they are really restricted, and rigid. Only text fields are available, along with hardcoded dropdowns. Further, the user is only able to select from among the hardcoded fields and toggle them on/off for his/her event.

Any component which extends the form mixin can make specify the validations required using the getValidationRules hook.

Current custom forms are really restricted, and rigid. Only text fields are available, along with hardcoded dropdowns. Further, the user is only able to select from among the hardcoded fields and toggle them on/off for his/her event.

We already have the framework to associate simple custom fields with individual events or orders, and an API to create them. The custom forms schema needs to be now expanded to allow more complex fields. Taking Google forms and our use case as an inspiration, the user should be able to create the following fields:

  • Simple text *
  • Paragraph *
  • Radio Buttons (single choice)
  • Checkboxes
  • Dropdown
  • File upload *
  • Time
  • Date
  • Date & Time

* Already implemented

The schema needs expansion to accommodate options for fields like dropdowns, checkboxes and radio buttons. Also, to store custom labels to the fields, which the user assigns. Currently, they are hardcoded by comparing the name of the field with if-else.


Thus we propose the following schema related changes to accomodate the complex custom forms.

Add a separate model called customFormOptions to store various options of radio buttons, checkboxes, and dropdowns.

They will have the following fields:

ColumnDescription
IDdefault unique ID
valuevalue of the custom form field options like ‘XS, XL’
custom_form_idforeign key – the id of the custom form field this option belongs to

CustomForm model will have a hasMany relationship with customFormOptions.

For text fields, and other fields which don’t require options within them can have the relationship as null.

The changes to customForm Model itself:

ColumnDescription
descriptionAn optional simple string column to store the custom messages/info the user may give to the custom form field like T-shirt Size Chart link etc.
isComplexBoolean field to indicate if a particular field is complex

The changes to event, speaker, session Models:

ColumnDescription
customFormValuesA JSON type column which stores all the complex custom form values(currently all the fields offered are hardcoded in the schema)

This expansion of schemas will allow the clients to create new, custom fields as per the requirement of the system. Future work may involve creating an API for validations of these fields.

Resources 

Tags : 

Continue Reading Implementing Complex Custom Forms in Open Event

Introducing Custom Validations for Start-End DateTime scenarios on Open Event Frontend

Several modules of Open Event Frontend involve start and end date-times. While for simple type fields like text, dropdowns or radio buttons, default semantic UI validations are available, which are used inside the app via the form mixin. 

Any component which extends the form mixin can make specify the validations required using the getValidationRules hook.

For instance, this set of rules will enforce validations on the field called ticket_price which will prohibit it from being left empty, or something other than a real number.

getValidationRules() {
 ticketPrice: {
          identifier : 'ticket_price',
          rules      : [
            {
              type   : 'empty',
              prompt : this.l10n.t('Please give your ticket a price')
            },
            {
              type   : 'number',
              prompt : this.l10n.t('Please give a proper price for you ticket')
            },
            {
              type   : 'decimal[0..]',
              prompt : this.l10n.t('Ticket price should be greater than 0')
            }
          ]
        },
}

The validations provided by semantic UI only extend to single fields, and are independent of each other. However, in our use case we have four fields:

  1. Start date
  2. Start time
  3. End date
  4. End time

The general requirement is that the DateTime object formed by joining the start date and the start time should be before the DateTime object obtained by joining the end date and time. Also, these four distributed fields exist only on the frontend, on the server they are actually just two fields startsAt and endsAt each carrying UTC time values of DateTime objects. To split them into date and time a new computed function is created which is invoked in the model definitions of various resources. Consider the two following complex fields defined in the model according to the schema on the server.

 startsAt               : attr('moment'),
 endsAt                 : attr('moment')

In order to split them into two that helper is used as follows:

startsAtDate : computedDateTimeSplit.bind(this)('startsAt', 'date', 'endsAt'),
startsAtTime : computedDateTimeSplit.bind(this)('startsAt', 'time', 'endsAt'),
endsAtDate   : computedDateTimeSplit.bind(this)('endsAt', 'date')
  endsAtTime   : computedDateTimeSplit.bind(this)('endsAt', 'time'),

These values can then be used inside individual fields. To enhance the user experience jquery Calendar module is used to allow the user to enter the date and time values using a calendar and time picker as shown below.

The computedDateTimeSplit helper, takes in the property whose part it is splitting, along with the specification of the part it will split. It also takes an optional endProperty argument, which is passed if it is being called for a start property. This function returns a pair of getters and setters,the get function returns the part of datetime object requested like, date or time where as the setter sets these values each time this function is called.

export const computedDateTimeSplit = function(property, segmentFormat, endProperty) {
  return computed(property, {
    get() {
      return moment(this.get(property)).format(getFormat(segmentFormat));
    },
    set(key, value) {
      const newDate = moment(value, getFormat(segmentFormat));
      let oldDate = newDate;

      if (segmentFormat === 'time') {
        oldDate.hour(newDate.hour());
        oldDate.minute(newDate.minute());
      } else if (segmentFormat === 'date') {
        oldDate.date(newDate.date());
        oldDate.month(newDate.month());
        oldDate.year(newDate.year());
      } else {
        oldDate = newDate;
      }
      this.set(property, oldDate);
      }
      return value;
    }
  });
};
 

With this complex set up it is not possible to use semantic UI validations, hence we extend the default semantic UI validations, by introducing  custom rules for the form.

A rule called checkDates is introduced inside the getValidationRules hook.

  getValidationRules() {
    window.$.fn.form.settings.rules.checkDates = () => {
      let startDatetime = moment(this.get('data.speakersCall.startsAt'));
      let endDatetime = moment(this.get('data.speakersCall.endsAt'));
      return (endDatetime.diff(startDatetime, 'minutes') > 0);
    };
}

This hook manually compares the end and start datetime objects, and if they are in violation, it returns false, which triggers a semantic UI styled validation.

To enforce the validation, the start and end datetime fields are embedded with 

startDate: {
            identifier : 'start_date',
            rules      : [
              {
                type   : 'empty',
                prompt : this.l10n.t('Please tell us when your event starts')
              },
              {
                type   : 'checkDates',
                prompt : this.l10n.t('Start date & time should be after End date and time ')
              }
            ]
          },
          endDate: {
            identifier : 'end_date',
            rules      : [
              {
                type   : 'empty',
                prompt : this.l10n.t('Please tell us when your event ends')
              },
              {
                type   : 'checkDates',
                prompt : this.l10n.t('Start date & time should be after End date and time')
              }
            ]
          },
          startTime: {
            identifier : 'start_time',
            depends    : 'start_date',
            rules      : [
              {
                type   : 'empty',
                prompt : this.l10n.t('Please give a start time')
              },
              {
                type   : 'checkDates',
                prompt : '.'
              }
            ]
          },
          endTime: {
            identifier : 'end_time',
            rules      : [
              {
                type   : 'empty',
                prompt : this.l10n.t('Please give an end time')
              },
              {
                type   : 'checkDates',
                prompt : '.'
              }
            ]

However one big problem with this approach is the clearing of validation messages, when one field is in violation, technically all four are in violation as you can either adjust the start time or end time to make the datetimes valid. However, semantic UI validations which operate upon the blur listening property operate with respect to a single field at a time, and any changes to a particular field does not retrigger validations on others. So, for instance an end date was added which was occurring on the same date as a start date, and times are the same too, the form will throw validation on all four fields. However, if we just change the time of one of the dates, the dates will be valid again, but the validation error dialogs of only that field will disappear which was actually modified. This default behavior of semantic UI problematic for us. 

onChange() {
      this.onValid(() => {});
    }

Thus an onChange action is attached with the focus-out event of the date and time inputs

{{input type='text' value=value placeholder=placeholder name=name focus-out=(action 'onChange')}}

This action triggers the validations on the entire form, which will clear at any redundant validation error prompts still remaining in the form. As a future improvement, instead of onChange triggering validations on the entire form, only the datetimes can be revalidated, though this will require extension of the form mixin to support more flexible methods to validate forms using latest semantic UI APIs.

Resources 

Continue Reading Introducing Custom Validations for Start-End DateTime scenarios on Open Event Frontend

Migrating to Ember Tables on Open Event Frontend – Part 3: Search module

This blog article will continue the discussions about setting up ember tables on open event frontend. The implementation and design of the search module of the ember tables will be discussed.

Open event server supports searching, using filter queries offered by flask-rest-json-api 

Leveraging the refreshModel property of the queryParams, we can bind the value of a potential search query of a user to a queryParam. As the user types the query, the queryParam will change, which in turn will refresh the model, and send a new request to the server, giving the impression of a basic search.

An icon input field with a search icon is placed on the top right corner of the table wrapped in its own separate component, search-box. The component is rendered within the default table component, which acts as a base for all other tables in the app.

Here, searchQuery is binded to the param search defined in the controller mixin for ember tables.

<div class="ui small icon input">
  {{input type="text" value=searchQuery placeholder="Search ..." }}
  <i class="search icon"></i>
</div>

Now in order to generate the query string which will result in a search involves complex array filter manipulations, hence the logic was abstracted into the route mixin to avoid it’s repetition across various routes.

Flast-rest-json api’s filters follow the following format for a  basic like styled sql filter:

GET /events?filter=[
  {
    "name": "event",
    "op": "ilike",
    "val": searchQuery  }
] HTTP/1.1
Accept: application/vnd.api+json

There are several other query combinations available for comparison, dates and other data-types or for relationships themselves. They needed to be appended to the filter portion of a query and then removed.

When the search param is null, the query string has the following form:

let queryString = {
      include        : 'tickets,user',
      filter         : filterOptions,
      'page[size]'   : params.per_page || 10,
      'page[number]' : params.page || 1
    };

filterOptions are other filters already present, for instance for listing orders on the basis of a dynamic status parameter,  filterOptions can be defined as follows.

 filterOptions = [
        {
          name : 'status',
          op   : 'eq',
          val  : params.orders_status
        }
      ];

Since filterOptions are an array, new filters are either added or appended to the array.

 applySearchFilters(options, params, searchField) {
    searchField = kebabCase(searchField);
    if (params.search) {
      options.pushObject({
        name : searchField,
        op   : 'ilike',
        val  : `%${params.search}%`
      });
    } else {
      options.removeObject({
        name : searchField,
        op   : 'ilike',
        val  : `%${params.search}%`
      });
    }
    return options;
  }

Thus a method called applySearchFilters is defined in the ember-table-router mixin.

It takes three arguments:

  • options
  • params
  • searchField

Options is the filter array, discussed above, params contains the value of the current search param, and the searchField is the target of the search which is defined in the route’s model hook itself. The open-event-api server, expects the name of fields in kebab case, where as the front-end uses camelCase, hence the searchField is explicitly converted into a kebab-case string before manipulating the filter array. If params are present, it appends the filter object to the queryString, else this function will remove the very filter it added to the query string.

Finally in the route’s model hook itself, queryString is passed to this function, and the filters get applied to it.

    queryString = this.applySortFilters(queryString, params);

Semantic UI has a dedicated class for search input fields, which has been used in the template. As a future goal, a slight delay can be introduced in the update cycle of the search queryParam, currently a network request is sent on each keystroke of the user’s input which can be taxing for the server. Introducing a slight delay or alternatively cancelling a request no longer needed will improve latency and response time of the API server, until elastic search is introduced.

Resources 

Continue Reading Migrating to Ember Tables on Open Event Frontend – Part 3: Search module

Migrating to Ember Tables on Open Event Frontend – Part 2: Pagination

This blog article will continue the discussions about setting up ember tables on open event frontend. The implementation and design of the pagination module of the ember tables will be discussed.

Open event server uses JSON:API spec. Whenever the server returns a query, the JSON contains a meta field, that field contains the total number of records on the API server, which satisfy the query. This number, of course is different from the actual number of records returned which depend on the page size, and page number. But this serves as a good starting point for implementing the pagination module. For the open event server, the property inside the meta tag which contains the total count is called count. Using this property we define a computed property called totalContentLength.

import Component from '@ember/component';
import { computed, action, get } from '@ember/object';

export default class extends Component {

  metaItemsCountProperty = 'count';

 @computed('metaData')
  get totalContentLength() {
    return get(this.metaData, this.metaItemsCountProperty);
  }
}

Ember has great support for queryParams, and if in their declaration inside the route, the property refreshModel is set to true, then the model refreshes and reloads the data whenever the query params update. Thus, the current page and page size are maintained in the form of query params. 

For computing the total number of pages (pageCount),  that the content will be split in, we have all three required variables, totalContentLength (computed above), currentPage and pageSize (both available as queryParams).

Once we have pagesCount, it is easy to decide if moving forward or backwards is possible and can themselves be stored as computed properties. Similarly, in order to move forwards or backwards by a page, or to last or first pages, the queryParam for currentPage can be altered.

import Component from '@ember/component';
import { computed, action, get } from '@ember/object';

export default class extends Component {

  metaItemsCountProperty = 'count';

 @computed('metaData')
  get totalContentLength() {
    return get(this.metaData, this.metaItemsCountProperty);
  }

@computed('currentPage', 'pageSize', 'totalContentLength')
  get pageCount() {
    let totalPages = 1;
    if (parseInt(this.pageSize) !== 0 && this.pageSize < this.totalContentLength) {
      totalPages = parseInt(this.totalContentLength / this.pageSize);
      if (this.totalContentLength % this.pageSize) {
        totalPages += 1;
      }
    }
    return totalPages;
  }
 @computed('currentPage')
  get moveToPreviousPageDisabled() {
    return this.currentPage <= 1;

  }
  @computed('currentPage', 'pageCount')
  get moveToNextPageDisabled() {
    return this.currentPage >= this.pageCount;
  }

  @action
  moveToNextPage() {
    if (!this.moveToNextPageDisabled) {
      this.incrementProperty('currentPage');
    }
  }

  @action
  moveToPreviousPage() {
    if (!this.moveToPreviousPageDisabled) {
      this.decrementProperty('currentPage');
    }
  }

 @action
  moveToLastPage() {
    if (!this.moveToNextPageDisabled) {
      this.set('currentPage', this.pageCount);
    }
  }

  @action
  moveToFirstPage() {
    if (!this.moveToPreviousPageDisabled) {
      this.set('currentPage', 1);
    }
  }
}

We also need to display, what is the current range of entries being shown. For eg, Showing 20-30 of 100 entries. This can again be computed by using current page, page size and  total number of entries.

Using all of these computed properties, we can render the template for a pagination menu, which the user can use in order to navigate through the data. 

<div class="ui small pagination menu">
  <a role="button" class="item {{if moveToPreviousPageDisabled 'disabled'}}" {{action 'moveToFirstPage'}}>
    <i class="angle double left icon"></i>
  </a>
  <a role="button" class="item {{if moveToPreviousPageDisabled 'disabled'}}" {{action 'moveToPreviousPage'}}>
    <i class="angle left icon"></i>
  </a>
  <a role="button" class="item {{if moveToNextPageDisabled 'disabled'}}" {{action 'moveToNextPage'}}>
    <i class="angle right icon"></i>
  </a>
  <a role="button" class="item {{if moveToNextPageDisabled 'disabled'}}" {{action 'moveToLastPage'}}>
    <i class="angle double right icon"></i>
  </a>
</div>
<div class="ui right floated less padding basic segment">
  {{t 'Showing'}} {{currentRange}} {{t 'of'}}  {{totalContentLength}} {{t 'entries'}}
</div> 

Semantic UI has a dedicated class for pagination menu, which has been used in this template. The buttons are disabled as per the computed properties to enhance the user experience.

Resources 

Continue Reading Migrating to Ember Tables on Open Event Frontend – Part 2: Pagination

Migrating to Ember Tables on Open Event Frontend – Part 1: The Set-Up

This blog article will illustrate how ember tables were set up, reopened as a component, for customization and how the pagination module was implemented.

Ember source 3.11 which Open event frontend uses is not compatible with the last release of ember tables, hence the master branch of ember tables which does support the latest ember source was chosen.  To install a dependency from a Github repository link instead of a yarn package, we can use

yarn add addepar/ember-table#0aa5637

Ember tables offer no inbuilt theme, and use the default HTML styles. They do have support for styling, but only via CSS selectors in CSS files. In our use case, we needed the styling to be those of semantic UI tables. However, for that the classes had to be added inside the table element, and ember tables by default, don’t allow addition of classes, as the table was under other layers. Even if the classNames property of component had  been specified, it would just append the specified class names to the wrapper of the table element, not the actual table itself. Ember’s reopen feature was used to solve this problem. The reopen method allows ‘reopening’ of the component in the sense that it’s existing properties can be overwritten, or new properties can be added.

It is a convention to store the reopened component files in a folder separate from external folder.

The definition of the table was inside a component.js file inside the source of ember tables. Hence a new file located at app/extensions/ember-table/component.js was created. 

Then the component.js was reopened to change the source of the template file which is used for ember tables.

import component from 'ember-table/components/ember-table/component';
import layout from './template';

component.reopen({
  layout
});

This reopening modifies the ember table component, such that it now searches for the layout file in the new directory at ember-table/components/ember-table/layout.hbs

The file layout.hbs now contains 

<div class="resize-container">
  <table class="ui unstackable table">
    {{yield (hash
              api=api
              head=(component "ember-thead" api=api)
              body=(component "ember-tbody" api=api)
              foot=(component "ember-tfoot" api=api)
            )}}
  </table>
</div> 

It is the exact same file which was present in the source of ember tables, with the only modification being the addition of class ui unstackable table to the table element. This class makes the table support semantic UI styling.

Ember tables follow a structure similar to ember model tables, wherein the columns are defined inside the controller of the route which will render the table. However, it is not possible to pass the actions defined in the controller to the final cell components for columns without passing them throughout until the very last layer. I.e. actions of controller are not passed to the custom cell components, automatically. Specifying them explicitly results in loss of generalisation, and a significant portion of code will be repeated.  Also, sometimes, we might need to pass more than one valuePaths if we don’t need to pass the entire entity, if a column cell needs only some of the properties. Custom options were a requirement as well. Hence, to generalise these properties, the expanded form of  ember table from ember table docs was slightly modified to :

{{#ember-table as |t|}}
      {{#t.head sortFunction=null columns=columns enableReorder=true as |h|}}
        {{#h.row as |r|}}
          {{#r.cell as |column|}}
            {{#if column.headerComponent}}
              {{#component
                column.headerComponent
              }}
                {{column.name}}
              {{/component}}
            {{else}}
              {{column.name}}
            {{/if}}
             {{/r.cell}}
            {{/h.row}}
         {{/t.head}}
      {{#t.body rows=rows as |b|}}
        {{#b.row as |r|}}
          {{#r.cell as |cell column row|}}
            {{#if column.cellComponent}}
              {{#component column.cellComponent
                           record=(get row column.valuePath)
                           extraRecords=(get-properties row column.extraValuePaths)
                           props=(hash options=column.options actions=column.actions)
              }}
                {{cell}}
              {{/component}}
            {{else}}
              {{cell}}
            {{/if}}
             {{/r.cell}}
            {{/b.row}}
         {{/t.body}}
    {{/ember-table}}

Two additions were made in terms of properties passed to a cell in the table.

A props object, which is a hash of the options and actions properties which will be defined inside the columns definition as column properties.

Also the extraValuePaths are passed as extraRecords using getProperties  helper.

The getProperties helper takes in an object and a list of properties, it then returns those properties of an object, as  hash.

export function getProperties(params = []) {
  if (params.length < 2 || !params[1]) {
    return {};
  }
  let inputParams = params.slice();
  const row = inputParams.shift();
  return emberGetProperties(row, flatten(inputParams));
}

An important thing to note is that even though the actions defined in the controller can be passed in the actions property, they still require a correct context of this. That is achieved using binding the current context of this to them before passing.

actions: {
          moveToPublic  : this.moveToPublic.bind(this),
          moveToDetails : this.moveToDetails.bind(this),
          editEvent     : this.editEvent.bind(this)
        }

The example above shows how the correct context of this is binded with actions.

The next blog will cover more details about the implementation ember tables, and focus on various utilities present inside tables like pagination, searching and sorting.

Resources 

Continue Reading Migrating to Ember Tables on Open Event Frontend – Part 1: The Set-Up