Implement Order Confirmation Feature in Eventyay

This post elaborates on the details of an endpoint which can be used to explicatively used to resend order confirmations. In the current implementation of the open event project, if the order has been confirmed, the ticket holders and buyers get an email each regarding their order confirmation. But in case that email has been accidentally deleted by any of the attendees, the event organizer / owner should have the power to resend the confirmations.

The first step to the implementation was to create the appropriate endpoint for the server to be pinged. I utilized the existing blueprint being used for serving tickets on eventyay frontend project and created a new endpoint on the route : orders/resend-email [POST]

# app/api/auth.py
@ticket_blueprint.route('/orders/resend-email', methods=['POST'])
@limiter.limit(
  '5/minute', key_func=lambda: request.json['data']['user'], error_message='Limit for this action exceeded'
)
@limiter.limit(
  '60/minute', key_func=get_remote_address, error_message='Limit for this action exceeded'
)
def resend_emails():
  """
  Sends confirmation email for pending and completed orders on organizer request
  :param order_identifier:
  :return: JSON response if the email was succesfully sent
  """
  order_identifier = request.json['data']['order']
  order = safe_query(db, Order, 'identifier', order_identifier, 'identifier')
  if (has_access('is_coorganizer', event_id=order.event_id)):
      if order.status == 'completed' or order.status == 'placed':
          # fetch tickets attachment
          order_identifier = order.identifier
          key = UPLOAD_PATHS['pdf']['tickets_all'].format(identifier=order_identifier)
          ticket_path = 'generated/tickets/{}/{}/'.format(key, generate_hash(key)) + order_identifier + '.pdf'
          key = UPLOAD_PATHS['pdf']['order'].format(identifier=order_identifier)
          invoice_path = 'generated/invoices/{}/{}/'.format(key, generate_hash(key)) + order_identifier + '.pdf'

          # send email.
          send_email_to_attendees(order=order, purchaser_id=current_user.id, attachments=[ticket_path, invoice_path])
          return jsonify(status=True, message="Verification emails for order : {} has been sent succesfully".
                          format(order_identifier))
      else:
          return UnprocessableEntityError({'source': 'data/order'},
                                          "Only placed and completed orders have confirmation").respond()
  else:
      return ForbiddenError({'source': ''}, "Co-Organizer Access Required").respond()

I utilized exiting send_email_to_attendees for the email purpose but for security reasons, the endpoint was limited to make sure that an organizer can request only 5 order confrimations to be resent each minute (implemented using flask limiter).

This was all for server implementation, to implement this on the front end, I just created a new action named as resendConfirmation implemented as given.

// app/controllers/events/view/tickets/orders/list.js
async resendConfirmation(order) {
    let payload = {};
    try {
      payload = {
        'data': {
          'order' : order.identifier,
          'user'  : this.authManager.currentUser.email
        }
      };
      await this.loader.post('orders/resend-email', payload);
      this.notify.success(this.l10n.t('Email confirmation has been sent to attendees successfully'));
    } catch (error) {
      if (error.status === 429) {
        this.notify.error(this.l10n.t('Only 5 resend actions are allowed in a minute'));
      }
      if (error.errors[0].detail) {
        this.notify.error(this.l10n.t(error.errors[0].detail));
      }
    }
  }

Using a simple post request, this was implemented on the frontend for sending the confirmation, but the additional work to be done was to handle the new error (429 status). The server throws the error but loader service hasn’t been configured yet to handle this error appropriately.

// app/services/loader.js
  if (!response.ok) {
    const defaultMessage = httpStatus[response.status];
    if (parsedResponse) {
      throw parsedResponse;
    }
    if (response.status === 429) {
      throw { status: 429, message: ‘TOO MANY REQUESTS’ };
    }
    throw new Error(
      getErrorMessage(
        response.statusText,
        defaultMessage
          ? `${response.status} – ${defaultMessage}`
          : `Could not make ${fetchOptions.type} request to ${fetchOptions.url}`
      )
    );
  }

The loader service has been modified in the following manner to accommodate the new error been thrown so that a more user friendly error could be shown on the controller level.

This was the whole mechanism which has been implemented for this particular problem. 

Resources

Related Work and Code Repository

Continue ReadingImplement Order Confirmation Feature in Eventyay

Snackbar for Error Handling in Twitter Followers Insight loklak App

In this blog post am going to explain how the Twitter Followers Insight app handles error which occurs when when no query is passed to the “Search” function i.e., when the search is proceeded with an empty query.

How to handle the Exception

In such cases, I have used SNACKBAR / TOAST. Snackbar / Toast is used to popup an error notification on the screen when an exception occurs.

Script for Snackbar / Toast:

In the below script, initially the error is set to null i.e., no error. The “showError” function is which is being called when there occurs a situation of no query or an empty query. The function below helps to show an error popup which only shows till a time limit.

    $scope.error = null;

    $scope.showError = function() {
        $(".snackbar").addClass("show");
        setTimeout(function(){ $(".snackbar").removeClass("show") }, 3000);
    }

 

In this script, it checks whether the query passed is undefined or empty. If yes, then the error message which was null earlier is changed and that error is showed up on the screen to the end user. Snackbars animate upwards from the edge of the screen.

        if ($scope.query === '' || $scope.query === undefined) {
            $scope.spinner = false;
            $scope.error = "Please enter a valid Username";
            $scope.showError();
            return;
        }

 

Resources:

Continue ReadingSnackbar for Error Handling in Twitter Followers Insight loklak App

Handling Errors While Parsing the yaml File in Yaydoc

Yaydoc, our automatic documentation generator uses a yaml file to read a user’s configuration. The internal configuration parser basically converts the yaml file to a python dictionary. Then, it serializes the values of that dictionary using a custom serialization format. From there it associates those values with environment variables which are then passed to bash scripts for various tasks such as deployment, generation, etc.. Some of those environment variables are again passed to another python layer which interacts with sphinx where they are deserialized before use. This whole system works pretty well for our use cases.

Now let’s assume a user adds a yaml file where they have a malformed section in the file. For example, to specify a theme, one needs to add the following to the yaml file.

build:
  theme:
    name: sphinx_fossasia_theme

But our user has the following in their yaml file.

build:
  theme: sphinx_fossasia_theme

Now this will raise an error as we expect a dictionary as a value for the key ‘theme’ but we got a string. Now how do we handle such cases without ignoring the entire file as that would be too much of a penalty for such a small mistake? One approach would have been to wrap each call to connect with a bunch of try-catch but that would render the code unreadable as the initial motivation for implementing the connect method was to abstract the internal implementation so that other contributors who may not be well versed with python can also easily add config options without needing to learn a bunch of python constructs.

So, what we did was that, while merging the dictionary containing default options and the dictionary containing the user preferences, we check whether the default has the same data type as that of the incoming value. If they are, It’s deemed safe to merge. There are certain relaxations though, like if the current type is a list, then the incoming value can be of any time as that can always be converted to a list of a single element. This is required to support the following syntax.

key:
  - value
key: value

The above two blocks are equivalent due to the above-mentioned approach although the type is different.

Now, after this pre-validation step is over we can ensure that the if the assumed type for a key is let’s say a dictionary, then it would be a dictionary. Hence no type errors would be raised like trying to access a dict method for another object, say a string which happened with the earlier implementation. After this, an extra parameter was added to the connect method to which we can now pass a validation function which if returns false, those values would be ignored. Usage of this feature has been implemented to a small level where we validate the links to subprojects and if they look like a valid github repo only then will they be included. Note that their existence is not checked. Only a regex based validation is performed.

It was also important to notify the user about these events when we detect that a specific section is invalid and provide informative and helpful error messages without failing the build. Hence proper error messages were also added which were informative so that the user knows exactly which section is to blame. This is similar to compilers where the error message is crucial to debug a certain piece of code.

Resources

Continue ReadingHandling Errors While Parsing the yaml File in Yaydoc

Implementing SUSI Linux App as a Finite State Machine

SUSI Linux app provides access to SUSI on Linux distributions on desktop as well as hardware devices like Raspberry Pi. It is a headless client that can be used to interact to SUSI via voice only. As more and more features like multiple hotword detection support and wake button support was added to SUSI Linux, the code became complex to understand and manage. A system was needed to model the app after. Finite State Machine is a perfect approach for such system.

The Wikipedia definition of State Machine is

It is an abstract machine that can be in exactly one of a finite number of states at any given time. The FSM can change from one state to another in response to some external inputs; the change from one state to another is called a transition.”

This means that if you can model your app into a finite number of states, you may consider using the State Machine implementation.

State Machine implementation has following advantages:

  • Better control over the working of the app.
  • Improved Error handling by making an Error State to handle errors.
  • States work independently which helps to modularize code in a better form.

To begin with, we declare an abstract State class. This class declares all the common properties of a state and transition method.

from abc import ABC, abstractclassmethod
import logging

class State(ABC):
   def __init__(self, components):
       self.components = components
       self.allowedStateTransitions = {}

   @abstractclassmethod
   def on_enter(self, payload=None):
       pass

   @abstractclassmethod
   def on_exit(self):
       pass

   def transition(self, state, payload=None):
       if not self.__can_transition(state):
           logging.warning("Invalid transition to State{0}".format(state))
           return

       self.on_exit()
       state.on_enter(payload)

   def __can_transition(self, state):
       return state in self.allowedStateTransitions.values()

We declared the on_enter() and on_exit() abstract method. These methods are executed on the entering and exiting a state respectively. The task designated for the state can be performed in the on_enter() method and it can free up resources or stop listening to callbacks in the on_exit() method. The transition method is to transition between one state to another. In a state machine, a state can transition to one of the allowed states only. Thus, we check if the transition is allowed or not before continuing it. The on_enter() and transition() methods additionally accepts a payload argument. This can be used to transfer some data to the state from the previous state.

We also added the components property to the State. Components store the shared components that can be used across all the State and are needed to be initialized only once. We create a component class declaring all the components that are needed to be used by states.

class Components:

   def __init__(self):
       recognizer = Recognizer()
       recognizer.dynamic_energy_threshold = False
       recognizer.energy_threshold = 1000
       self.recognizer = recognizer
       self.microphone = Microphone()
       self.susi = susi
       self.config = json_config.connect('config.json')

       if self.config['hotword_engine'] == 'Snowboy':
           from main.hotword_engine import SnowboyDetector
           self.hotword_detector = SnowboyDetector()
       else:
           from main.hotword_engine import PocketSphinxDetector
           self.hotword_detector = PocketSphinxDetector()

       if self.config['wake_button'] == 'enabled':
           if self.config['device'] == 'RaspberryPi':
               from ..hardware_components import RaspberryPiWakeButton
               self.wake_button = RaspberryPiWakeButton()
           else:
               self.wake_button = None
       else:
           self.wake_button = None

Now, we list out all the states that we need to implement in our app. This includes:

  • Idle State: App is listening for Hotword or Wake Button.
  • Recognizing State: App actively records audio from Microphone and performs Speech Recognition.
  • Busy State: SUSI API is called for the response of the query and the reply is spoken.
  • Error State: Upon any error in the above state, control transfers to Error State. This state needs to handle the speak the correct error message and then move the machine to Idle State.

Each state can be implemented by inheriting the base State class and implementing the on_enter() and on_exit() methods to implement the correct behavior.

We also declare a SUSI State Machine class to store the information about current state and declare the valid transitions for all the states.

class SusiStateMachine:
   def __init__(self):
       super().__init__()
       components = Components()
       self.__idle_state = IdleState(components)
       self.__recognizing_state = RecognizingState(components)
       self.__busy_state = BusyState(components)
       self.__error_state = ErrorState(components)
       self.current_state = self.__idle_state

       self.__idle_state.allowedStateTransitions = \
           {'recognizing': self.__recognizing_state, 'error': self.__error_state}
       self.__recognizing_state.allowedStateTransitions = \
           {'busy': self.__busy_state, 'error': self.__error_state}
       self.__busy_state.allowedStateTransitions = \
           {'idle': self.__idle_state, 'error': self.__error_state}
       self.__error_state.allowedStateTransitions = \
           {'idle': self.__idle_state}

       self.current_state.on_enter(payload=None)

We also set Idle State as the current State of the System. In this way, State Machine approach is implemented in SUSI Linux.

Resources:

 

Continue ReadingImplementing SUSI Linux App as a Finite State Machine

Checking Image Size to Avoid Crash in Android Apps

In Giggity app a user can create a shortcut for the event by clicking on “home shortcut” button in the navigation drawer. Open Event format provides the logo URL in the return data so we do not need to provide it separately in the app’s raw file.

Sometimes the image can be too big to be put on screen as icon for shortcut. In this blog I describe a very simple method to check if we should use the image or not to avoid the crash and pixelation due to resolution.

We can store the image received in bitmap format. A bitmap is a type of memory organization or image file format used to store digital images. The term bitmap comes from the computer programming terminology, meaning just a map of bits, a spatially mapped array of bits. By storing it in bitmap format we can easily get the necessary information about the image to check if it is suitable for use.

We can use the BitmapFactory class which provides several decoding methods like (decodeByteArray(), decodeFile(), decodeResource(), etc.) for creating a Bitmap from various sources. Choose the most appropriate decode method based on your image data source. These methods attempt to allocate memory for the constructed bitmap and therefore can easily result in an OutOfMemory exception. Each type of decode method has additional signatures that let you specify decoding options via the BitmapFactory.Options class. Setting the inJustDecodeBounds property to true while decoding avoids memory allocation, returning null for the bitmap object but setting outWidth, outHeight and outMimeType. This technique allows you to read the dimensions and type of the image data prior to construction (and memory allocation) of the bitmap.

BitmapFactory.Options options = new BitmapFactory.Options();
options.inJustDecodeBounds = true;
BitmapFactory.decodeResource(getResources(), R.id.myimage, options);
int imageHeight = options.outHeight;
int imageWidth = options.outWidth;
String imageType = options.outMimeType;

To avoid java.lang.OutOfMemory exceptions, check the dimensions of a bitmap before decoding it, unless you absolutely trust the source to provide you with predictably sized image data that comfortably fits within the available memory.

So here is the particular example from Giggity app, it avoids crash on the recieving a large image for the icon. So once we store the the image in bitmap format we check if the height and width of the icon is exceeding the maximum limit.

public Bitmap getIconBitmap() {

 InputStream stream = getIconStream();
 Bitmap ret = null;

 if (stream != null) {
 ret = BitmapFactory.decodeStream(stream);
 if (ret == null) {
 Log.w("getIconBitmap", "Discarding unparseable file");
 return null;
 }
 if (ret.getHeight() > 512 || ret.getHeight() != ret.getWidth()) {
 Log.w("getIconBitmap", "Discarding, icon not square or >512 pixels");
 return null;
 }
 if (!ret.hasAlpha()) {
 Log.w("getIconBitmap", "Discarding, no alpha layer");
 return null;
 }
 }
 
 return ret;
}

If it does then we can avoid the icon. In this case we check if the icon is more than 512 pixels in height and width. If it is so then we could avoid it.

We could also check if the icon has a transparent background by using “hasAlpha” so we could have uniformity in the icons displayed on the screen. In the final result you can see the icon of the TUBIX 2017 conference added on the screen as it was following all those defined criterias.

Now that the image dimensions are known, they can be used to decide if the full image should be loaded into memory or if a subsampled version should be loaded instead. Here are some factors to consider:

  • Estimated memory usage of loading the full image in memory.
  • Amount of memory you are willing to commit to loading this image given any other memory requirements of your application.
  • Dimensions of the target ImageView or UI component that the image is to be loaded into.
  • Screen size and density of the current device.

For example, it’s not worth loading a 1024×768 pixel image into memory if it will eventually be displayed in a 128×96 pixel thumbnail in an ImageView.

 

References:

Continue ReadingChecking Image Size to Avoid Crash in Android Apps

R14 – Memory Quota Exceeded

We, like many other organisations, are using heroku as the deployment server for our project open event organizer server. Things are pretty simple and awesome when your project is in its beginning phase and things run pretty smoothly. But as your project grows, there comes some server problem. And one of the biggest problems as your project grows is memory. Now since various packages have a different amount of memory assigned to you in case of hosting in generic servers such as heroku, so it might result in memory quota exceeded. Recently, we faced such a problem. R14 – Memory Quota Exceeded. Took us quite some time to understand what and why and how this occurred. So let me share a few things I found about this error.

Continue ReadingR14 – Memory Quota Exceeded

Error Handling in Retrofit 2

For the Open Event android app we were using retofit 1.9 with an okhttp stack plus a gson parser but recently retrofit 2.0 was released and it was a major update in the sense that it a lot of things have been changed.

For starters, you don’t have to declare synchronous and asynchronous requests upfront and you can just decide that while executing. The code for that will look something like this. This is how we define our request methods in our api service

import retrofit.Call;
public interface APIService {
   @POST(“/list”)
   Call<Repo> loadRepo();
}

Now if we want to make a synchronous request, we can make it like

Call<Repo> call = service.loadRepo();
Repo repo = call.execute();

and for an asynchronous request, we can call enqueue()

Call<Repo> call = service.loadRepo();
call.enqueue(new Callback<Repo>() {
    @Override
    public void onResponse(Response<Repo> response) {
    // Get result Repo from response.body()    
    }
    @Override
    public void onFailure(Throwable t) {

    }
});

And another thing that changed in the async call throws a throwable on failure, so essentially the RetrofitError class is gone and since we were using that in our app, we had to modify the whole error handling in the app, basically from the grounds up.

So, when we decided to move to retrofit 2 after the stable version was released, we had to change a lot of code and the main part that was affected was the error handling. So, replacing the retrofitError class, I used the throwable directly to retrieve the error type something like this

if (error.getThrowable() instanceof IOException) { 
    errorType = “Timeout”; 
    errorDesc = String.valueOf(error.getThrowable().getCause()); 
} 
else if (error.getThrowable() instanceof IllegalStateException) {                 
    errorType = “ConversionError”; 
    errorDesc = String.valueOf(error.getThrowable().getCause()); 
} else { 
    errorType = “Other Error”; 
    errorDesc = String.valueOf(error.getThrowable().getLocalizedMessage()); 
}

This was ofcourse for all failure events. And to handle all response events I compared the HTTP status codes and displayed the errors :

Integer statusCode = response.getStatusCode(); 
if (statusCode.equals(404)) { 
    // Show Errors in a dialog
    showErrorDialog(“HTTP Error”, statusCode + “Api Not Found”); 
}

This is how we can compare other HTTP errors in retrofit and assign the correct status accordingly. I personally think that this is a better implementation than Retrofit 1.9 and the RetrofitError was a bit tedious to work with. It wasn’t very thought of before implementation because it was not easy to tell what kind of error exactly occured. With Response codes, one can see what are the exact error one faces and can gracefully handle these errors.

Continue ReadingError Handling in Retrofit 2