Feature to generate Config File in PSLab Android application

In this blog, I will explain the feature to generate “Config File” in PSLab Android Application 

What is a Config File?

The main aim of this feature is to make PSLab board a self data logger, which would read user-defined configs from a config file stored on SD card connected to PSLab board and based on instrument, parameters and time interval stored in config file PSLab board would automatically log those values. 

Now as the first step of this feature, an option is added to PSLab Android application, where user can create a config file. User can select an instrument, parameters associated with that instrument and time interval. With this feature, user can easily generate a config file which can later be used by PSLab board for logging.

User Interface

The option to generate a config file is given in the side navigation menu on the main screen. 

(Figure 1: Generate Config file menu)

Once the user selects the “Generate Config File” option, the user will be directed to the following screen where user can create a config file with intended parameters

(Figure 2: Generate Config File UI)

As can be seen in the screenshot above the user can select instruments for which the config file needs to be created from a drop-down menu. User can specify the time interval, for which the data should be logged by the PSLab board. Based on the instrument selected by the user corresponding parameters will be shown at the bottom. User can select whichever parameters are required and click on “CREATE CONFIG FILE” button and a config file will be saved on device local storage. 

A config file for Oscilloscope with 25-sec interval and CH1, CH2 and CH3 parameters would look something like below,

(Figure 3: Sample config File )

Implementation

When a user clicks on Create Config File button, First we check whether the user has provided a time interval, if not a toast message appears to let the user know that time interval is missing. This is done using the following lines of code,

createConfigFileBtn.setOnClickListener(new View.OnClickListener() {
  @Override
  public void onClick(View v) {
     interval = intervalEditText.getText().toString();
     if (interval.length() == 0) {
        Toast.makeText(CreateConfigActivity.this, getResources().getString(R.string.no_interval_message), Toast.LENGTH_SHORT).show();
                }

Once the user sets the time interval and selects the parameters, the following lines of code generates a string array containing params selected by the user.

ArrayList<String> selectedParamsList = new ArrayList<>();
for (int i = 0; i < paramsListContainer.getChildCount(); i ++) {
    CheckBox checkBox = (CheckBox) paramsListContainer.getChildAt(i);
    if (checkBox.isChecked()) {
       selectedParamsList.add(instrumentParamsList.get(selectedItem)[i]);
    }
}

After we have the list of selected parameters we call the following function to create the config file

private void createConfigFile(ArrayList<String> params) {
        String instrumentName = instrumentsList.get(selectedItem);
        String fileName = "pslab_config.txt";
        String basepath = Environment.getExternalStorageDirectory().getAbsolutePath();

        File baseDirectory = new File(basepath + File.separator + CSVLogger.CSV_DIRECTORY);
        if (!baseDirectory.exists()) {
            try {
                baseDirectory.mkdir();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }

        File configFile = new File(basepath + File.separator + CSVLogger.CSV_DIRECTORY + File.separator + fileName);
        if (!configFile.exists()) {
            try {
                configFile.createNewFile();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        try {
            FileWriter writer = new FileWriter(configFile);
            writer.write("instrument: " + instrumentName + "\n");
            writer.write("interval: " + interval + " " + intervalUnit + "\n");
            String param = String.join(",", params);
            writer.write("params: " + param);
            writer.flush();
            writer.close();
            CustomSnackBar.showSnackBar(rootView, getString(R.string.file_created_success_message), null, null, Snackbar.LENGTH_SHORT);
        } catch (IOException e) {
            e.printStackTrace();
            CustomSnackBar.showSnackBar(rootView, getString(R.string.file_created_fail_message), null, null, Snackbar.LENGTH_SHORT);
        }

    }

In the first part of this function, we check whether there exists a PSLab directory in the local storage of the device, if not the directory is created. After that, we create a file named “pslab_config.txt”. After that, we use FileWriter to write data to the file. 

In a nutshell with this feature user can create config files easily. The following GIF demonstrated this functionality.

(Figure 4: GIF of the functionality)

References

Tags: PSLab, Android, GSoC 19, Config File, data logger

Continue ReadingFeature to generate Config File in PSLab Android application

Implementing Render Route & Security Checks for Attendee Tickets

This blog post explains the requirements & implementation details of a secure route over which the tickets could be served in the Open Event Project (Eventyay). Eventyay is the Open Event management solution using standardized event formats developed at FOSSASIA. Sometimes, tickets of a user can be utilized in the process of fraudulent actions. To prevent this, security is of the utmost importance.

Prior to this feature, anonymous/unauthorized users were able to access the tickets which belonged to another user with a simple link. There was no provision of any authentication check.  An additional problem with the tickets were the storage methodology where the tickets were stored in a top-level folder which was not protected. Therefore, there was a necessity to implement a flask route which could check if the user was authenticated, the ticket belonged to the authorized user/admin/organizer of the event and provide proper exceptions in other cases. 

Ticket completion page with option to download tickets

When the user places an order and it goes through successfully,the ticket is generated, stored in a protected folder and the user is redirected to the order completion page where they would be able to download their tickets. When the user clicks on the Download Tickets Button, the ticket_blueprint route is triggered.

@ticket_blueprint.route('/tickets/<string:order_identifier>')
@jwt_required
def ticket_attendee_authorized(order_identifier):
    if current_user:
        try:
            order = Order.query.filter_by(identifier=order_identifier).first()
        except NoResultFound:
            return NotFoundError({'source': ''}, 'This ticket is not associated with any order').respond()
        if current_user.can_download_tickets(order):
            key = UPLOAD_PATHS['pdf']['tickets_all'].format(identifier=order_identifier)
            file_path = '../generated/tickets/{}/{}/'.format(key, generate_hash(key)) + order_identifier + '.pdf'
            try:
                return return_file('ticket', file_path, order_identifier)
            except FileNotFoundError:
                create_pdf_tickets_for_holder(order)
                return return_file('ticket', file_path, order_identifier)
        else:
            return ForbiddenError({'source': ''}, 'Unauthorized Access').respond()
    else:
        return ForbiddenError({'source': ''}, 'Authentication Required to access ticket').respond()

                         tickets_route – the logic pertaining to security module for attendee tickets

The function associated with the ticket downloads queries the Order model using the order identifier as a key. Then, it checks if the current authenticated user is either a staff member, the owner of the ticket or the organizer of the ticket. If it passes this check, the file path is generated and tickets are downloaded using the return_tickets function.

In the return_tickets function, we utilize the send_file function imported from Flask and wrap it with flask’s make_response function. In addition to that, we attach headers to specify that it is an attachment and add an appropriate name to it.

def return_file(file_name_prefix, file_path, identifier):
    response = make_response(send_file(file_path))
    response.headers['Content-Disposition'] = 'attachment; filename=%s-%s.pdf' % (file_name_prefix, identifier)
    return response

return_tickets function – sends the file as a make_response with appropriate headers

When it comes to exception handling, at each stage whenever a ticket is not to be found while querying or the authentication check fails, a proper exception is thrown to the user. For example, at the step where an attempt is made to return the file using file path after authentication, if the tickets are NotFound, the tickets are generated on the fly. 

 def can_download_tickets(self, order):
        permissible_users = [holder.id for holder in order.ticket_holders] + [order.user.id]
        if self.is_staff or self.has_event_access(order.event.id) or self.id in permissible_users:
            return True
        return False

can_download_tickets – check for proper ticket access 

Resources:

Related work and code repo:

Tags:

Eventyay, FOSSASIA, Flask, SQLAlchemy, Open Event, Python, JWT

Continue ReadingImplementing Render Route & Security Checks for Attendee Tickets

Creating SMTP as a Fallback Function to Ensure Emails Work

This blog post explains the solution to scenarios pertaining to failure/ unavailability of Sendgrid service when an attempt is made to send emails in Eventyay. Eventyay is the outstanding Open Event management solution using standardized event formats developed at FOSSASIA.

Currently, the Open Event Project utilizes 2 protocols namely, SendGrid and SMTP. Previously, they could be configured only be configured individually. If either protocol failed, there was no provision for a backup and the task to send the email would fail. 

Therefore, there was a necessity to develop a feature where the SMTP protocol, which is usually more reliable than 3rd party services, could act as a backup when sendgrid server is unavailable or the allotted quota has been exceeded.

def check_smtp_config(smtp_encryption):
    """
    Checks config of SMTP
    """
    config = {
                'host': get_settings()['smtp_host'],
                'username': get_settings()['smtp_username'],
                'password': get_settings()['smtp_password'],
                'encryption': smtp_encryption,
                'port': get_settings()['smtp_port'],
    }
    for field in config:
        if field is None:
            return False
    return True

  Function to check if SMTP has been properly configured

The main principle which was followed to implement this feature was to prevent sending emails when SMTP is not configured. Ergo, a function was implemented to check if the host, username, password, encryption and port was present in the model before proceeding. If this was configured properly, we move on to determining the protocol which was enabled. For this, we have 2 separate celery tasks, one for SMTP and the other for Sendgrid. 

@celery.task(name='send.email.post.sendgrid')
def send_email_task_sendgrid(payload, headers, smtp_config):
    try:
        message = Mail(from_email=From(payload['from'], payload['fromname']),
                       to_emails=payload['to'],
                       subject=payload['subject'],
                       html_content=payload["html"])
        if payload['attachments'] is not None:
            for attachment in payload['attachments']:
                with open(attachment, 'rb') as f:
                    file_data = f.read()
                    f.close()
                encoded = base64.b64encode(file_data).decode()
                attachment = Attachment()
                attachment.file_content = FileContent(encoded)
                attachment.file_type = FileType('application/pdf')
                attachment.file_name = FileName(payload['to'])
                attachment.disposition = Disposition('attachment')
                message.add_attachment(attachment)
        sendgrid_client = SendGridAPIClient(get_settings()['sendgrid_key'])
        logging.info('Sending an email regarding {} on behalf of {}'.format(payload["subject"], payload["from"]))
        sendgrid_client.send(message)
        logging.info('Email sent successfully')
    except urllib.error.HTTPError as e:
        if e.code == 429:
            logging.warning("Sendgrid quota has exceeded")
            send_email_task_smtp.delay(payload=payload, headers=None, smtp_config=smtp_config)
        elif e.code == 554:
            empty_attachments_send(sendgrid_client, message)
        else:
            logging.exception("The following error has occurred with sendgrid-{}".format(str(e)))


@celery.task(name='send.email.post.smtp')
def send_email_task_smtp(payload, smtp_config, headers=None):
    mailer_config = {
            'transport': {
                'use': 'smtp',
                'host': smtp_config['host'],
                'username': smtp_config['username'],
                'password': smtp_config['password'],
                'tls': smtp_config['encryption'],
                'port': smtp_config['port']
            }
        }

    try:
        mailer = Mailer(mailer_config)
        mailer.start()
        message = Message(author=payload['from'], to=payload['to'])
        message.subject = payload['subject']
        message.plain = strip_tags(payload['html'])
        message.rich = payload['html']
        if payload['attachments'] is not None:
            for attachment in payload['attachments']:
                message.attach(name=attachment)
        mailer.send(message)
        logging.info('Message sent via SMTP')
    except urllib.error.HTTPError as e:
        if e.code == 554:
            empty_attachments_send(mailer, message)
    mailer.stop()

Falling back to SMTP when the system has exceeded the sendgrid quota

Consider the function associated with the sendgrid task. The logic which sends the emails along with the payload is present in a try/catch block. When an exception occurs while attempting to send the email, it is caught via the requests library and checks for the HTTP code. If the code is determined as 429, this implies that there were TOO_MANY_REQUESTS going through or otherwise in sendgrid lingo, it means that you’ve exceeded your quota. In this case, we will not stop sending the email, rather, we would alternate to a backup solution of sending it via the SMTP protocol. In this way, we can ensure that emails would work without any kind of hassle.

def send_email(to, action, subject, html, attachments=None):
    """
    Sends email and records it in DB
    """
    from .tasks import send_email_task_sendgrid, send_email_task_smtp
    if not string_empty(to):
        email_service = get_settings()['email_service']
        email_from_name = get_settings()['email_from_name']
        if email_service == 'smtp':
            email_from = email_from_name + '<' + get_settings()['email_from'] + '>'
        else:
            email_from = get_settings()['email_from']
        payload = {
            'to': to,
            'from': email_from,
            'subject': subject,
            'html': html,
            'attachments': attachments
        }

        if not current_app.config['TESTING']:
            smtp_encryption = get_settings()['smtp_encryption']
            if smtp_encryption == 'tls':
                smtp_encryption = 'required'
            elif smtp_encryption == 'ssl':
                smtp_encryption = 'ssl'
            elif smtp_encryption == 'tls_optional':
                smtp_encryption = 'optional'
            else:
                smtp_encryption = 'none'

            smtp_config = {
                'host': get_settings()['smtp_host'],
                'username': get_settings()['smtp_username'],
                'password': get_settings()['smtp_password'],
                'encryption': smtp_encryption,
                'port': get_settings()['smtp_port'],
            }
            smtp_status = check_smtp_config(smtp_encryption)
            if smtp_status:
                if email_service == 'smtp':
                    send_email_task_smtp.delay(payload=payload, headers=None, smtp_config=smtp_config)
                else:
                    key = get_settings().get('sendgrid_key')
                    if key:
                        headers = {
                            "Authorization": ("Bearer " + key),
                            "Content-Type": "application/json"
                        }
                        payload['fromname'] = email_from_name
                        send_email_task_sendgrid.delay(payload=payload, headers=headers, smtp_config=smtp_config)
                    else:
                        logging.exception('SMTP & sendgrid have not been configured properly')

            else:
                logging.exception('SMTP is not configured properly. Cannot send email.')
        # record_mail(to, action, subject, html)
        mail = Mail(
            recipient=to, action=action, subject=subject,
            message=html, time=datetime.utcnow()
        )

        save_to_db(mail, 'Mail Recorded')
        record_activity('mail_event', email=to, action=action, subject=subject)
    return True

  Fallback logic – SMTP/Sendgrid

Resources:

Related work and code repo:

Tags:

Eventyay, FOSSASIA, Flask, SMTP, Open Event, Sendgrid, Python

Continue ReadingCreating SMTP as a Fallback Function to Ensure Emails Work

Allowing Event Owner to Transfer an event in Eventyay

This blog post will showcase an option which can be used by an event owner to transfer his event to another user in Open Event Frontend. Till the invited user accepts the invitation, the previous owner will have all the rights but as soon as the invited user becomes the new owner of the event the previous owner will cease to have any control over the event.

It is a 2-step process just to ensure that user doesn’t transfers the event accidentally.

The user needs to go to Settings option of the event. The user will get an option to transfer event in the form of a red button along with the following text:

Transfer ownership of this event to another user. You’ll lose all the owner rights once they accept the ownership.
Option to transfer event in Open Event Frontend

When a user clicks on the option to transfer the event, a modal pops up asking the user to confirm the event name. Once user fills in correct event name the Proceed button becomes active.

Modal to confirm event name

The code snippet which triggers the action to open the modal event-transfer-modal is given below:

<button {{action 'openEventTransferModal' model.event.id
model.event.name}} class='ui red button'>
   {{t 'Transfer Ownership'}}
</button>
openEventTransferModal(id, name) {    
    this.setProperties({
       'isEventTransferModalOpen' : true,
       'confirmEventName'         : '',
       'eventId'                  : id,
       'eventName'                : name
     });
 }

The code snippet which takes care of event name confirmation to make Proceed button active:

isNameDifferent  : computed('confirmEventName',  
      'eventName', function() {   
           return this.eventName ? 
           this.confirmEventName.toLowerCase() !==
           this.eventName.toLowerCase() : true;
       })

When user confirms the event name and hits Proceed button, a new modal appears which asks users to fill in the email of the user to whom the event is to be transferred. Also, the user needs to check a checkbox to ensure that he/she agrees to the terms of event transferring.

Final confirmation to transfer the event

The code snippet which triggers the action to open the modal confirm-event-transfer-modal is given below:

<button {{action openConfirmEventTransferModal}} class="ui red button {{if isNameDifferent 'disabled'}}">   
    {{t 'Proceed'}}
</button>
openConfirmEventTransferModal() {     
     const currentInvite =
             this.model.roleInvites.createRecord({});
     let { roles } = this.model;
     for (const role of roles ? roles.toArray() : [])    
     {
         if (role.name === 'owner') {
             currentInvite.set('role', role);
         }
     }
     this.setProperties({
         'isEventTransferModalOpen'        : false,
         'isConfirmEventTransferModalOpen' : true,
         'checked'                         : false,
         currentInvite
     });
   }

When the confirm-event-transfer-modal is to be opened, a new role invite is created and passed to the modal so that when the user fills in the email of the new owner, the role invite is simply updated by a PATCH request. 

When the user fills in email ID and enters Transfer button, the transferEvent() function is called.

async transferEvent() {
     try {
       this.set('isLoading', true);
       this.currentInvite.set('roleName', 'owner');
       await this.currentInvite.save();
       this.setProperties({
         'isConfirmEventTransferModalOpen' : false,
         'checked'                         : false
       });
       this.notify.success(this.l10n.t('Owner Role Invite sent
       successfully.'));
     } catch (error) {
       this.notify.error(this.l10n.t(error.message));
     }
     this.set('isLoading', false);
   }
 }

The transferEvent() function updates the role invited created while opening of confirm-event-transfer-modal and then save() function is called upon the role invite. All the modals are then closed and role invite mail is sent to the new owner. When the new owner clicks on the link in the mail and accepts the invite, event is transferred to him/her and the previous owner is deprived of any control over the event.

Resources:

Related work and code repo:

Continue ReadingAllowing Event Owner to Transfer an event in Eventyay

Migration to Model-View-ViewModel Architecture and LiveData in Open Event Organizer App

Open Event Organizer App (Eventyay Organizer App) is the Android app used by event organizers to create and manage events on the Eventyay platform as well as check-in and check-out attendees along with other functionalities. The app used the MVP (Model-View-Presenter) architecture and is being ported to MVVM (Model-View-ViewModel). This article will explain the procedure of migrating MVP to MVVM architecture and implementing LiveData. 

Why migrate to MVVM?

The MVVM architecture is designed to store and manage UI-related data in a lifecycle conscious way. Configuration changes such as screen rotations are handled properly by ViewModels.

Tight Coupling:

The issue of tight coupling is resolved since only the View holds the reference to ViewModel and not vice versa. A single View can hold references to multiple ViewModels.

Testability:

Since Presenters are hard bound to Views, writing unit tests becomes slightly difficult as there is a dependency of a View.

ViewModels are more unit test friendly as they can be independently tested. There is no dependency of the View.

Here, the implementation is being described with the example of About Event module in the Open Event Organizer App.

First step is the creation of a new class AboutEventViewModel which extends ViewModel.

@Binds
@IntoMap
@ViewModelKey(AboutEventViewModel.class)
public abstract ViewModel bindAboutEventViewModel(AboutEventViewModel aboutEventViewModel);

The new ViewModel has to be added to the ViewModelModule:

Constructor for the ViewModel:

@Inject
public AboutEventViewModel(EventRepository eventRepository,  CopyrightRepository copyrightRepository,
DatabaseChangeListener<Copyright> copyrightChangeListener) {
    this.eventRepository = eventRepository;
    this.copyrightRepository = copyrightRepository;
    this.copyrightChangeListener = copyrightChangeListener;

    eventId = ContextManager.getSelectedEvent().getId();
}

We are using Dagger2 for dependency injection. 

LiveData

LiveData is a lifecycle-aware data holder with the observer pattern.

When we have a LiveData object (e.g. list of attendees), we can add some LifecycleOwner (it can be Activity or Fragment) as an observer. Using this:

The Activity or Fragment will remain updated with the data changes.

Observers are only notified if they are in the STARTED or RESUMED state which is also known as the active state. This prevents memory leaks and NullPointerExceptions because inactive observers are not notified about changes.

Now, let’s discuss about the implementation of LiveData. We will create objects of SingleEventLiveData<> class.

private final SingleEventLiveData<Boolean> progress = new SingleEventLiveData<>();
private final SingleEventLiveData<String> error = new SingleEventLiveData<>();
private final SingleEventLiveData<Event> success = new SingleEventLiveData<>();
private final SingleEventLiveData<Copyright> showCopyright = new SingleEventLiveData<>();
private final SingleEventLiveData<Boolean> changeCopyrightMenuItem = new SingleEventLiveData<>();
private final SingleEventLiveData<String> showCopyrightDeleted = new SingleEventLiveData<>();

The functions to get the LiveData objects:

public LiveData<Boolean> getProgress() {
    return progress;
}

public LiveData<Event> getSuccess() {
    return success;
}

public LiveData<String> getError() {
    return error;
}

public LiveData<Copyright> getShowCopyright() {
    return showCopyright;
}

public LiveData<Boolean> getChangeCopyrightMenuItem() {
    return changeCopyrightMenuItem;
}

public LiveData<String> getShowCopyrightDeleted() {
    return showCopyrightDeleted;
}

Now, we can remove getView() methods and instead, these objects will be used to call various methods defined in the fragment.

Let’s discuss the changes required in the AboutEventFragment now.

The Fragment will have ViewModelProvider.Factory injected.

@Inject
ViewModelProvider.Factory viewModelFactory;

Declare an object of the ViewModel.

private AboutEventViewModel aboutEventViewModel;

Then, in onCreateView(), viewModelFactory will be passed to the ViewModelProviders.of() method as the factory, which is the second parameter.

aboutEventViewModel = ViewModelProviders.of(this, viewModelFactory).get(AboutEventViewModel.class);

Replace all references to the Presenter with references to the ViewModel.

Add the Fragment as an observer to the changes by adding the following in the onStart() method:

aboutEventViewModel.getProgress().observe(this, this::showProgress);
aboutEventViewModel.getSuccess().observe(this, this::showResult);
aboutEventViewModel.getError().observe(this, this::showError);
aboutEventViewModel.getShowCopyright().observe(this, this::showCopyright);
aboutEventViewModel.getChangeCopyrightMenuItem().observe(this, this::changeCopyrightMenuItem);
aboutEventViewModel.getShowCopyrightDeleted().observe(this, this::showCopyrightDeleted);

Two parameters are passed to the observe() method  –  first one is LifecycleOwner, which is our Fragment in this case. The second one is a callback along with a parameter and is used to call the required method.

With this, the implementation of MVVM and LiveData is brought to completion.

Resources:

Documentation: ViewModel, LiveData

Further reading:

Open Event Organizer App: Project repo, Play Store, F-Droid

Continue ReadingMigration to Model-View-ViewModel Architecture and LiveData in Open Event Organizer App

Transmitting data from SD card through Arduino – Neurolab record feature

In the Neurolab-Android app, we have been using an Arduino board for development as a workaround to the actual Neurolab hardware which is currently in the manufacturing stages. Along with the Arduino, we use a Micro SD card module with an SD card attached containing a dataset file in it. This combination of the Arduino and the SD card module serves as the source of dataflow into the Android app.

Firstly, we need to get the Arduino programmed for reading the dataset file from the SD card in the SD card module. We program the Arduino using the Arduino IDE on any desktop platform.

1. Import required libraries before starting off, namely the SPI and SD libraries for serial communication and SD card related functions respectively. Let us also set up some constant values which are going to be used in various parts of the code needed to program the Arduino for our required purpose.

#include <SD.h>
#include <SPI.h>
int baudRate = 9600;
int chipSelect = 4;
String fileName = "Dataset1.csv";

The ‘chipSelect’ variable denotes the chip select pin number for the connected Micro SD card adapter (module).

The baud rate for the Arduino board has been set to 9600 as a default. The dataset stored in the SD card from which the data needs to be read has been named “Dataset1”.

2. Next, we will need to initialize the SD card to check even if an SD card is inserted or not in the SD card module. We create a function for this purpose named ‘initializeSDCard’ in the following way:

bool initializeSDCard() {
  if (!SD.begin(chipSelect)) { 
    return false;   }
  return true;
}

The function will return true if initialization was successful. An unsuccessful initialization may also be due to the fact of SD card corrupted apart from not being properly inserted into the module.

3. Now we will be opening the dataset file from the SD card, making it ready to be read from.

File openTheFileFromSDCard(String fileName) {
  File file = SD.open(fileName);  
  if (!file) {
    Serial.println("error opening: " + fileName);
    return file;
  }
  return file;
}

The function will take in the file name as an argument and open it from the SD card. If the file is not found in the SD card, it will simply print out an error message to the serial output. It returns the file object in a true or false context accordingly.

4. We are now going to read data from the dataset in the SD card line by line using a function. The dataset file instance is passed as an argument to this function. This file is read and transmitted over the serial output channel line by line with ‘\n’ as the line delimiter, skipping null lines.

void readFromSDCardToSerialOutputLineByLine(File file) {
  String line;
  while (file.available()) {
    line = file.readStringUntil('\n');
    line.trim();
    if (line != "") {
      Serial.println(line);
    }
  }
  Serial.println("Recorded");
  file.close();
}

This function will keep reading from the file untill it has nothing more i.e till the end of file.

5. Next, we update the setup function which is the function to be executed for the programming of the Arduino board. Here, we call our defined functions according to our need logical need.

void setup() {
  Serial.begin(baudRate);

  pinMode(chipSelect, OUTPUT);

  if (initializeSDCard()) {
    File file = openTheFileFromSDCard(fileName);   
    if (file) {
      readFromSDCardToSerialOutputLineByLine(file);
    }
  }
}

Here, we initialize the serial communication at 9600 bits per second (baud rate), specify the pin mode which is the chip select pin in our case.

6. We leave the loop( ) function module as it is, as we do not need any iterative routine which runs over and over while programming our Arduino.

We are good to go now. Connect the Arduino to the desktop, compile the code and upload it to the circuit board.

Once programmed, we can get the output of the data transmission on the serial monitor in the following way:

Note – In the picture, the dataset name is ‘k24bit’.

Hope this blog adds value to your software development skills.

References:

  1. https://www.hackerearth.com/blog/developers/arduino-programming-for-beginners/
  2. https://youtu.be/sS_oW81NweI
  3. https://www.arduino.cc/en/main/software 

Tags: FOSSASIA, GSOC19, Arduino, Neurolab, Programming, Hardware

Continue ReadingTransmitting data from SD card through Arduino – Neurolab record feature

Designing and optimising new invoice PDFs

The Open Event project has proven to be an excellent event management application with a growing user base. With recent workflow refactors in the order process in open-event-frontend and introduction of event invoices (to be rolled out this month as a work product), the open-event-server’s invoices required a makeover. A ticket buyer is now required to give their billing information if the order is comprised of paid tickets and to accommodate this, and long information addresses, optimisation was required.

Restructuring order invoices

The new order invoices use nested tables concept instead of previously used two-cell tables. The pros of this new design is the accomodation of long-addresses and corresponding changes in billing information display.

{% if order.is_billing_enabled %}
                      <
td style="text-align:center;">
                          <
table>
                              <
tr>
                                  <
td>
                                      <
strong>Company :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.company }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td valign="top">
                                      <
strong>Tax Info :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.tax_business_info }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td valign="top">
                                      <
strong>Address :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.address }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>City :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.city }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>State :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.state }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>Zipcode :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.zipcode }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>Country :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.country }}</strong>
                                  </
td>
                              </
tr>
                          </
table>
                      </
td>
                      {% endif %}

This made sure that the new orders have enough space to prevent information overflow and still maintain the original structure to give a sense of uniformity in old and new PDFs.

Generating new event invoices

The new event invoices needed an overall change in structure. They will be rolling out on 1st of every month, according to current implementation. This required an overall implementation of new invoices. 

First, the published events are taken in consideration for generation of invoices for a particular user. It has been implemented as a scheduled job accordingly.

events = Event.query.filter_by(deleted_at=None, state='published').all()
      for event in events:
          # calculate net & gross revenues
          user = event.owner
          admin_info = get_settings()
          currency = event.payment_currency
          ticket_fee_object = db.session.query(TicketFees).filter_by(currency=currency).one()
          ticket_fee_percentage = ticket_fee_object.service_fee
          ticket_fee_maximum = ticket_fee_object.maximum_fee
          orders = Order.query.filter_by(event=event).all()
          gross_revenue = event.calc_monthly_revenue()
          ticket_fees = event.tickets_sold * (ticket_fee_percentage / 100)
          if ticket_fees > ticket_fee_maximum:
              ticket_fees = ticket_fee_maximum
          net_revenue = gross_revenue - ticket_fees
          payment_details = {
              'tickets_sold': event.tickets_sold,
              'gross_revenue': gross_revenue,
              'net_revenue': net_revenue,
              'amount_payable': ticket_fees
          }
          # save invoice as pdf
          pdf = create_save_pdf(render_template('pdf/event_invoice.html', orders=orders, user=user,
                                admin_info=admin_info, currency=currency, event=event,
                                ticket_fee_object=ticket_fee_object, payment_details=payment_details,
                                net_revenue=net_revenue), UPLOAD_PATHS['pdf']['event_invoice'],
                                dir_path='/static/uploads/pdf/event_invoices/', identifier=event.identifier)
          # save event_invoice info to DB

          event_invoice = EventInvoice(amount=net_revenue, invoice_pdf_url=pdf, event_id=event.id)
          save_to_db(event_invoice)

This function also required one minor modification. The function for calculating monthly revenue had to be updated as to dodge certain unseen bugs related to non completed order amount calculations hence restructuring the function as follows.

def calc_monthly_revenue(self):
      """Returns revenue of current month. Invoice sent every 1st of the month for the previous month"""
      previous_month = datetime.now().month - 1
      orders = Order.query.filter_by(event_id=self.id, status='completed').all()


      monthly_revenue = sum([o.amount for o in orders if o.completed_at and o.completed_at.month == previous_month])
      return monthly_revenue

This enabled the system to finally serve event invoice PDFs. One of whose examples are given above, With this, the open-event-server is finally able to serve event invoices accordingly which can be paid via PayPal to the Eventyay account.

Resources

Related Work and Code Repository

Continue ReadingDesigning and optimising new invoice PDFs

Tracking location on Android – using GPS to record data in Neurolab

In the Neurolab-Android app, we have a feature for recording data. It uses data incoming from the hardware device and stores it in a data table format with various parameters. Two of these parameters happened to be the latitude and longitude (location) of the user using the app. For that, we needed to implement a location tracking feature which can be used while recording the data in the data table.

Let’s start off with adding the required permission uses in the Android Manifest file.

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>

These will be used to ask permissions from the user to access the devices’ internet and location/GPS.

Now, we will be making our Location Tracker class that will be used in tracking the location and getting the corresponding latitude and longitude. 

Firstly, we are going to define some variables – an array of manifest permissions for enabling GPS, some constant values for requesting specific permissions, a provider for the GPS provider.

private String[] mapPermissions = new String[]{
            Manifest.permission.ACCESS_FINE_LOCATION
    };
    public static final int GPS_PERMISSION = 103;
    private static final int UPDATE_INTERVAL_IN_MILLISECONDS = 400;
    private static final int MIN_DISTANCE_CHANGE_FOR_UPDATES = 1;
    private String provider = LocationManager.GPS_PROVIDER;

We also need to initialize and have a location manager ready to request location updates from time to time. Defining a locationManager need to get the system service for location. We define it in the following way:

LocationManager locationManager = (LocationManager)getContext().getSystemService(Context.LOCATION_SERVICE)

Next, we set up a location listener which listens to location changes of the device. We define that in the following way:

private LocationListener locationListener = new LocationListener() {
        @Override
        public void onLocationChanged(Location location) {
            bestLocation = location;
        }

Now, that we have our variables, permissions, location manager and listener set up, we can start capturing the location.

@SuppressLint("MissingPermission")
    public void startCaptureLocation() {
        if (PermissionUtils.checkRuntimePermissions(context, mapPermissions)) {
            locationManager.requestLocationUpdates(provider, UPDATE_INTERVAL_IN_MILLISECONDS, MIN_DISTANCE_CHANGE_FOR_UPDATES,
                    locationListener);
        } else {
            PermissionUtils.requestRuntimePermissions(context, mapPermissions, GPS_PERMISSION);
        }
    }

Firstly, we need to check for the runtime permissions required for our task. We achieve this with the function ‘checkRuntimePermissions’ which we have created in a utility class named ‘PermissionUtils’. The code of this utility class can be found here: PermissionUtils.java. It basically self checks individual permissions from the array passed in the arguments with the Android system.

We then use the location manager instance to request current location updates using the constants and the location listener we defined earlier.

So now, that we have started capturing the user device location, we can get the device Location object values (latitude and longitude). 

@SuppressLint("MissingPermission")
    public Location getDeviceLocation() {
        if (bestLocation == null) {
            if (PermissionUtils.checkRuntimePermissions(context, mapPermissions)) {
                locationManager.requestLocationUpdates(provider,
                        UPDATE_INTERVAL_IN_MILLISECONDS, MIN_DISTANCE_CHANGE_FOR_UPDATES,
                        locationListener);
                return locationManager.getLastKnownLocation(provider);
            } else {
                return defaultLocation();
            }
        } else {
            return bestLocation;
        }
    }

This method requests the location and returns a Location object. If there is an internet connection problem or the device location is disabled from the device settings, it returns a default location object. Here in the Neurolab project, we had set the default location latitude and longitude to 0.0.

Now we can use this location while creating the recorded file in our app directory which can be used after importing that recorded file in the app. We will be storing the location in two columns in the recorded file which is in a table format.

We write the latitude and longitude of the device in the file using a PrintWriter object and get the latitude and longitude of the device in the following way:

long latitude = locationTracker.getDeviceLocation().getLatitude()
long longitude = locationTracker.getDeviceLocation().getLongitude()

Then, using the PrintWriter object we can write the data into the file in the following way:

PrintWriter out = new PrintWriter(new BufferedWriter(new                        FileWriter(csvFile, true)));
out.write(data + "\n");

Here, the ‘data’ contains the latitude and longitude converted to String type.

That’s it! Now you can use the LocationTracker object to capture the location and get the current device location using it in your own app as well.

Hope this blog, adds value to your Android development skills.

References:

  1. https://developer.android.com/reference/android/location/LocationManager.html
  2. https://stackoverflow.com/a/43319075 
  3. https://youtu.be/Ak8uRvlpGS0 

Tags: FOSSASIA, Android, GPS, GSOC 19, Neurolab, Location

Continue ReadingTracking location on Android – using GPS to record data in Neurolab

Neurolab data transfer – Establishing serial communication between Arduino and Android

In the development process of the Neurolab Android, we needed an Arduino-Android connection for transfer of data from datasets which included String and float data type values. In this blog post, I will show you how to establish a serial communication channel between

Android and Arduino through USB cable through which we can transmit data bidirectionally.

Requirements

Hardware:

  1. Android Phone
  2. Arduino (theoretically from any type, but I’ll be using Arduino Uno)
  3. USB 2.0 Cable Type A/B (for Arduino)
  4. OTG Cable (On The Go)
  5. Normal USB Cable (for transferring the data from Android Studio to your phone)

Software:

  1. Android Studio
  2. Arduino IDE

Wiring and Setup

Wiring must be established in the following way:

                                                                                 Figure: Android-Arduino Setup

Working on the Android Side

We would be using the UsbSerial Library by felHR85 for establishing serial

communication.

1. Adding the dependency:

a) Add the following line of code to your app level build.gradle file.

implementation "com.github.felHR85:UsbSerial:$rootProject.usbSerialLibraryVersion"

Note: The ‘usbSerialLibraryVersion’ may change from time to time. Please keep your project with the latest library version. Updates can be found here.

b) Add jitpack to your project.build.gradle file.

allprojects {
   repositories {
       jcenter()
       maven { url "https://jitpack.io" }
   }
}

2. Working with Arduino:

We need to program the Arduino to send and receive data. We achieve that with the help of Arduino IDE as mentioned above. Verify, compile and upload a sketch to the Arduino for sending and receiving data. If you are a complete beginner in Arduino programming, there are example sketches for this same purpose. Load an example sketch from under the communication segment and choose the serial communication sketch. Here, we will be working with a simple sketch for the Arduino such that it simply echoes whatever it receives on the serial port. Here is sketch code:

// the setup routine runs once when you press reset:
void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
}

// the loop routine runs over and over again forever:
void loop() {
  char incomingByte;
   // If there is a data stored in the serial receive buffer, read it and print it to the serial port as human-readable ASCII text.
  if(Serial.available()){  
    incomingByte = Serial.read();
    Serial.print(incomingByte);  
  }
}

Feel free to compile and upload it to your own Arduino.

2. Working with Android:

  1. Firstly, we need an USBManager instance initialized with the system service – ‘USB_SERVICE’. This needs to be done in an Activity (preferably main) so that this instance can be passed to the Communication Handler class, which we are going to create next.
  2. Now, we will be working with a class for handling the serial communications with our Android device and the Arduino board. We would pass the USBManager instance to this class wherein work will be done with that to find the USBDevice and USBDeviceConnection

Starting with, we need to search for any attached Arduino devices to the Android device. We create a method for this and use the ‘getDevicesList’ callback to achieve this in the following way:

public void searchForArduinoDevice(Context context) {
        HashMap usbDevices = usbManager.getDeviceList();

        if (!usbDevices.isEmpty()) {
            boolean keep = true;
            for (Object object : usbDevices.entrySet()) {
                Map.Entry<String, UsbDevice> entry = (Map.Entry<String, UsbDevice>) object;
                device = entry.getValue();

                int deviceVID = device.getVendorId();
                if (deviceVID == ARDUINO_DEVICE_ID) { //Arduino Vendor ID = 0x2341
                    PendingIntent pi = PendingIntent.getBroadcast(context, 0,
                            new Intent(ACTION_USB_PERMISSION), 0);
                    usbManager.requestPermission(device, pi);
                    keep = false;
                } else {
                    connection = null;
                    device = null;
                }
                if (!keep)
                    break;
            }
        }
    }

c. Now, in the Activity where we will be testing or intend to work with the serial connection, we check for the usb permission in a broadcast receiver which is registered in the onCreate method of the activity along with an Intent Filter. The Intent filter has the usp permission as an action added to it.

usbCommunicationHandler = USBCommunicationHandler.getInstance(this, NeuroLab.getUsbManager());

        deviceConnector = new DeviceConnector(NeuroLab.getUsbManager());

        IntentFilter intentFilter = new IntentFilter();
 
        intentFilter.addAction(ACTION_USB_PERMISSION);
        registerReceiver(broadcastReceiver, intentFilter);

d. In the onReceive callback of the broadcast receiver, if the usb permission is granted, we initialize the serial connection with a baud rate for our Arduino device. In this initialization method, we get the connection and serial port of the connected Arduino to the Android device with which we can work. The method is implemented in the following way:

public boolean initializeSerialConnection(int baudRate) {
        connection = usbManager.openDevice(device);
        serialPort = UsbSerialDevice.createUsbSerialDevice(device, connection);
        if (serialPort != null) {
            if (serialPort.open()) { 
                serialPort.setBaudRate(baudRate);
                serialPort.setDataBits(UsbSerialInterface.DATA_BITS_8);
                serialPort.setStopBits(UsbSerialInterface.STOP_BITS_1);
                serialPort.setParity(UsbSerialInterface.PARITY_NONE);
                serialPort.setFlowControl(UsbSerialInterface.FLOW_CONTROL_OFF);
            } else {
                Log.d("SERIAL", "PORT NOT OPEN");
                return false;
            }
        } else {
            Log.d("SERIAL", "PORT IS NULL");
            return false;
        }
        setSerialPort(serialPort);
        return true;
    }

e. Now, with this ‘serialPort’ we can read data from the Arduino in the UsbReadCallback callback from the USBSerialInterface. The data is read in the form of array of bytes. This read data can be used to carry out the various functionalities we want to achieve.

With the above steps we can establish a serial connection between Android and Arduino, transmit data from Arduino to Android device for processing.

The whole working source code of the Neurolab Android project can be found here:

https://github.com/fossasia/neurolab-android/

Thanks for taking the time to read this blog. Hope it was able to make some good contributions to your knowledge base.

References

  1. https://github.com/felHR85/UsbSerial
  2. https://www.arduino.cc/reference/en/language/functions/communication/serial/

Tags: FOSSASIA, Neurolab, GSOC19, Open-source, Arduino, Serial terminal

Continue ReadingNeurolab data transfer – Establishing serial communication between Arduino and Android

Workflow of Admin Translations downloads as ZIP

This blog post emphasizes on the feature of translation archiving and download routes. The Open Event Project, popularly known as Eventyay is an event management solution which provides a robust platform to manage events, schedules multi-track sessions and supports many other features.

Prior to the actual implementation of this feature, a blueprint had to be registered in the flask app which handles the archiving and downloads logic for the translations. In addition, as this is only specific for administrators, the route had to be secured with access control. The access is controlled with the help of the is_admin decorator which checks if the user attempting to access the route is an admin. 

from flask import send_file, Blueprint
import shutil
import uuid
import tempfile
import os
from app.api.helpers.permissions import is_admin

admin_blueprint = Blueprint('admin_blueprint', __name__, url_prefix='/v1/admin/content/translations/all')
temp_dir = tempfile.gettempdir()
translations_dir = 'app/translations'

@admin_blueprint.route('/', methods=['GET'])
@is_admin
def download_translations():
    """Admin Translations Downloads"""
    uuid_literal = uuid.uuid4()
    zip_file = "translations{}".format(uuid_literal)
    zip_file_ext = zip_file+'.zip'
    shutil.make_archive(zip_file, "zip", translations_dir)
    shutil.move(zip_file_ext, temp_dir)
    path_to_zip = os.path.join(temp_dir, zip_file_ext)
    from .helpers.tasks import delete_translations
    delete_translations.apply_async(kwargs={'zip_file_path': path_to_zip}, countdown=600)
    return send_file(path_to_zip,  mimetype='application/zip',
                     as_attachment=True,
                     attachment_filename='translations.zip')

                                                         Code Snippet – Translations archiving

We utilize the shutil library to create archives of the specified directory which is the translations directory here. The directory file path which is to be archived must point to the folder with sub-directories of various language translations. We append a unique name generated by the UUID(Universally Unique Identifier) to the generated translations zip file. Also observe that the translations archive is saved to the temp folder. This is because the zip archive needs to be deleted after a certain amount of time. This time window mustn’t be too bounded as there might be multiple admins who might want to access these translations at the same time. Therefore, it is saved in the temp folder for 10 mins.

import Controller from '@ember/controller';
import { action } from '@ember/object';

export default class extends Controller {
  isLoading = false;

  @action
  async translationsDownload() {
    this.set('isLoading', true);
    try {
      const result = this.loader.downloadFile('/admin/content/translations/all/');
      const anchor = document.createElement('a');
      anchor.style.display = 'none';
      anchor.href = URL.createObjectURL(new Blob([result], { type: 'octet/stream' }));
      anchor.download = 'Translations.zip';
      anchor.click();
      this.notify.success(this.l10n.t('Translations Zip generated successfully.'));
    } catch (e) {
      console.warn(e);
      this.notify.error(this.l10n.t('Unexpected error occurred.'));
    }
    this.set('isLoading', false);
  }
}

                                           Code Snippet – Frontend logic for anchor downloads

To make this work on all browsers, the frontend part handles the downloads via an action in the controller. The action translationsDownload is linked to the download button which uses the download method in the loader service to hit the specific route and fetch the resource intuitively. This is stored in a constant result.

After this, an HTML anchor is created explicitly and its attribute values are updated.

The href of the anchor is set to the result of the resource fetched using the loader service with a Blob.

 Then a click action is triggered to download this file. As the loader service returns a Promise, we use an async function as the action to resolve it. In cases of failures, the notify service raises the error related and the frontend triggers this notification.

Resources:

Related work and code repo:

Tags:

Eventyay, FOSSASIA, Flask, Ember.js, Open Event, API

Continue ReadingWorkflow of Admin Translations downloads as ZIP