Uploading Badges To Google Cloud Badgeyay

Badgeyay is an open source project developed by FOSSASIA community. This project mainly aims for generating badges for technical conferences and events.The project is divided into two parts mainly. Backend is developed in flask and frontend is developed in emberjs. The problem is after the badge generation, the flask server is storing and serving those files. In practise this is not a good convention to do so. This should be handled by secondary hosting server program like gunicorn or nginx. Better approach would be to consume the firebase storage and admin sdk for storing the badges on google cloud. This will also offload storage needs from the flask server and also give a public link over the network to access.

Procedure

  1. Get the file path of the temporary badge generated on the flask server. Currently badges are saved in the directory of the image file uploaded and final badge generated is written in all-badges.pdf`
badgePath = os.getcwd() + ‘/static/temporary/’ + badgeFolder
badgePath + ‘/all-badges.pdf’

 

  1. Create the blob path for the storage. Blob can be understood as the final reference to the location where the contents are saved onto the server. This can be a nested directory structure or simply a filename in root directory.
‘badges/’ + badge_created.id + ‘.pdf’

 

In our case it is the id of the badge that is generated in the badges directory.

  1. Function for uploading the file generated in temporary storage to google cloud storage.
def fileUploader(file_path, blob_path):
  bucket = storage.bucket()
  fileUploaderBlob = bucket.blob(blob_path)
  try:
      with open(file_path, ‘rb’) as file_:
          fileUploaderBlob.upload_from_file(file_)
  except Exception as e:
      print(e)
  fileUploaderBlob.make_public()
  return fileUploaderBlob.public_url

 

It creates a bucket using the firebase admin SDK and then open the file from the file path. After opening the file from the path it writes the data to the cloud storage. After the data is written, the blob is made public and the public access link to the blob is fetched, which then later returned and saved in the local database.

Topics Involved

  • Firebase admin sdk for storage
  • Google cloud storage sdk

Resources

  • Firebase admin sdk documentation – Link
  • Google Cloud Storage SDK Python – Link
  • Blob Management – Link

 

Continue ReadingUploading Badges To Google Cloud Badgeyay

Ember Controller for Badge Generation In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community. This project aims towards giving a platform for badge generation using several customizations options. Current structure of project is in two parts to maintain modularity, which are namely backend, developed in flask, and frontend, developed in ember.

After refactoring the frontend and backend API we need to create a controller for the badge generation in frontend. Controller will help the components to send and receive data from them and prepare the logic for sending request to API so that badges can be generated and can receive the result as response from the server. Particularly we need to create the controller for badge generation route, create-badges.

As there are many customizations option presented to user, we need to chain the requests so that they sync with each other and the logic should not break for the badge generation.

Procedure

  1. Creating the controller from the ember-cli
ember g controller create-badge

 

  1. After the component generation, we need to create actions that can be passed to components. Let’s build action to submit form and then chain the different actions together for the badge generation.
submitForm() {
  const _this = this;
  const user = _this.get(‘store’).peekAll(‘user’);
  let uid;
  user.forEach(user_ => {
    uid = user_.get(‘id’);
  });
  if (uid !== undefined && uid !== ) {
    _this.set(‘uid’, uid);
  }

  let badgeData = {
    uid     : _this.uid,
    badge_size : ‘A3’
  };

  if (_this.csvEnable) {
    badgeData.csv = _this.csvFile;
  }
  if (_this.defFontColor !== && _this.defFontColor !== undefined) {
    badgeData.font_color = ‘#’ + _this.defFontColor;
  }
  if (_this.defFontSize !== && _this.defFontSize !== undefined) {
    badgeData.font_size = _this.defFontSize.toString();
  }
  if (_this.defFont !== && _this.defFont !== undefined) {
    badgeData.font_type = _this.defFont;
  }

  _this.send(‘sendManualData’, badgeData);

},

 

  1. As we can see in the above code snippet that _this.send(action_name, arguments) is calling another action sendManualData. This action then sends a network request to the backend if the Manual data is selected as input source otherwise will go with the CSV upload. If no option is chosen then it will show an error on the user screen, notifying him to select one input source.
sendManualData(badgeData) {
    const _this = this;
    if (_this.manualEnable) {
      let textEntry = _this.get(‘store’).createRecord(‘text-data’, {
        uid   : _this.uid,
        manual_data : _this.get(‘textData’),
        time   : new Date()
      });
      textEntry.save().then(record => {
        _this.set(‘csvFile’, record.filename);
        badgeData.csv = _this.csvFile;
        _this.send(‘sendDefaultImg’, badgeData);
        _this.get(‘notify’).success(‘Text saved Successfully’);
      }).catch(err => {
        let userErrors = textEntry.get(‘errors.user’);
        if (userErrors !== undefined) {
          _this.set(‘userError’, userErrors);
        }
      });
    } else if (_this.csvEnable) {
      if (_this.csvFile !== undefined && _this.csvFile !== ) {
        badgeData.csv = _this.csvFile;
        _this.send(‘sendDefaultImg’, badgeData);
      }
    } else {
      // No Input Source specified Error
    }
  },

 

The above code will choose the manual data if the manual data boolean flag is set else not, and then does a network request and wait for the promise to be resolved. As soon as the promise is resolved it calls another action to for the default image.

  1. After selecting the input source, now the background for the badge has to be selected. It will look for the boolean flags of the defaultImage, backgroundColorImage, customImage and will make the network request accordingly.
sendDefaultImg(badgeData) {
    const _this = this;
    if (_this.defImage) {
      let imageRecord = _this.get(‘store’).createRecord(‘def-image-upload’, {
        uid    : _this.uid,
        defaultImage : _this.defImageName
      });
      imageRecord.save()
        .then(record => {
          _this.set(‘custImgFile’, record.filename);
          badgeData.image = _this.custImgFile;
          _this.send(‘sendBadge’, badgeData);
        })
        .catch(error => {
          let userErrors = imageRecord.get(‘errors.user’);
          if (userErrors !== undefined) {
            _this.set(‘userError’, userErrors);
          }
        });
    } else if (_this.custImage) {
      if (_this.custImgFile !== undefined && _this.custImgFile !== ) {
        badgeData.image = _this.custImgFile;
        _this.send(‘sendBadge’, badgeData);
      }
    } else if (_this.colorImage && _this.defColor !== undefined && _this.defColor !== ) {
      console.log(_this.defColor);
      let imageRecord = _this.get(‘store’).createRecord(‘bg-color’, {
        uid : _this.uid,
        bg_color : _this.defColor
      });
      imageRecord.save()
        .then(record => {
          badgeData.image = record.filename;
          _this.send(‘sendBadge’, badgeData);
        })
        .catch(error => {
          let userErrors = imageRecord.get(‘errors.user’);
          if (userErrors !== undefined) {
            _this.set(‘userError’, userErrors);
          }
        });
    } else {
      // Inflate error for No Image source.
    }
  },

 

After the promise resolvement, the final action is called to send badge data payload to backend api for badge generation.

  1. After the complete preparation of the payload, now it’s time to send the payload to the backend api for the badge generation and after the promise resolvement showing the respective downloadable link in the frontend.
sendBadge(badgeData) {
    const _this = this;
    let badgeRecord = _this.get(‘store’).createRecord(‘badge’, badgeData);
    badgeRecord.save()
      .then(record => {
        _this.set(‘badgeGenerated’, true);
        _this.set(‘genBadge’, record.id);
        _this.get(‘notify’).success(‘Badge generated Successfully’);
      })
      .catch(err => {
        console.error(err.message);
      });
  },

 

Now after the promise resolvement the local variable badgGenerated is set to true so that the success message can be shown in the frontend for successful badge generation along with the link.

Link to respective PR – Link

Topics Involved

  • Chaining of actions and requests
  • Manipulating DOM on the conditional statements
  • Component bindings
  • Ember data
  • Promise resolvement

Resources

  • Link to ember data for the API requests and promise resolvement – Link
  • Implementing Controllers in Ember – Link
  • Chaining actions together in ember – Link
Continue ReadingEmber Controller for Badge Generation In Badgeyay

Integrating Firebase Cloud Functions In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community for generating badges for conferences and events. The Project is divided into two parts frontend, which is in ember, and backend, which is in flask. Backend uses firebase admin SDK (Python) and Frontend uses firebase javascript client with emberfire wrapper for ember. Whenever an user signs up on the website, database listener that is attached to to the Model gets triggered and uses flask-mail for sending welcome mail to the user and in case of email and password signup, verification mail as well.

Problem is sending mail using libraries is a synchronous process and takes a lot of processing on the server. We can use messaging queues like RabbitMQ and Redis but that will be burden as server cost will increase. The workaround is to remove the code from the server and create a firebase cloud function for the same task.

Firebase cloud functions lets you run backend code on the cloud and can be triggered with HTTP events or listen for the events on the cloud, like user registration.

Procedure

  1. Firebase uses our Gmail ID for login, so make sure to have a Gmail ID and on the first sight we will be greeted with Firebase console, where we can see our created or imported firebase apps.

  2. Create the app by clicking on the Add Project Icon and write the name of the application (e.g. Test Application) and select the region, in my case it is India. Firebase will automatically generated an application ID for the app. Click on Create Project to complete creation of project

  3. After Completion, click on the project to enter into the project. You will be greeted with an overview saying to integrate firebase with your project. We will click on the Add Firebase to web App and save the config as JSON in a file as clientKey.json for later use.

  4. Now we need to install the firebase tools on our local machine so for that execute
    npm i -g firebase-tools
    1. Now login from the CLI so that firebase gets token for the Gmail ID of the user and can access the firebase account of that Gmail ID.
    firebase login
    1. After giving permissions to the firebase CLI from your Gmail account in the new tab opened in browser, create a folder named cloud_functions in the project directory and in that execute
    firebase init
    1. Select only functions from the list of options by pressing space.

    2. After this select the project from the list where you want to use the cloud function. You can skip the step if you later want to add the cloud function to project by selecting don’t setup a default project and can later be used by command
      firebase use –add

    3. Choose the language of choice

    4. If you want, you can enforce eslint on the project and after this the cloud function is set up and the directory structure looks as follows.

    5. We will write our cloud function in index.js. So let’s take a look at index.js
      const functions = require(‘firebase-functions’);

      // // Create and Deploy Your First Cloud Functions
      // // https://firebase.google.com/docs/functions/write-firebase-functions
      //
      // exports.helloWorld = functions.https.onRequest((request, response) => {
      //  response.send(“Hello from Firebase!”);
      // });

      As we can see there is a sample function already given, we don’t need that sample function so we will remove it and will write the logic for sending mail. Before that we need to acquire the key for service accounts so that admin functionality can be accessed in the cloud function. So for that go to project settings and then service accounts and click on Generate New Private Key  and save it as serviceKey.json

    6. Now the directory structure will look like this after adding the clientKey.json and serviceKey.json

    7. We will use node-mailer for sending mails in cloud functions and as there is a limitation on the gmail account to send only 500 mails in a day, we can use third party services like sendGrid and others for sending mails with firebase. Configure node-mailer for sending mails as
      const nodemailer = require(‘nodemailer’);

      const gmailEmail = functions.config().gmail.email;
      const gmailPassword = functions.config().gmail.password;
      const mailTransport = nodemailer.createTransport({
       service: ‘gmail’,
       auth: {
      user: gmailEmail,
      pass: gmailPassword
       }
      });

      Also set the environment variables for the cloud functions like email and password:

      firebase functions:config:set gmail.email=“Email ID” gmail.password=“Password”
      1. Logic for sending Greeting Mail on user registration
      exports.greetingMail = functions.auth.user().onCreate((user) => {
       const email = user.email;
       const displayName = user.displayName;

       return sendGreetingMail(email, displayName);
      });

      function sendGreetingMail(email, displayName) {
       const mailOptions = {
      from: `${APP_NAME}<noreply@firebase.com>`,
      to: email,
       };

       mailOptions.subject = `Welcome to Badgeyay`;
       mailOptions.text = `Hey ${displayName || ”}! Welcome to Badgeyay. We welcome you onboard and pleased to offer you service.`;
       return mailTransport.sendMail(mailOptions).then(() => {
      return console.log(‘Welcome mail sent to: ‘, email)
       }).catch((err) => {
      console.error(err.message);
       });
      }

      Function will get triggered on creation of user in firebase and calls the greeting mail function with parameters as the email id of the registered user and the Display name. Then a default template is used to send mail to the recipient and Logged on successful submission.

      1. Currently firebase admin sdk doesn’t support the functionality to send verification mail but the client SDK does. So the approach which is followed in badgeyay is that admin SDK will create a custom token and client sdk will use that custom token to sign in and them send verification mail to the user.
      exports.sendVerificationMail = functions.auth.user().onCreate((user) => {
       const uid = user.uid;
       if (user.emailVerified) {
      console.log(‘User has email already verified: ‘, user.email);
      return 0;
       } else {
      return admin.auth().createCustomToken(uid)
        .then((customToken) => {
          return firebase.auth().signInWithCustomToken(customToken)
        })
        .then((curUser) => {
          return firebase.auth().onAuthStateChanged((user_) => {
            if (!user.emailVerified) {
              user_.sendEmailVerification();
              return console.log(‘Verification mail sent: ‘, user_.email);
            } else {
              return console.log(‘Email is already verified: ‘, user_.email);
            }
          })
        })
        .catch((err) => {
          console.error(err.message);
        })
       }
      });
      1. Now we need to deploy the functions to firebase.
      firebase deploy –only functions

      Link to the respective PR  : Link

      Topics Involved

      • Firebase Admin SDK
      • Configuring Gmail for third party apps
      • Token Verification and verification mail by client SDK
      • Nodemailer and Express.js

      Resources

      • Firebase Cloud functions – Link
      • Extending authentication with cloud function – Link
      • Custom Token Verification – Link
      • Nodemailer message configuration – Link
      • Issue discussion on sending verification mail with admin SDK – Link
Continue ReadingIntegrating Firebase Cloud Functions In Badgeyay

Adding Online Payment Support in Open Event Frontend via PayPal

Open Event Frontend involves ticketing system which supports both paid and free tickets. To buy a paid ticket Open Event provides several options such as debit card, credit card, cheque, bank transfer and onsite payments. So to add support for debit and credit card payments Open Event uses Paypal checkout as one of the options. Using paypal checkout screen users can enter their card details and pay for their ticket or they can use their paypal wallet money to pay for their tickets.

Given below are some steps which are to be followed for successfully charging a user for ticket using his/her card.

  • We create an application on paypal developer dashboard to receive client id and secret key.
  • We set these keys in admin dashboard of open event and then while checkout we use these keys to render checkout screen.
  • After clicking checkout button a request is sent to create-paypal-payment endpoint of open event server to create a paypal token which is used in checkout procedure.
  • After user’s verification paypal generates a payment id is which is used by open event frontend to charge the user for stipulated amount.
  • We send this token to open event server which processes the token and charge the user.
  • We get error or success message from open event server as per the process outcome.

To render the paypal checkout elements we use paypal checkout library provided by npm. Paypal button is rendered using Button.render method of paypal checkout library. Code snippet is given below.

// app/components/paypal-button.js

paypal.Button.render({
   env: 'sandbox',

   commit: true,

   style: {
     label : 'pay',
     size  : 'medium', // tiny, small, medium
     color : 'gold', // orange, blue, silver
     shape : 'pill'    // pill, rect
   },

   payment() {
   // this is used to obtain paypal token to initialize payment process 
   },

   onAuthorize(data) {
     // this callback will be for authorizing the payments
    }

 }, this.elementId);

 

After button is rendered next step is to obtain a payment token from create-paypal-payment endpoint of open event server. For this we use the payment() callback of paypal-checkout. Code snippet for payment callback method is given below:

// app/components/paypal-button.js

let createPayload = {
      'data': {
        'attributes': {
          'return-url' : `${window.location.origin}/orders/${order.identifier}/placed`,
          'cancel-url' : `${window.location.origin}/orders/${order.identifier}/placed`
        },
        'type': 'paypal-payment'
      }
    };
paypal.Button.render({
     //Button attributes

      payment() {
        return loader.post(`orders/${order.identifier}/create-paypal-payment`, createPayload)
          .then(res => {
            return res.payment_id;
          });
      },

      onAuthorize(data) {
        // this callback will be for authorizing the payments
      }

    }, this.elementId);

 

After getting the token payment screen is initialized and user is asked to enter his/her credentials. This process is handled by paypal servers. After user verifies his/her payment paypal generates a paymentId and a payerId and sends it back to open event. After the payment authorization onAuthorize() method of paypal is called and payment is further processed in this callback method. Payment ID and payer Id received from paypal is sent to charge endpoint of open event server to charge the user. After receiving success or failure message from paypal proper message is displayed to users and their order is confirmed or cancelled respectively. Code snippet for onAuthorize is given below:

// app/components/paypal-button.js

onAuthorize(data) {
        // this callback will be for authorizing the payments
        let chargePayload = {
          'data': {
            'attributes': {
              'stripe'            : null,
              'paypal_payer_id'   : data.payerID,
              'paypal_payment_id' : data.paymentID
            },
            'type': 'charge'
          }
        };
        let config = {
          skipDataTransform: true
        };
        chargePayload = JSON.stringify(chargePayload);
        return loader.post(`orders/${order.identifier}/charge`, chargePayload, config)
          .then(charge => {
            if (charge.data.attributes.status) {
              notify.success(charge.data.attributes.message);
              router.transitionTo('orders.view', order.identifier);
            } else {
              notify.error(charge.data.attributes.message);
            }
          });
      }

 

Full code can be seen here.

In this way we achieve the functionality of adding paypal payment support in open event frontend. Please follow the links below for further clarification and detailed overview.

Resources:

Continue ReadingAdding Online Payment Support in Open Event Frontend via PayPal

Open Event Frontend – Settings Service

This blog illustrates how the settings of a particular user are obtained in the Open Event Frontend web app. To access the settings of the user a service has been created which fetches the settings from the endpoint provided by Open Event Server.

Let’s see why a special service was created for this.

Problem

In the first step of the event creation wizard, the user has the option to link Paypal or Stripe to accept payments. The option to accept payment through Paypal or Stripe was shown to the user without checking if it was enabled by the admin in his settings. To solve this problem, we needed to access the settings of the admin and check for the required conditions. But since queryRecord() returned a promise we had to re-render the page for the effect to show which resulted in this code:

canAcceptPayPal: computed('data.event.paymentCurrency', function() {     this.get('store').queryRecord('setting', {}) .then(setting => { this.set('canAcceptPayPal', (setting.paypalSandboxUsername || setting.paypalLiveUsername) && find(paymentCurrencies, ['code', this.get('data.event.paymentCurrency')]).paypal); this.rerender(); });

This code was setting a computed property inside it and then re-rendering which is bad programming and can result in weird bugs.

Solution

The above problem was solved by creating a service for settings. This made sense as settings would be required at other places as well. The file was called settings.js and was placed in the services folder. Let me walk you through its code.

  • Extend the default Service provided by Ember.js and initialize store, session, authManager and _lastPromise.
import Service, { inject as service } from '@ember/service';
import { observer } from '@ember/object';

export default Service.extend({

 store       : service(),
 session     : service(),
 authManager : service(),

_lastPromise: Promise.resolve(),
  • The main method which fetches results from the server is called _loadSettings(). It is an async method. It queries setting from the server and then iterates through every attribute of the setting model and stores the corresponding value from the fetched result.
/**
* Load the settings from the API and set the attributes as properties on the service
*
* @return {Promise<void>}
* @private
*/
async _loadSettings() {
 const settingsModel = await this.get('store').queryRecord('setting', {});
 this.get('store').modelFor('setting').eachAttribute(attributeName => {
   this.set(attributeName, settingsModel.get(attributeName));
 });
},
  • The initialization of the settings service is handled by initialize(). This method returns a promise.
/**
* Initialize the settings service
* @return {*|Promise<void>}
*/
initialize() {
 const promise = this._loadSettings();
 this.set('_lastPromise', promise);
 return promise;
}
  • _authenticationObserver observes for changes in authentication changes and reloads the settings as required.
/**
* Reload settings when the authentication state changes.
*/
_authenticationObserver: observer('session.isAuthenticated', function() {
 this.get('_lastPromise')
   .then(() => this.set('_lastPromise', this._loadSettings()))
   .catch(() => this.set('_lastPromise', this._loadSettings()));
}),

The service we created can be directly used in the app to fetch the settings for the user. To solve the Paypal and Stripe payment problem described above, we use it as follows:

canAcceptPayPal: computed('data.event.paymentCurrency', 'settings.paypalSandboxUsername', 'settings.paypalLiveUsername', function() {
 return (this.get('settings.paypalSandboxUsername') || this.get('settings.paypalLiveUsername')) && find(paymentCurrencies, ['code', this.get('data.event.paymentCurrency')]).paypal;
}),

canAcceptStripe: computed('data.event.paymentCurrency', 'settings.stripeClientId', function() {
 return this.get('settings.stripeClientId') && find(paymentCurrencies, ['code', this.get('data.event.paymentCurrency')]).stripe;
}),

Thus, there is no need to re-render the page and dangerously set the property inside its computed method.

References

Continue ReadingOpen Event Frontend – Settings Service

Open Event Server – Export Speakers as CSV File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the speakers in a very detailed view in the event management dashboard. He can see the statuses of all the speakers. The possible statuses are pending, accepted and rejected. He/she can take actions such as editing/viewing speakers.

If the organizer wants to download the list of all the speakers as a CSV file, he or she can do it very easily by simply clicking on the Export As CSV button in the top right-hand corner.

Let us see how this is done on the server.

Server side – generating the Speakers CSV file

Here we will be using the csv package provided by python for writing the csv file.

import csv
  • We define a method export_speakers_csv which takes the speakers to be exported as a CSV file as the argument.
  • Next, we define the headers of the CSV file. It is the first row of the CSV file.
def export_speakers_csv(speakers):
   headers = ['Speaker Name', 'Speaker Email', 'Speaker Session(s)',
              'Speaker Mobile', 'Speaker Bio', 'Speaker Organisation', 'Speaker Position']
  • A list is defined called rows. This contains the rows of the CSV file. As mentioned earlier, headers is the first row.
rows = [headers]
  • We iterate over each speaker in speakers and form a row for that speaker by separating the values of each of the columns by a comma. Here, every row is one speaker.
  • As a speaker can contain multiple sessions we iterate over each session for that particular speaker and append each session to a string. ‘;’ is used as a delimiter. This string is then added to the row. We also include the state of the session – accepted, rejected, confirmed.
  • The newly formed row is added to the rows list.
for speaker in speakers:
   column = [speaker.name if speaker.name else '', speaker.email if speaker.email else '']
   if speaker.sessions:
       session_details = ''
       for session in speaker.sessions:
           if not session.deleted_at:
               session_details += session.title + ' (' + session.state + '); '
       column.append(session_details[:-2])
   else:
       column.append('')
   column.append(speaker.mobile if speaker.mobile else '')
   column.append(speaker.short_biography if speaker.short_biography else '')
   column.append(speaker.organisation if speaker.organisation else '')
   column.append(speaker.position if speaker.position else '')
   rows.append(column)
  • rows contains the contents of the CSV file and hence it is returned.
return rows
  • We iterate over each item of rows and write it to the CSV file using the methods provided by the csv package.
with open(file_path, "w") as temp_file:
   writer = csv.writer(temp_file)
   from app.api.helpers.csv_jobs_util import export_speakers_csv
   content = export_speakers_csv(speakers)
   for row in content:
       writer.writerow(row)

Obtaining the Speakers CSV file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/speakers/csv

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the speakers of the event as a CSV file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Resources

Continue ReadingOpen Event Server – Export Speakers as CSV File

Open Event Server – Export Sessions as CSV File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the sessions in a very detailed view in the event management dashboard. He can see the statuses of all the sessions. The possible statuses are pending, accepted, confirmed and rejected. He/she can take actions such as accepting/rejecting the sessions.

If the organizer wants to download the list of all the sessions as a CSV file, he or she can do it very easily by simply clicking on the Export As CSV button in the top right-hand corner.

Let us see how this is done on the server.

Server side – generating the Sessions CSV file

Here we will be using the csv package provided by python for writing the csv file.

import csv
  • We define a method export_sessions_csv which takes the sessions to be exported as a CSV file as the argument.
  • Next, we define the headers of the CSV file. It is the first row of the CSV file.
def export_sessions_csv(sessions):
   headers = ['Session Title', 'Session Speakers',
              'Session Track', 'Session Abstract', 'Created At', 'Email Sent']
  • A list is defined called rows. This contains the rows of the CSV file. As mentioned earlier, headers is the first row.
rows = [headers]
  • We iterate over each session in sessions and form a row for that session by separating the values of each of the columns by a comma. Here, every row is one session.
  • As a session can contain multiple speakers we iterate over each speaker for that particular session and append each speaker to a string. ‘;’ is used as a delimiter. This string is then added to the row.
  • The newly formed row is added to the rows list.
for session in sessions:
   if not session.deleted_at:
       column = [session.title + ' (' + session.state + ')' if session.title else '']
       if session.speakers:
           in_session = ''
           for speaker in session.speakers:
               if speaker.name:
                   in_session += (speaker.name + '; ')
           column.append(in_session[:-2])
       else:
           column.append('')
       column.append(session.track.name if session.track and session.track.name else '')
       column.append(strip_tags(session.short_abstract) if session.short_abstract else '')
       column.append(session.created_at if session.created_at else '')
       column.append('Yes' if session.is_mail_sent else 'No')
       rows.append(column)
  • rows contains the contents of the CSV file and hence it is returned.
return rows
  • We iterate over each item of rows and write it to the CSV file using the methods provided by the csv package.
writer = csv.writer(temp_file)
from app.api.helpers.csv_jobs_util import export_sessions_csv
content = export_sessions_csv(sessions)
for row in content:
   writer.writerow(row)

Obtaining the Sessions CSV file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/sessions/csv

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the sessions of the event as a CSV file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Resources

Continue ReadingOpen Event Server – Export Sessions as CSV File

Retrofit2 Rxjava2 Error Response Handling in Open Event Organizer App

In the Open Event Organizer Android app the challenge is to provide user, a readable error description for the input requests. The Organizer App was showing a coded message which was understandable only to a programmer making it unfriendly to the common user. The blog describes how we tackled this problem and implemented a mechanism to provide user friendly error messages for user requests (if any).

Let’s consider the scenario when an organizer want to create an Event or maybe a Ticket so what he has to do is fill out a form containing some user input fields and click on submit button to send the data to the server which in turn sends a response for the request. If the information is valid the server sends a HTTP 201 Response and user gets feedback that the Event/Ticket is created. But if the information is not valid the user gets feedback that there is HTTP 422 error in your request. There is no readable description of error provided to the user.

To handle this problem we will be using Retrofit 2.3.0 to make Network Requests, which is a REST client for Android and Java by Square Inc. It makes it relatively easy to retrieve and upload JSON (or other structured data) via a REST based Web Service. Further, we will be using other awesome libraries like RxJava 2.1.10 (by ReactiveX) to handle tasks asynchronously, Jackson, Jasminb-Json-Api in an MVP architecture. Let’s move on to the code.

Retrofit to the rescue

Now we will see how we can extract the details about the error response and provide user a better feedback rather than just throwing the error code.

Firstly let’s make the Request to our REST API Server using Retrofit2.

@POST(“faqs”)
Observable<Faq> postFaq(@Body Faq faq);

Here we are making a POST request and expecting a Observable<Faq> Response from the server.The  @Body annotation indicates the Request Body is an Faq object.

Please note that Observable used here is ReactiveX Observable so don’t confuse it with java.util Observable.

public class Faq {
@Id(LongIdHandler.class)
public Long id;  @Relationship(“event”)
@ForeignKey(stubbedRelationship = true,

onDelete = ForeignKeyAction.CASCADE)
public Event event;

public String question;
public String answer;
}

Let’s say the API declares both question and answer as mandatory fields  for creation of an Faq. We supply the following input to the app.

question = “Do I need to buy a Ticket to get In ?”;
answer = null

We used RxJava to make an asynchronous request to server. In case of error we will get a Retrofit throwable and we will pass that on to ErrorUtils.java which will do all the processing and return a readable error message.

faqRepository
.createFaq(faq)
.compose(dispose(getDisposable()))
.compose(progressive(getView()))  .doOnError(throwable ->

getView().showError(ErrorUtils.getMessage(throwable)))
.subscribe(createdFaq -> {
getView().onSuccess(“Faq Created”);
getView().dismiss();
}, Logger::logError);

Now we will extract the ResponseBody from the Retrofit throwable.

ResponseBody responseBody = ((HttpException) throwable)

.response().errorBody();
return getErrorDetails(responseBody);

The ResponseBody is a JSON containing all the information about the error.

{
“errors”: [
{
“status”: “422”,
“source”: {
“pointer”: “/data/attributes/answer”
},
“detail”: “Missing data for required field.”,
“title”: “Validation error”
}
],
“jsonapi”: {
“version”: “1.0”
}
}

In order to provide better feedback to user regarding the error we have to parse the response JSON and extract the pointed field (which in this case is – “answer”) and then combine it with the value corresponding to the “detail” key. Following is the relevant section from ErrorUtils.java (for viewing the complete ErrorUtils.java please refer here).

public static String getErrorDetails(ResponseBody responseBody) {
try {
JSONObject jsonObject = new JSONObject(responseBody.string());
JSONObject jsonArray = new    JSONObject(jsonObject.getJSONArray(“errors”).get(0).toString());
JSONObject errorSource =

new JSONObject(jsonArray.get(“source”).toString());

try {
String pointedField =

getPointedField(errorSource.getString(“pointer”));
if (pointedField == null)
return jsonArray.get(“detail”).toString();
else
return jsonArray.get(“detail”).toString()

.replace(“.”, “”) + “: ” + pointedField;
} catch (Exception e) {
return jsonArray.get(“detail”).toString();
}

} catch (Exception e) {
return null;
}
}

public static String getPointedField(String pointerString) {
if (pointerString == null || Utils.isEmpty(pointerString))
return null;
else {
String[] path = pointerString.split(“/”);
if (path.length > 3)
return path[path.length – 1];
else
return null;
}
}

The final error message returned by ErrorUtils in case of HTTP 422 –

Missing data for required field: answer

Similar approach can be followed for extracting the error “title” or “status” .

References

  1. Official documentation of Retrofit 2.x http://square.github.io/retrofit/
  2. Official documentation for RxJava 2.x https://github.com/ReactiveX/RxJava
  3. Codebase for Open Event Organizer App on Github https://github.com/fossasia/open-event-orga-app
Continue ReadingRetrofit2 Rxjava2 Error Response Handling in Open Event Organizer App

Implementing Endpoint to Resend Email Verification

Earlier, when a user registered via Open Event Frontend, s/he received a verification link via email to confirm their account. However, this was not enough in the long-term. If the confirmation link expired, or for some reasons the verification mail got deleted on the user side, there was no functionality to resend the verification email, which prevented the user from getting fully registered. Although the front-end already showed the option to resend the verification link, there was no support from the server to do that, yet.

So it was decided that a separate endpoint should be implemented to allow re-sending the verification link to a user. /resend-verification-email was an endpoint that would fit this action. So we decided to go with it and create a route in `auth.py` file, which was the appropriate place for this feature to reside. First step was to do the necessary imports and then definition:

from app.api.helpers.mail import send_email_confirmation
from app.models.mail import USER_REGISTER_WITH_PASSWORD
...
...
@auth_routes.route('/resend-verification-email', methods=['POST'])
def resend_verification_email():
...

Now we safely fetch the email mentioned in the request and then search the database for the user corresponding to that email:

def resend_verification_email():
    try:
        email = request.json['data']['email']
    except TypeError:
        return BadRequestError({'source': ''}, 'Bad Request Error').respond()

    try:
        user = User.query.filter_by(email=email).one()
    except NoResultFound:
        return UnprocessableEntityError(
{'source': ''}, 'User with email: ' + email + ' not found.').respond()
    else:

    ...

Once a user has been identified in the database, we proceed further and create an essentially unique hash for the user verification. This hash is in turn used to generate a verification link that is then ready to be sent via email to the user:

else:
    serializer = get_serializer()
    hash_ = str(base64.b64encode(str(serializer.dumps(
[user.email, str_generator()])).encode()), 'utf-8')
    link = make_frontend_url(
'/email/verify'.format(id=user.id), {'token': hash_})

Finally, the email is sent:

send_email_with_action(
user, USER_REGISTER_WITH_PASSWORD,
app_name=get_settings()['app_name'], email=user.email)
    if not send_email_confirmation(user.email, link):
        return make_response(jsonify(message="Some error occured"), 500)
    return make_response(jsonify(message="Verification email resent"), 200)

But this was not enough. When the endpoint was tested, it was found that actual emails were not being delivered, even after correctly configuring the email settings locally. So, after a bit of debugging, it was found that the settings, which were using Sendgrid to send emails, were using a deprecated Sendgrid API endpoint. A separate email function is used to send emails via Sendgrid and it contained an old endpoint that was no longer recommended by Sendgrid:

@celery.task(name='send.email.post')
def send_email_task(payload, headers):
   requests.post(
       "https://api.sendgrid.com/api/mail.send.json",
       data=payload,
       headers=headers
   )

The new endpoint, as per Sendgrid’s documentation, is:

https://api.sendgrid.com/v3/mail/send

But this was not the only change required. Sendgrid had also modified the structure of requests they accepted, and the new structure was different from the existing one that was used in the server. Following is the new structure:

'{"personalizations": [{"to": [{"email": "example@example.com"}]}],"from": {"email": "example@example.com"},"subject": "Hello, World!","content": [{"type": "text/plain", "value": "Heya!"}]}'

The header structure was also changed, so the structure in the server was also updated to

headers = {
"Authorization": ("Bearer " + key),
"Content-Type": "application/json"
}

The Sendgrid function (which is executed as a Celery task) was modified as follows, to incorporate the changes in the API endpoint and structure:

import json
...
@celery.task(name='send.email.post')
def send_email_task(payload, headers):
    data = {"personalizations": [{"to": []}]}
    data["personalizations"][0]["to"].append({"email": payload["to"]})
    data["from"] = {"email": payload["from"]}
    data["subject"] = payload["subject"]
    data["content"] = [{"type": "text/html", "value": payload["html"]}]
    requests.post(
        "https://api.sendgrid.com/v3/mail/send",
        data=json.dumps(data),
        headers=headers,
        verify=False  # doesn't work with verification in celery context
    )

 

As can be seen, there is a bug that doesn’t allow SSL verification within the celery context. However, the verification is successful when the functionality is executed independent of the celery context. But now email sending via Sendgrid actually works, which makes our verification resend endpoint functional:Screen Shot 2018-08-10 at 10.04.12 PM.pngEmail is received successfully by the recipient:

Screen Shot 2018-08-10 at 10.04.30 PM.png

Thus, a working email verification endpoint is implemented, which can be easily integrated in the frontend.


Resources:

Continue ReadingImplementing Endpoint to Resend Email Verification

Using Two-Way Data Binding in Open Event Organizer Android App:

Data Binding is the simple approach which relieves developers of repeated findViewById() calls. It is something that every developer must use if not using ButterKnife.

The Open Event Organizer Android App provides options to fill in an extensive set of details while creating an event, or any other entities. The problem at hand is that many of these options are common to many of these entities. For instance, currently the element date-time-picker and text fields are common to elements of different forms, as each one of them requires date-time checkboxes.

We need to be able to <include> a separate smaller and reusable layout file and bind event data to make the code shorter. This would help decreasing unnecessary code base and improving code readability.

We will see how using 2 way data binding and <include> tags in the small PR #929 reduced the code of 112 lines to just 9 lines:

Step 1: Configuration:

The very first step is to configure your project to enable data bindings in your
build.gradle (Module:app) file.

dataBinding should be included as follows:

android {
   // Rest of gradle file…
   dataBinding {
   enabled true
   }    // Rest of gradle file…
}

Step 2: Import and variable tags:

Data Binding uses the tag <data> to signify the data which will be referred to in lambda expressions inside the XML.

We also need to import any class, whose methods we need to use. This can be done using the <import> tag.

Finally, the <variable> tag is used to define any variables that will be referenced in the XML.

 

<data>
  <import type=”android.view.View” />
  <variable
      name=“date”
      type=”String” />
  <variable
      name=“label”
      type=”String“/>
</data>

Step 3: Binding the declared variables:

Data binding recognizes methods of the type set<variable>, where <variable> is event in our case.

We need to use  executePendingBindings();  so that any pending bindings are done and the UI of our app responds correctly as soon as the view data is updated.

@Override
public void showResult(Event event) {
  binding.setEvent(event);
  binding.executePendingBindings();
}

Step 4: Using the declared variables:

Making use of the declared variables is a very simple task and is as simple as a java statement. You can do almost everything that’s possible in the java file, the only constraint being that the used variables are declared in the xml and binded appropriately.

Most of the data binding expressions use data binding to condense the expression to its smallest possible form.

<LinearLayout
  android:layout_width=“match_parent”
  android:layout_height=“wrap_content”
  android:padding=“@dimen/spacing_extra_small”
  android:orientation=“horizontal”
  android:visibility=“@{ picker.checked ? View.VISIBLE : View.GONE }”>

2 Way Data Binding

In case of the Organizer App, we are using 2 way data binding.

Data Binding allows us to do much more than just set text in TextView or create listener in Button. If we want to use EditText and automatically update text variable in java code, we need to use observable fields and two way binding.

Thus, most variables like date, event that we are binding, are Observable fields.

* Sometimes there’s a use case of using a variable declared in another layout file.

For example, in:

<org.fossasia.openevent.app.ui.views.DatePicker
  style=“?attr/borderlessButtonStyle”
  android:layout_width=“wrap_content”
  android:layout_height=“wrap_content”
  android:textColor=“@color/purple_500”
  app:value=“@={ date }” />

The variable date isn’t binded in the java file but the xml files which include the layout time_picker.xml

Using the <include> tag:

The include tag is very simple to use, and we can simply bind the date and label element. The event_create_form.xml binds the variable using the bind attribute like this:

<include
  layout=“@layout/time_picker”
  bind:date=“@={ event.startsAt }”
  bind:label=“@{ @string/starts_at }”/>

The most common error you will face:

Often, when there’s something wrong with the XML, the most common error you will face is:

“Cannot resolve Data Binding class…”

This error is because Android Studio couldn’t generate the Data Binding class for your XML file because of some error. Presently, it doesn’t give much details about what’s wrong, so you’ll have to look for the errors yourselves.

The most common mistake newbie developers make is forgetting to bind the variables appropriately.

References:

Android Developer Guide:

https://developer.android.com/topic/libraries/data-binding/

Continue ReadingUsing Two-Way Data Binding in Open Event Organizer Android App: