Implement Sensor Data Fetching Using AsyncTask

In PSLab android app we have implemented sensor data fetching of various sensors inbuilt in the mobile device like light sensor, accelerometer, gyrometer. We can use PSLab to log the data and show in the form of the graph or maybe export the data in the form of CSV format for future use.

But recording data from the phone sensor imposes a serious problem in the performance of the Android app as it is a costly to process in terms of memory, resources and time. In CS terms there is too much work that has to be done on the single main thread which sometimes leads to lag and compromises the UX.

So as a solution we applied a concept of the Multithreading provided by Java in which we can shift the heavy process to a separate background thread so that the main thread never gets interrupted during fetching the sensor data and the background thread handles all the fetching and updates the UI as soon as it gets the data, till then the Main thread continues to serves the user so to user the application remains always responsive.

For implementing this we used a special class provided by Android Framework called AsyncTask. Which provides below methods:-

  • doInBackground() : This method contains the code which needs to be executed in the background. In this method, we can send results multiple times to the UI thread by publishProgress() method.

  • onPreExecute() : This method contains the code which is executed before the background processing starts.

  • onPostExecute() : This method is called after doInBackground method completes processing. Result from doInBackground is passed to this method.

  • onProgressUpdate() : This method receives progress updates from doInBackground() method, which is published via publishProgress() method, and this method can use this progress update to update the UI thread.

  • onCancelled(): This method is called when the background task has been canceled. Here we can free up resources or write some cleanup code to avoid memory leaks.

We created a class SensorDataFetch and extended this AsyncTask class and override its methods according to our needs.

private class SensorDataFetch extends AsyncTask<Void, Void, Void> implements SensorEventListener {

   private float data;
   private long timeElapsed;

   @Override
   protected Void doInBackground(Void... params) {
      
       sensorManager.registerListener(this, sensor, updatePeriod);
       return null;
   }

   protected void onPostExecute(Void aVoid) {
       super.onPostExecute(aVoid);
       visualizeData();
   }

   @Override
   protected void onPreExecute() {
 super.onPreExecute();
   //do nothing
   }

   @Override
   protected void onProgressUpdate(Void... values) {
       super.onProgressUpdate(values);
          //do nothing
   }

   @Override
   protected void onCancelled() {
       super.onCancelled();
          //do nothing
   }

In doInBackground() method we implemented the fetching raw data from the sensor by registering the listener and in onPostExecute() method we updated that data on the UI to be viewed by the user.

When this process is being run in the background thread the Main UI thread is free and remains responsive to the user. We can see in Figure 1 below that the UI is responsive to the user swipe action even when the sensor data is updating continuously on the screen.

Figure 1 shows Lux Meter responding to user swipe while fetching sensor data flawlessly.

 

Resources

https://developer.android.com/reference/android/os/AsyncTask – Android Developer documentation for Async Task class.

Continue ReadingImplement Sensor Data Fetching Using AsyncTask

Snackbar Implementation in PSLab Android App

In PSLab android app we have developed the functionality of logging sensor data in CSV format. We can start and stop the data recording using the save button in the upper right corner of the menu bar and toast message was shown to notify the user for logging status whether it is started or stopped but it leads to some problem like:-

  • The user doesn’t know where the logged file has been created in the external storage.
  • If the user accidentally clicked on the save button the data logging will start the user have to manually go the storage location and delete the recently created unwanted CSV file.

What’s the solution?

The solution to both these problem is solved by implementing Snackbar instead of Toast message.

According to Material Design documentation:-

The Snackbar widget provides brief feedback about an operation through a message at the bottom of the screen. Snackbar disappears automatically, either after a timeout or after a user interaction elsewhere on the screen, and can also be swiped off the screen.

Snackbar can also offer the ability to perform an action, such as undoing an action that was just taken or retrying an action that had failed.

 

Figure 1 shows a Snackbar sample
(Source: – https://material.io/develop/android/components/snackbar/ )

 

To implement the Snackbar in our Android app I started by creating a custom snack bar class which contains all the code to create and show the Snackbar on the screen.

public class CustomSnackBar {

   public static void showSnackBar(@NonNull CoordinatorLayout holderLayout,  
                                   @NonNull String displayText,
                                   String actionText, 
                                   View.OnClickListener clickListener){
       
   Snackbar snackbar =     
              Snackbar.make(holderLayout,displayText,Snackbar.LENGTH_LONG)
              .setAction(actionText, clickListener);

  //do your customization here
}

The custom class contains a static method ‘showSnackBar()’ having parameters:

Parameter Return Type Description
holderLayout CoordinatorLayout Container layout in which the snack bar will be shown at the bottom (should not be null)
displayText String Text to be displayed in the content of Snackbar (should not be null)
actionText String Clickable text which has some action associated with it
clickListener View.OnClickListener On click listener specifying an action to be performed when actionText is clicked

 

Inside the method, I called the static make()  method provided by the Snackbar class and passed holderlayout, displayText and duration of Snackbar in this case Snackbar.LENGTH_LONG as parameters.

Then I called setAction() and passed in the actionText and the clickListener as parameters in it to set the action text. If we pass in null no action text will be generated.

Then, if we want to changes the action text color we can do that by calling setActionTextColor() and passing in the desired color.

snackbar.setActionTextColor(ContextCompat.getColor(holderLayout.getContext(), R.color.colorPrimary));

And if we want to change the content text color then we need to first get the view then we need to get the instance of TextView containing the content text using findViewById() and passing android.support.design.R.id.snackbar_text which is default ID for context TextView, and then call setTextColor() to set the desired color.

View sbView = snackbar.getView();
   TextView textView =     
             sbView.findViewById(android.support.design.R.id.snackbar_text);
       textView.setTextColor(Color.WHITE);
   }

So, now our Snackbar engine is complete now we need to call CustomSnackBar class static method showSnackbar() in our sensor data logger.

For doing this I replaced all the instances of the Toast message with the ‘CustomSnackBar’ by passing in the desired messages that were being passed in Toast message.

But I still need to find the location of our stored CSV file and a method to delete the current generated CSV file.

For that, I did below modification to the CSVLogger class in PSLab android app.

public class CSVLogger {
   private static final String CSV_DIRECTORY = "PSLab";
   public CSVLogger(String category) {
       this.category = category;
       setupPath();
   }
   /*Below methods are included at the bottom of the class */
   public String getCurrentFilePath() {
       return Environment.getExternalStorageDirectory().getAbsolutePath() +
               File.separator + CSV_DIRECTORY + File.separator + category;
   }
   public void deleteFile() {
       csvFile.delete();
   }
}

Now for passing the location of the stored file and implementing delete option, I called the below method when the CSV logging is stopped by the user:

CustomSnackBar.showSnackBar((CoordinatorLayout) parent.findViewById(R.id.cl),

                    “CSV File stored at” + " " +lux_logger.getCurrentFilePath(),
  
                    “DELETE”,

                    new View.OnClickListener() {
                              Override
                              public void onClick(View view) {
                                             lux_logger.deleteFile();    
                              });

By doing this I get a Snackbar as shown in Figure 2, clicking on the “DELETE” text deletes the current CSV file.

Figure 2 shows snackbar showing file stored location and delete option

 

So, implementing Snackbar helped to make the app interactive and keeps user notified and control the data logging.

Resources

  1. https://www.journaldev.com/10324/android-snackbar-example-tutorial – Android SnackBar example implemetation tutorial
  2. https://material.io/develop/android/components/snackbar/ – Android Material Desing implementation of Snackbar.
Continue ReadingSnackbar Implementation in PSLab Android App

Implementing API to allow Admins to modify config of devices of any user

As any user can add or remove devices from their account, there needed to be a way by which Admins can manage the user devices. The Admins and higher user roles should have the access to modify the config of devices of any user. This blog post explains how an API has been implemented to facilitate Admins and higher user roles to change config of devices of any user.

Implementing a servlet to allow changing review status of a Skill

The basic task of the servlet is to allow Admin and higher user roles to modify the config of devices of any user. The Admin should be allowed to edit the name of the device and also the room of the device, similar to how a user can edit his own devices.

Here is the implementation of the API:

  1. The API should be usable to only the users who have a user role Admin or higher. Only those with minimum Admin rights should be allowed to control what Skills are displayed on the CMS site. This is implemented as follows:

   @Override
    public UserRole getMinimalUserRole() {
        return UserRole.ADMIN;
    }

 

  1. The endpoint for the API is ‘/cms/modifyUserDevices.json’. This is implemented as follows:

   @Override
    public String getAPIPath() {
        return "/cms/modifyUserDevices.json";
    }

 

  1. The main method of the servlet is the serviceImpl() method. This is where the actual code goes which will be executed each time the API is called. This is implemented as follows:

    JSONObject result = new JSONObject(true);
    Collection<ClientIdentity> authorized = DAO.getAuthorizedClients();
    List<String> keysList = new ArrayList<String>();
    authorized.forEach(client -> keysList.add(client.toString()));
    String[] keysArray = keysList.toArray(new 
    String[keysList.size()]);

    List<JSONObject> userList = new ArrayList<JSONObject>();
    for (Client client : authorized) {
        JSONObject json = client.toJSON();

        if(json.get("name").equals(email)) {
            ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, client.getName());
            Authorization authorization = DAO.getAuthorization(identity);

            ClientCredential clientCredential = new ClientCredential(ClientCredential.Type.passwd_login, identity.getName());
            Authentication authentication = DAO.getAuthentication(clientCredential);

            Accounting accounting = DAO.getAccounting(authorization.getIdentity());

            if(accounting.getJSON().has("devices")) {

                JSONObject userDevice = accounting.getJSON().getJSONObject("devices");
                if(userDevice.has(macid)) {
                    JSONObject deviceInfo = userDevice.getJSONObject(macid);
                    deviceInfo.put("name", name);
                    deviceInfo.put("room", room);
                }
                else {
                    throw new APIException(400, "Specified device does not exist.");
                }

            } else {
                json.put("devices", "");
            }
            accounting.commit();
        }
    }

 

Firstly, the list of authorized clients is fetched using DAO.getAuthorizedClients() and is put in an ArrayList. Then we traverse through each element of this ArrayList and check if the device exists by checking if there’s a key-value pair corresponding to the macid passed in the query parameter. If the device doesn’t exist, then an exception is thrown. However, if the macid exists in the traversed element of the ArrayList, then we put the name and the room of the device as passed as query parameters in that particular element of the ArrayList, so as to overwrite the existing name and room of the device of the user.

This is how an API has been implemented which allows Admins and higher user roles to modify the config of devices of any user.

Resources

Continue ReadingImplementing API to allow Admins to modify config of devices of any user

Implement Table Sorting In Badgeyay

In this blog post I am going to explain about implementation of inplace table sorting in badgeyay. This is not about just adding the sortable class as described in the semantic docs, but the data inside the table has different characteristics and needs to be sorted in a different manner. Not like the traditional way of comparing strings as it will not be suitable for dates. For creating a custom comparison function for sorting, either we can implement a custom comparator using JQuery or we can use the data values for comparison. The latter option is more preferable as it can be extended  to different columns in the table.

Procedure

  1. Adding the sortable class in the table, which needs to be sorted.
<table class=“ui sortable table”>

  . . .

</table>

 

  1. We need to enable a javascript function when DOM completely gets loaded.
<script type=“text/javascript”>
 $(‘table’).tablesort();
</script>

 

  1. After this we need to create a template helper to return us the time stamp from the UTC formatted DateTime string. The value that will be returned by the helper will be used as the data value for the column entries.
import { helper } from ‘@ember/component/helper’;

export function extractTimeStamp(date) {
return Math.floor((new Date(date)).getTime() / 100);
}

export default helper(extractTimeStamp);

 

  1. The value that will be returned by the helper will be used as data value for comparison by table sorter.
<td data-sort-value={{extract-time-stamp user.created_at}}>{{sanitizeDate user.created_at}}</td>

 

  1. Now we need that certain columns do not sort, as there is no need. Such columns are photoURL, actions etc. These columns should be ignored by the sorter for sorting, so we will add a class to avoid sorting of these columns.
<th class=“no-sort”>User Photo</th>

Resources

  • Semantic UI table class – Link
  • Data sorting in the table API – Link
  • Pull Request for the same – Link
  • Template helper guide ember – Link
Continue ReadingImplement Table Sorting In Badgeyay

Displaying name of users in Users tab in Admin Panel

In the Users tab in the Admin Panel, we have a lot of user information displayed in a tabular form. This information is fetched from the accounting objects of each user. As the users are now able to also store their name in their respective accounting object, hence we needed to implement a feature to display the name of the users in the Users table in a separate column. This blog post explains how the user names are fetched from the respective accounting objects and are then displayed in the Users table in the Admin Panel.

How is name of user stored on the server?

The name of any user is stored in the user’s accounting object. All the settings of a user are stored in a JSONObject with the key name as ‘settings’. The name of a user is also stored in ‘settings’ JSONObject. This is shown as follows:

Modifying GetUsers.java to return name of users

The endpoint /aaa/getUsers.json is used to return the accounting info of all users. This includes their signup time, last login time, last login IP, etc. We needed to modify it to return the name of users also along with the already returned data. This is implemented as follows:

   if(accounting.getJSON().has("settings")) {
        JSONObject settings = accounting.getJSON().getJSONObject("settings");
        if(settings.has("userName")) {
            json.put("userName", settings.get("userName"));
        }
        else {
            json.put("userName", "");
        }
    } else {
        json.put("userName", "");
    }
    accounting.commit();

 

Fetching names of all users from the server

We need to make an AJAX call to ‘/aaa/getUsers.json’ as soon as we switch to the Users tab in the Admin Panel. We need to extract all the required data from the JSON response object and put them in state variables so that they can further be used as data indexes for different columns of the table. The implementation of the AJAX call is as follows:

   let url =
      `${urls.API_URL}/aaa/getUsers.json?access_token=` +
      cookies.get('loggedIn') +
      '&page=' +
      page;
    $.ajax({
      url: url,
      dataType: 'jsonp',
      jsonp: 'callback',
      crossDomain: true,
      success: function(response) {
        let userList = response.users;
        let users = [];
        userList.map((data, i) => {
          let user = {
            userName: data.userName,
          };
          users.push(user);
          return 1;
        });
        this.setState({
          data: users,
        });
      }.bind(this)
    });

 

Displaying name of users in Users tab in Admin Panel

We needed to add another column titled ‘User Name’ in the Users table in the Admin Panel. The ‘dataIndex’ attribute of the Ant Design table component specifies the data value which is to be used for that particular column. For our purpose, our data value which needs to be displayed in the ‘User Name’ column is ‘userName’. We also specify a width of the column as another attribute. The implementation is as follows:

   this.columns = [
      // other columns
      {
        title: 'User Name',
        dataIndex: 'userName',
        width: '12%',
      }
      // other columns
    ];

 

This is how the names of users are fetched from their accounting object and are then being displayed in the Users tab in Admin Panel.

Resources

Continue ReadingDisplaying name of users in Users tab in Admin Panel

Forgot Password Service in Badgeyay

Badgeyay is an open source badge generator service for generating badges developed by FOSSASIA community for technical events and conferences. The project is divided into two components mainly frontend and backend. After creating the user registration functionality in application, if the user forgets the credentials for the login, then there must be a way to recreate the credentials using a secure channel. This is only valid for the users signed up through email login as for the case of OAuth they must have access to their ID on respective social platform. The main challenges in resetting password for the user is to provide a secure channel. So the problem can be breakdown into following issues:

  • Creating a token for reset action
  • Sending that token via mail to user
  • Verifying that token on the server and giving access
  • Changing the credentials  of the user

Procedure

  1. Generating token for the request to change credentials for the user. The token will be an expiry token and will be expired in the mentioned duration. So the token is valid for only a limited period of time and will prevent fraudulent requests.
def pwd_reset_token():
  data = request.get_json()[‘data’][‘attributes’]
  if ’email’ not in data.keys():
      print(‘Email not found’)
  email = data[’email’]
  user = User.getUser(email=email)
  if not user:
      return ErrorResponse(UserNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()
  expire = datetime.datetime.utcnow() + datetime.timedelta(seconds=900)
  token = jwt.encode({
      ‘id’: user.id,
      ‘exp’: expire
  }, app.config.get(‘SECRET_KEY’))
  resetObj = ResetPasswordToken(user.id, token.decode(‘UTF-8’))
  resetObj.save_to_db()
  return jsonify(TokenSchema().dump(resetObj).data)

Model for ResetPasswordToken

class ResetPasswordToken(db.Model):

  __tablename__ = ‘Reset Password Token’

  id = db.Column(db.String, primary_key=True)
  token = db.Column(db.String, nullable=False)

  def __init__(self, uid, token):
      self.id = uid
      self.token = token

  def save_to_db(self):
      try:
          db.session.add(self)
          db.session.commit()
      except Exception as e:
          db.session.rollback()
          db.session.flush()
          print(e)

 

  1. Sending the password reset link via mail to the user. The link will contain the token (expiry token) that will be used to validate the request. For the case we will be using Firebase Cloud functions as an HTTP Trigger.
exports.sendResetMail = functions.https.onRequest((req, res) => {
let token = req.query[‘token’];
let email = req.query[’email’];
res.setHeader(‘Content-Type’, ‘application/json’);
sendResetMail(token, email)
  .then(() => {
    console.log(‘Reset mail sent to’, email);
    res.json({ data: { attributes: { status: 200 }, id: token, type: ‘reset-mails’ } });
    return 0;
  })
  .catch(err => {
    console.error(err);
    res.json({ data: { attributes: { status: 500 }, id: token, type: ‘reset-mails’ } });
    return -1;
  });
});

function sendResetMail(token, email) {
const mailOptions = {
  from: `${APP_NAME}<noreply@firebase.com>`,
  to: email,
};

mailOptions.subject = `Password reset link`;
mailOptions.html = ‘<p>Hey ‘ + email + ‘! Here is your password reset <a href=\” + PASSWORD_RESET_LINK
  + token + ‘\’>Link</a><p>’;
return mailTransport.sendMail(mailOptions);
}

 

  1. Verifying the token on the server side to validate the user request
def validate_reset_token():
  args = request.args
  if ‘token’ in args.keys():
      token = args.get(‘token’)
  resp = {‘id’: token}
  try:
      jwt.decode(token, app.config[‘SECRET_KEY’])
      resp[‘valid’] = True
      return jsonify(ValidTokenSchema().dump(resp).data)
  except Exception as e:
      resp[‘valid’] = False
      print(e)
      return jsonify(ValidTokenSchema().dump(resp).data)

 

  1. After user has access to change the credentials, then user can send a POST request to backend through a form shown in UI to change its password.
def changePwd():
  try:
      data = request.get_json()[‘data’][‘attributes’]
  except Exception as e:
      print(e)
      return ErrorResponse(PayloadNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  token = data[‘token’]
  try:
      decoded_res = jwt.decode(token, app.config[‘SECRET_KEY’])
  except Exception as e:
      print(e)
      return ErrorResponse(SignatureExpired().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  user = User.getUser(user_id=decoded_res[‘id’])

  if ‘pwd’ not in data.keys():
      return ErrorResponse(PasswordNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  pwd = data[‘pwd’]
  oldPwd = user.password
  user.password = generate_password_hash(pwd)
  user.save_to_db()

  resp = {‘id’: token}
  if update_firebase_password(user.id, pwd):
      resp[‘status’] = ‘Changed’
      return jsonify(ResetPasswordOperation().dump(resp).data)
  else:
      print(‘Firebase not uploaded’)
      user.password = oldPwd
      user.save_to_db()
      resp[‘status’] = ‘Not Changed’
      return jsonify(ResetPasswordOperation().dump(resp).data)

 

  1. After this the password of the user will be changed and allowed to login through new credentials.

Link to PRs:

  • PR for forgot password reset form – #1
  • PR for implementing forgot password on firebase side – #2
  • PR for password reset mail functionality – #3

Resources

  • HTTP Trigger Cloud functions – Link
  • Nodemailer message configuration – Link
  • Ember Data Guide – Link

 

Continue ReadingForgot Password Service in Badgeyay

Metadata Updation in Badgeyay

Badgeyay is a simple badge generator service to develop badges for technical events and conferences developed by FOSSASIA. Badgeyay is a SPA (Single Page Application) developed in ember, whose backend is in Flask. Now when user logins, he can see an option for user profile, in which all the metadata of its profile can be seen (extracted from Firebase). Now user should be able to change its metadata like profile image and username etc. So we will look how the profile image is being changed and updated in badgeyay.

Procedure

  1. Create function in frontend to listen for onclick events and initiate a file upload dialog box for selecting an image. We will use document property to initiate a dummy click event, else there will be a button with the text to upload a file and that won’t look consistent as we only need an image and nothing else on the UI.

class=“ui small circular image profile-image”>
    “{{user.photoURL}}”>
    “display: none;” id=“profileImageSelector” type=“file” onchange={{action “profileImageSelected”}}>
    

“profile-change” onclick={{action “updateProfileImage”}}>Change


</div>

 

  1. Function to upload file and initiate a dummy click event
updateProfileImage() {
    // Initate a dummy click event
    document.getElementById(‘profileImageSelector’).click();
  },

  profileImageSelected(event) {
    const reader = new FileReader();
    const { target } = event;
    const { files } = target;
    const [file] = files;
    const _this = this;

    reader.onload = () => {
      _this.get(‘sendProfileImage’)(reader.result, file.type.split(‘/’)[1]);
    };

    reader.readAsDataURL(file);
  }

 

  1. Profile update function in the main controller to call the API endpoint to upload the data to backend. This will send the payload to backend which will later upload the image to cloud storage and save in the link in the database.
updateProfileImage(profileImageData, extension) {
    const _this = this;
    const user = this.get(‘store’).peekAll(‘user’);
    user.forEach(user_ => {
      _this.set(‘uid’, user_.get(‘id’));
    });
    let profileImage = _this.get(‘store’).createRecord(‘profile-image’, {
      image   : profileImageData,
      uid   : _this.uid,
      extension : ‘.’ + extension
    });
    profileImage.save()
      .then(record => {
        user.forEach(user_ => {
          user_.set(‘photoURL’, record.photoURL);
        });
      })
      .catch(err => {
        let userErrors = profileImage.get(‘errors.user’);
        if (userErrors !== undefined) {
          _this.set(‘userError’, userErrors);
        }
      });
  }
  1. Route to update profile image from backend
@router.route(‘/profileImage’, methods=[‘POST’])
def update_profile_image():
  try:
      data = request.get_json()[‘data’][‘attributes’]
  except Exception:
      return ErrorResponse(PayloadNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  if not data[‘image’]:
      return ErrorResponse(ImageNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  if not data[‘extension’]:
      return ErrorResponse(ExtensionNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  uid = data[‘uid’]
  image = data[‘image’]
  extension = data[‘extension’]

  try:
      imageName = saveToImage(imageFile=image, extension=extension)
  except Exception:
      return ErrorResponse(ImageNotFound().message, 422, {‘Content-Type’: ‘application/json’}).respond()

  fetch_user, imageLink = update_database(uid, imageName)
  return jsonify(UpdateUserSchema().dump(fetch_user).data)

 

This will first create a temp file with the data URI and them upload that file to cloud storage and generate the link and then update the user in the database.

def update_database(uid, imageName):
  fetch_user = User.getUser(user_id=uid)
  if fetch_user is None:
      return ErrorResponse(UserNotFound(uid).message, 422, {‘Content-Type’: ‘application/json’}).respond()
  imagePath = os.path.join(app.config.get(‘BASE_DIR’), ‘static’, ‘uploads’, ‘image’) + ‘/’ + imageName
  imageLink = fileUploader(imagePath, ‘profile/images/’ + imageName)
  fetch_user.photoURL = imageLink
  fetch_user.save_to_db()

  try:
      os.unlink(imagePath)
  except Exception:
      print(‘Unable to delete the temporary file’)

  return fetch_user, imageLink

 

Link to PR – Link

Topics Involved

  • Google Cloud Admin Storage SDK
  • Ember data

Resources

  • Firebase admin sdk documentation – Link
  • Google Cloud Storage SDK Python – Link
  • Blob Management – Link
  • Documents API – Link
Continue ReadingMetadata Updation in Badgeyay

Uploading Badges To Google Cloud Badgeyay

Badgeyay is an open source project developed by FOSSASIA community. This project mainly aims for generating badges for technical conferences and events.The project is divided into two parts mainly. Backend is developed in flask and frontend is developed in emberjs. The problem is after the badge generation, the flask server is storing and serving those files. In practise this is not a good convention to do so. This should be handled by secondary hosting server program like gunicorn or nginx. Better approach would be to consume the firebase storage and admin sdk for storing the badges on google cloud. This will also offload storage needs from the flask server and also give a public link over the network to access.

Procedure

  1. Get the file path of the temporary badge generated on the flask server. Currently badges are saved in the directory of the image file uploaded and final badge generated is written in all-badges.pdf`
badgePath = os.getcwd() + ‘/static/temporary/’ + badgeFolder
badgePath + ‘/all-badges.pdf’

 

  1. Create the blob path for the storage. Blob can be understood as the final reference to the location where the contents are saved onto the server. This can be a nested directory structure or simply a filename in root directory.
‘badges/’ + badge_created.id + ‘.pdf’

 

In our case it is the id of the badge that is generated in the badges directory.

  1. Function for uploading the file generated in temporary storage to google cloud storage.
def fileUploader(file_path, blob_path):
  bucket = storage.bucket()
  fileUploaderBlob = bucket.blob(blob_path)
  try:
      with open(file_path, ‘rb’) as file_:
          fileUploaderBlob.upload_from_file(file_)
  except Exception as e:
      print(e)
  fileUploaderBlob.make_public()
  return fileUploaderBlob.public_url

 

It creates a bucket using the firebase admin SDK and then open the file from the file path. After opening the file from the path it writes the data to the cloud storage. After the data is written, the blob is made public and the public access link to the blob is fetched, which then later returned and saved in the local database.

Topics Involved

  • Firebase admin sdk for storage
  • Google cloud storage sdk

Resources

  • Firebase admin sdk documentation – Link
  • Google Cloud Storage SDK Python – Link
  • Blob Management – Link

 

Continue ReadingUploading Badges To Google Cloud Badgeyay

Ember Controller for Badge Generation In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community. This project aims towards giving a platform for badge generation using several customizations options. Current structure of project is in two parts to maintain modularity, which are namely backend, developed in flask, and frontend, developed in ember.

After refactoring the frontend and backend API we need to create a controller for the badge generation in frontend. Controller will help the components to send and receive data from them and prepare the logic for sending request to API so that badges can be generated and can receive the result as response from the server. Particularly we need to create the controller for badge generation route, create-badges.

As there are many customizations option presented to user, we need to chain the requests so that they sync with each other and the logic should not break for the badge generation.

Procedure

  1. Creating the controller from the ember-cli
ember g controller create-badge

 

  1. After the component generation, we need to create actions that can be passed to components. Let’s build action to submit form and then chain the different actions together for the badge generation.
submitForm() {
  const _this = this;
  const user = _this.get(‘store’).peekAll(‘user’);
  let uid;
  user.forEach(user_ => {
    uid = user_.get(‘id’);
  });
  if (uid !== undefined && uid !== ) {
    _this.set(‘uid’, uid);
  }

  let badgeData = {
    uid     : _this.uid,
    badge_size : ‘A3’
  };

  if (_this.csvEnable) {
    badgeData.csv = _this.csvFile;
  }
  if (_this.defFontColor !== && _this.defFontColor !== undefined) {
    badgeData.font_color = ‘#’ + _this.defFontColor;
  }
  if (_this.defFontSize !== && _this.defFontSize !== undefined) {
    badgeData.font_size = _this.defFontSize.toString();
  }
  if (_this.defFont !== && _this.defFont !== undefined) {
    badgeData.font_type = _this.defFont;
  }

  _this.send(‘sendManualData’, badgeData);

},

 

  1. As we can see in the above code snippet that _this.send(action_name, arguments) is calling another action sendManualData. This action then sends a network request to the backend if the Manual data is selected as input source otherwise will go with the CSV upload. If no option is chosen then it will show an error on the user screen, notifying him to select one input source.
sendManualData(badgeData) {
    const _this = this;
    if (_this.manualEnable) {
      let textEntry = _this.get(‘store’).createRecord(‘text-data’, {
        uid   : _this.uid,
        manual_data : _this.get(‘textData’),
        time   : new Date()
      });
      textEntry.save().then(record => {
        _this.set(‘csvFile’, record.filename);
        badgeData.csv = _this.csvFile;
        _this.send(‘sendDefaultImg’, badgeData);
        _this.get(‘notify’).success(‘Text saved Successfully’);
      }).catch(err => {
        let userErrors = textEntry.get(‘errors.user’);
        if (userErrors !== undefined) {
          _this.set(‘userError’, userErrors);
        }
      });
    } else if (_this.csvEnable) {
      if (_this.csvFile !== undefined && _this.csvFile !== ) {
        badgeData.csv = _this.csvFile;
        _this.send(‘sendDefaultImg’, badgeData);
      }
    } else {
      // No Input Source specified Error
    }
  },

 

The above code will choose the manual data if the manual data boolean flag is set else not, and then does a network request and wait for the promise to be resolved. As soon as the promise is resolved it calls another action to for the default image.

  1. After selecting the input source, now the background for the badge has to be selected. It will look for the boolean flags of the defaultImage, backgroundColorImage, customImage and will make the network request accordingly.
sendDefaultImg(badgeData) {
    const _this = this;
    if (_this.defImage) {
      let imageRecord = _this.get(‘store’).createRecord(‘def-image-upload’, {
        uid    : _this.uid,
        defaultImage : _this.defImageName
      });
      imageRecord.save()
        .then(record => {
          _this.set(‘custImgFile’, record.filename);
          badgeData.image = _this.custImgFile;
          _this.send(‘sendBadge’, badgeData);
        })
        .catch(error => {
          let userErrors = imageRecord.get(‘errors.user’);
          if (userErrors !== undefined) {
            _this.set(‘userError’, userErrors);
          }
        });
    } else if (_this.custImage) {
      if (_this.custImgFile !== undefined && _this.custImgFile !== ) {
        badgeData.image = _this.custImgFile;
        _this.send(‘sendBadge’, badgeData);
      }
    } else if (_this.colorImage && _this.defColor !== undefined && _this.defColor !== ) {
      console.log(_this.defColor);
      let imageRecord = _this.get(‘store’).createRecord(‘bg-color’, {
        uid : _this.uid,
        bg_color : _this.defColor
      });
      imageRecord.save()
        .then(record => {
          badgeData.image = record.filename;
          _this.send(‘sendBadge’, badgeData);
        })
        .catch(error => {
          let userErrors = imageRecord.get(‘errors.user’);
          if (userErrors !== undefined) {
            _this.set(‘userError’, userErrors);
          }
        });
    } else {
      // Inflate error for No Image source.
    }
  },

 

After the promise resolvement, the final action is called to send badge data payload to backend api for badge generation.

  1. After the complete preparation of the payload, now it’s time to send the payload to the backend api for the badge generation and after the promise resolvement showing the respective downloadable link in the frontend.
sendBadge(badgeData) {
    const _this = this;
    let badgeRecord = _this.get(‘store’).createRecord(‘badge’, badgeData);
    badgeRecord.save()
      .then(record => {
        _this.set(‘badgeGenerated’, true);
        _this.set(‘genBadge’, record.id);
        _this.get(‘notify’).success(‘Badge generated Successfully’);
      })
      .catch(err => {
        console.error(err.message);
      });
  },

 

Now after the promise resolvement the local variable badgGenerated is set to true so that the success message can be shown in the frontend for successful badge generation along with the link.

Link to respective PR – Link

Topics Involved

  • Chaining of actions and requests
  • Manipulating DOM on the conditional statements
  • Component bindings
  • Ember data
  • Promise resolvement

Resources

  • Link to ember data for the API requests and promise resolvement – Link
  • Implementing Controllers in Ember – Link
  • Chaining actions together in ember – Link
Continue ReadingEmber Controller for Badge Generation In Badgeyay

Integrating Firebase Cloud Functions In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community for generating badges for conferences and events. The Project is divided into two parts frontend, which is in ember, and backend, which is in flask. Backend uses firebase admin SDK (Python) and Frontend uses firebase javascript client with emberfire wrapper for ember. Whenever an user signs up on the website, database listener that is attached to to the Model gets triggered and uses flask-mail for sending welcome mail to the user and in case of email and password signup, verification mail as well.

Problem is sending mail using libraries is a synchronous process and takes a lot of processing on the server. We can use messaging queues like RabbitMQ and Redis but that will be burden as server cost will increase. The workaround is to remove the code from the server and create a firebase cloud function for the same task.

Firebase cloud functions lets you run backend code on the cloud and can be triggered with HTTP events or listen for the events on the cloud, like user registration.

Procedure

  1. Firebase uses our Gmail ID for login, so make sure to have a Gmail ID and on the first sight we will be greeted with Firebase console, where we can see our created or imported firebase apps.

  2. Create the app by clicking on the Add Project Icon and write the name of the application (e.g. Test Application) and select the region, in my case it is India. Firebase will automatically generated an application ID for the app. Click on Create Project to complete creation of project

  3. After Completion, click on the project to enter into the project. You will be greeted with an overview saying to integrate firebase with your project. We will click on the Add Firebase to web App and save the config as JSON in a file as clientKey.json for later use.

  4. Now we need to install the firebase tools on our local machine so for that execute
    npm i -g firebase-tools
    1. Now login from the CLI so that firebase gets token for the Gmail ID of the user and can access the firebase account of that Gmail ID.
    firebase login
    1. After giving permissions to the firebase CLI from your Gmail account in the new tab opened in browser, create a folder named cloud_functions in the project directory and in that execute
    firebase init
    1. Select only functions from the list of options by pressing space.

    2. After this select the project from the list where you want to use the cloud function. You can skip the step if you later want to add the cloud function to project by selecting don’t setup a default project and can later be used by command
      firebase use –add

    3. Choose the language of choice

    4. If you want, you can enforce eslint on the project and after this the cloud function is set up and the directory structure looks as follows.

    5. We will write our cloud function in index.js. So let’s take a look at index.js
      const functions = require(‘firebase-functions’);

      // // Create and Deploy Your First Cloud Functions
      // // https://firebase.google.com/docs/functions/write-firebase-functions
      //
      // exports.helloWorld = functions.https.onRequest((request, response) => {
      //  response.send(“Hello from Firebase!”);
      // });

      As we can see there is a sample function already given, we don’t need that sample function so we will remove it and will write the logic for sending mail. Before that we need to acquire the key for service accounts so that admin functionality can be accessed in the cloud function. So for that go to project settings and then service accounts and click on Generate New Private Key  and save it as serviceKey.json

    6. Now the directory structure will look like this after adding the clientKey.json and serviceKey.json

    7. We will use node-mailer for sending mails in cloud functions and as there is a limitation on the gmail account to send only 500 mails in a day, we can use third party services like sendGrid and others for sending mails with firebase. Configure node-mailer for sending mails as
      const nodemailer = require(‘nodemailer’);

      const gmailEmail = functions.config().gmail.email;
      const gmailPassword = functions.config().gmail.password;
      const mailTransport = nodemailer.createTransport({
       service: ‘gmail’,
       auth: {
      user: gmailEmail,
      pass: gmailPassword
       }
      });

      Also set the environment variables for the cloud functions like email and password:

      firebase functions:config:set gmail.email=“Email ID” gmail.password=“Password”
      1. Logic for sending Greeting Mail on user registration
      exports.greetingMail = functions.auth.user().onCreate((user) => {
       const email = user.email;
       const displayName = user.displayName;

       return sendGreetingMail(email, displayName);
      });

      function sendGreetingMail(email, displayName) {
       const mailOptions = {
      from: `${APP_NAME}<noreply@firebase.com>`,
      to: email,
       };

       mailOptions.subject = `Welcome to Badgeyay`;
       mailOptions.text = `Hey ${displayName || ”}! Welcome to Badgeyay. We welcome you onboard and pleased to offer you service.`;
       return mailTransport.sendMail(mailOptions).then(() => {
      return console.log(‘Welcome mail sent to: ‘, email)
       }).catch((err) => {
      console.error(err.message);
       });
      }

      Function will get triggered on creation of user in firebase and calls the greeting mail function with parameters as the email id of the registered user and the Display name. Then a default template is used to send mail to the recipient and Logged on successful submission.

      1. Currently firebase admin sdk doesn’t support the functionality to send verification mail but the client SDK does. So the approach which is followed in badgeyay is that admin SDK will create a custom token and client sdk will use that custom token to sign in and them send verification mail to the user.
      exports.sendVerificationMail = functions.auth.user().onCreate((user) => {
       const uid = user.uid;
       if (user.emailVerified) {
      console.log(‘User has email already verified: ‘, user.email);
      return 0;
       } else {
      return admin.auth().createCustomToken(uid)
        .then((customToken) => {
          return firebase.auth().signInWithCustomToken(customToken)
        })
        .then((curUser) => {
          return firebase.auth().onAuthStateChanged((user_) => {
            if (!user.emailVerified) {
              user_.sendEmailVerification();
              return console.log(‘Verification mail sent: ‘, user_.email);
            } else {
              return console.log(‘Email is already verified: ‘, user_.email);
            }
          })
        })
        .catch((err) => {
          console.error(err.message);
        })
       }
      });
      1. Now we need to deploy the functions to firebase.
      firebase deploy –only functions

      Link to the respective PR  : Link

      Topics Involved

      • Firebase Admin SDK
      • Configuring Gmail for third party apps
      • Token Verification and verification mail by client SDK
      • Nodemailer and Express.js

      Resources

      • Firebase Cloud functions – Link
      • Extending authentication with cloud function – Link
      • Custom Token Verification – Link
      • Nodemailer message configuration – Link
      • Issue discussion on sending verification mail with admin SDK – Link
Continue ReadingIntegrating Firebase Cloud Functions In Badgeyay