Integrating Firebase Cloud Functions In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community for generating badges for conferences and events. The Project is divided into two parts frontend, which is in ember, and backend, which is in flask. Backend uses firebase admin SDK (Python) and Frontend uses firebase javascript client with emberfire wrapper for ember. Whenever an user signs up on the website, database listener that is attached to to the Model gets triggered and uses flask-mail for sending welcome mail to the user and in case of email and password signup, verification mail as well.

Problem is sending mail using libraries is a synchronous process and takes a lot of processing on the server. We can use messaging queues like RabbitMQ and Redis but that will be burden as server cost will increase. The workaround is to remove the code from the server and create a firebase cloud function for the same task.

Firebase cloud functions lets you run backend code on the cloud and can be triggered with HTTP events or listen for the events on the cloud, like user registration.

Procedure

  1. Firebase uses our Gmail ID for login, so make sure to have a Gmail ID and on the first sight we will be greeted with Firebase console, where we can see our created or imported firebase apps.

  2. Create the app by clicking on the Add Project Icon and write the name of the application (e.g. Test Application) and select the region, in my case it is India. Firebase will automatically generated an application ID for the app. Click on Create Project to complete creation of project

  3. After Completion, click on the project to enter into the project. You will be greeted with an overview saying to integrate firebase with your project. We will click on the Add Firebase to web App and save the config as JSON in a file as clientKey.json for later use.

  4. Now we need to install the firebase tools on our local machine so for that execute
    npm i -g firebase-tools
    1. Now login from the CLI so that firebase gets token for the Gmail ID of the user and can access the firebase account of that Gmail ID.
    firebase login
    1. After giving permissions to the firebase CLI from your Gmail account in the new tab opened in browser, create a folder named cloud_functions in the project directory and in that execute
    firebase init
    1. Select only functions from the list of options by pressing space.

    2. After this select the project from the list where you want to use the cloud function. You can skip the step if you later want to add the cloud function to project by selecting don’t setup a default project and can later be used by command
      firebase use –add

    3. Choose the language of choice

    4. If you want, you can enforce eslint on the project and after this the cloud function is set up and the directory structure looks as follows.

    5. We will write our cloud function in index.js. So let’s take a look at index.js
      const functions = require(‘firebase-functions’);

      // // Create and Deploy Your First Cloud Functions
      // // https://firebase.google.com/docs/functions/write-firebase-functions
      //
      // exports.helloWorld = functions.https.onRequest((request, response) => {
      //  response.send(“Hello from Firebase!”);
      // });

      As we can see there is a sample function already given, we don’t need that sample function so we will remove it and will write the logic for sending mail. Before that we need to acquire the key for service accounts so that admin functionality can be accessed in the cloud function. So for that go to project settings and then service accounts and click on Generate New Private Key  and save it as serviceKey.json

    6. Now the directory structure will look like this after adding the clientKey.json and serviceKey.json

    7. We will use node-mailer for sending mails in cloud functions and as there is a limitation on the gmail account to send only 500 mails in a day, we can use third party services like sendGrid and others for sending mails with firebase. Configure node-mailer for sending mails as
      const nodemailer = require(‘nodemailer’);

      const gmailEmail = functions.config().gmail.email;
      const gmailPassword = functions.config().gmail.password;
      const mailTransport = nodemailer.createTransport({
       service: ‘gmail’,
       auth: {
      user: gmailEmail,
      pass: gmailPassword
       }
      });

      Also set the environment variables for the cloud functions like email and password:

      firebase functions:config:set gmail.email=“Email ID” gmail.password=“Password”
      1. Logic for sending Greeting Mail on user registration
      exports.greetingMail = functions.auth.user().onCreate((user) => {
       const email = user.email;
       const displayName = user.displayName;

       return sendGreetingMail(email, displayName);
      });

      function sendGreetingMail(email, displayName) {
       const mailOptions = {
      from: `${APP_NAME}<noreply@firebase.com>`,
      to: email,
       };

       mailOptions.subject = `Welcome to Badgeyay`;
       mailOptions.text = `Hey ${displayName || ”}! Welcome to Badgeyay. We welcome you onboard and pleased to offer you service.`;
       return mailTransport.sendMail(mailOptions).then(() => {
      return console.log(‘Welcome mail sent to: ‘, email)
       }).catch((err) => {
      console.error(err.message);
       });
      }

      Function will get triggered on creation of user in firebase and calls the greeting mail function with parameters as the email id of the registered user and the Display name. Then a default template is used to send mail to the recipient and Logged on successful submission.

      1. Currently firebase admin sdk doesn’t support the functionality to send verification mail but the client SDK does. So the approach which is followed in badgeyay is that admin SDK will create a custom token and client sdk will use that custom token to sign in and them send verification mail to the user.
      exports.sendVerificationMail = functions.auth.user().onCreate((user) => {
       const uid = user.uid;
       if (user.emailVerified) {
      console.log(‘User has email already verified: ‘, user.email);
      return 0;
       } else {
      return admin.auth().createCustomToken(uid)
        .then((customToken) => {
          return firebase.auth().signInWithCustomToken(customToken)
        })
        .then((curUser) => {
          return firebase.auth().onAuthStateChanged((user_) => {
            if (!user.emailVerified) {
              user_.sendEmailVerification();
              return console.log(‘Verification mail sent: ‘, user_.email);
            } else {
              return console.log(‘Email is already verified: ‘, user_.email);
            }
          })
        })
        .catch((err) => {
          console.error(err.message);
        })
       }
      });
      1. Now we need to deploy the functions to firebase.
      firebase deploy –only functions

      Link to the respective PR  : Link

      Topics Involved

      • Firebase Admin SDK
      • Configuring Gmail for third party apps
      • Token Verification and verification mail by client SDK
      • Nodemailer and Express.js

      Resources

      • Firebase Cloud functions – Link
      • Extending authentication with cloud function – Link
      • Custom Token Verification – Link
      • Nodemailer message configuration – Link
      • Issue discussion on sending verification mail with admin SDK – Link
Continue ReadingIntegrating Firebase Cloud Functions In Badgeyay

Integrating Firebase Cloud Functions In Badgeyay

Badgeyay is an open source project developed by FOSSASIA Community for generating badges for conferences and events. The Project is divided into two parts frontend, which is in ember, and backend, which is in flask. Backend uses firebase admin SDK (Python) and Frontend uses firebase javascript client with emberfire wrapper for ember. Whenever an user signs up on the website, database listener that is attached to to the Model gets triggered and uses flask-mail for sending welcome mail to the user and in case of email and password signup, verification mail as well.

Problem is sending mail using libraries is a synchronous process and takes a lot of processing on the server. We can use messaging queues like RabbitMQ and Redis but that will be burden as server cost will increase. The workaround is to remove the code from the server and create a firebase cloud function for the same task.

Firebase cloud functions lets you run backend code on the cloud and can be triggered with HTTP events or listen for the events on the cloud, like user registration.

Procedure

  1. Firebase uses our Gmail ID for login, so make sure to have a Gmail ID and on the first sight we will be greeted with Firebase console, where we can see our created or imported firebase apps.

  1. Create the app by clicking on the Add Project Icon and write the name of the application (e.g. Test Application) and select the region, in my case it is India. Firebase will automatically generated an application ID for the app. Click on Create Project to complete creation of project

  2. After Completion, click on the project to enter into the project. You will be greeted with an overview saying to integrate firebase with your project. We will click on the Add Firebase to web App and save the config as JSON in a file as clientKey.json for later use.

  1. Now we need to install the firebase tools on our local machine so for that execute
npm i -g firebase-tools

 

  1. Now login from the CLI so that firebase gets token for the Gmail ID of the user and can access the firebase account of that Gmail ID.
firebase login

 

  1. After giving permissions to the firebase CLI from your Gmail account in the new tab opened in browser, create a folder named cloud_functions in the project directory and in that execute
firebase init

 

  1. Select only functions from the list of options by pressing space.

  2. After this select the project from the list where you want to use the cloud function. You can skip the step if you later want to add the cloud function to project by selecting don’t setup a default project and can later be used by command
firebase use --add

  1. Choose the language of choice

  2. If you want, you can enforce eslint on the project and after this the cloud function is set up and the directory structure looks as follows.

  3. We will write our cloud function in index.js. So let’s take a look at index.js
const functions = require('firebase-functions');

// // Create and Deploy Your First Cloud Functions
// // https://firebase.google.com/docs/functions/write-firebase-functions
//
// exports.helloWorld = functions.https.onRequest((request, response) => {
// response.send("Hello from Firebase!");
// });

 

As we can see there is a sample function already given, we don’t need that sample function so we will remove it and will write the logic for sending mail. Before that we need to acquire the key for service accounts so that admin functionality can be accessed in the cloud function. So for that go to project settings and then service accounts and click on Generate New Private Key  and save it as serviceKey.json

  1. Now the directory structure will look like this after adding the clientKey.json and serviceKey.json

  2. We will use node-mailer for sending mails in cloud functions and as there is a limitation on the gmail account to send only 500 mails in a day, we can use third party services like sendGrid and others for sending mails with firebase. Configure node-mailer for sending mails as
const nodemailer = require('nodemailer');

const gmailEmail = functions.config().gmail.email;
const gmailPassword = functions.config().gmail.password;
const mailTransport = nodemailer.createTransport({
service: 'gmail',
auth: {
user: gmailEmail,
pass: gmailPassword
}
});

 

Also set the environment variables for the cloud functions like email and password:

firebase functions:config:set gmail.email="Email ID" gmail.password="Password"

 

  1. Logic for sending Greeting Mail on user registration
exports.greetingMail = functions.auth.user().onCreate((user) => {
const email = user.email;
const displayName = user.displayName;

return sendGreetingMail(email, displayName);
});

function sendGreetingMail(email, displayName) {
const mailOptions = {
from: `${APP_NAME}<noreply@firebase.com>`,
to: email,
};

mailOptions.subject = `Welcome to Badgeyay`;
mailOptions.text = `Hey ${displayName || ''}! Welcome to Badgeyay. We welcome you onboard and pleased to offer you service.`;
return mailTransport.sendMail(mailOptions).then(() => {
return console.log('Welcome mail sent to: ', email)
}).catch((err) => {
console.error(err.message);
});
}

 

Function will get triggered on creation of user in firebase and calls the greeting mail function with parameters as the email id of the registered user and the Display name. Then a default template is used to send mail to the recipient and Logged on successful submission.

  1. Currently firebase admin sdk doesn’t support the functionality to send verification mail but the client SDK does. So the approach which is followed in badgeyay is that admin SDK will create a custom token and client sdk will use that custom token to sign in and them send verification mail to the user.
exports.sendVerificationMail = functions.auth.user().onCreate((user) => {
const uid = user.uid;
if (user.emailVerified) {
console.log('User has email already verified: ', user.email);
return 0;
} else {
return admin.auth().createCustomToken(uid)
.then((customToken) => {
return firebase.auth().signInWithCustomToken(customToken)
})
.then((curUser) => {
return firebase.auth().onAuthStateChanged((user_) => {
if (!user.emailVerified) {
user_.sendEmailVerification();
return console.log('Verification mail sent: ', user_.email);
} else {
return console.log('Email is already verified: ', user_.email);
}
})
})
.catch((err) => {
console.error(err.message);
})
}
});

 

  1. Now we need to deploy the functions to firebase.
firebase deploy --only functions

 

Link to the respective PR  : Link

 

Topics Involved

  • Firebase Admin SDK
  • Configuring Gmail for third party apps
  • Token Verification and verification mail by client SDK
  • Nodemailer and Express.js

 

Resources

  • Firebase Cloud functions – Link
  • Extending authentication with cloud function – Link
  • Custom Token Verification – Link
  • Nodemailer message configuration – Link
  • Issue discussion on sending verification mail with admin SDK – Link
Continue ReadingIntegrating Firebase Cloud Functions In Badgeyay

Scaling the logo of the generated events properly in Open Event Webapp

In the Facebook Developer Conference, the logo was too small

27190158-334a5446-5211-11e7-95e8-9690dfe8312e.png

In the Open Tech Summit Event, the logo was too long and increased the height of the navigation bar

77dc9fb0-3966-11e7-92c3-d6321dc98ac7.png

We decide some constraints regarding the width and the height of the logo. We don’t want the width of the logo to exceed greater than 110 pixels in order to not let it become too wide. It would look odd on small and medium screen if barely passable on bigger screens. We also don’t want the logo to become too long so we set a max-height of 45 pixels on the logo. So, we apply a class on the logo element with these properties

.logo-image {
 max-width: 110px;
 max-height: 45px;
}

But simply using these properties doesn’t work properly in some cases as shown in the above screenshots. An alternative approach is to resize the logo appropriately during the generation process itself. There are many different ways in which we can resize the logo. One of them was to scale the logo to a fixed dimension during the generation process. The disadvantage of that approach was that the event logo comes in different size and shapes. So resizing them to a fixed size will change its aspect ratio and it will appear stretched and pixelated. So, that approach is not feasible. We need to think of something different.  After a lot of thinking, we came up with an algorithm for the problem. We know the height of the logo would not be greater than 45px. We calculate the appropriate width and height of the logo, resize the image, and calculate dynamic padding which we add to the anchor element (inside which the image is located) if the height of the image comes out to be less than 45px. This is all done during the generation of the app. Note that the default padding is 5px and we add the extra pixels on top of it. This way, the logo doesn’t appear out of place or pixelated or extra wide and long. The detailed steps are mentioned below

  • Declare variable padding = 5px
  • Get the width, height and aspect ratio of the image.
  • Set the height to 45px and calculate the width according to the aspect ratio. If the width <= 110px, then directly resize the image and no change in padding is necessary
  • If the width > 110px, then make width constant to 110px and calculate height according to the aspect ratio. It will surely come less than 45px. Subtract the difference = (45 – height), divide it by 2 and add it to the padding variable.
  • Apply padding variable on the anchor tag. Now every logo should be displayed nicely and we have fixed the height of the navigation bar = 55px for all cases.

Here is an excerpt of the code. The whole work and discussion can be viewed here

var optimizeLogo = function(image, socket, done) {
 sharp(image).metadata(function(err, metaData) {
   if(err) {
     return done(err);
   }
   var width = metaData.width;
   var height = metaData.height;
   var ratio = width/height;
   var padding = 5;
   var diffHeight = 0;

   height = 45;
   width = Math.floor(45 * ratio);
   if (width > 110) {
     width = 110;
     height = Math.floor(width/ratio);
     diffHeight = 45 - height;
     padding = padding + (diffHeight)/2;
   }
   sharp(image).resize(width, height).toFile(image + '.new', function(err, info) {
     return done(null, padding);
   });
 });
};

It solved the problem. Now the logos of all the events were displaying properly. They were neither too wide, long or short. Here are some screenshots to show the improvements.

Facebook Developer Conference

27262455-d3558540-5474-11e7-852e-ef98888ef647.png

Open Tech Summit 2017

27262033-6f9d2df8-546c-11e7-811b-e35f849072eb.png

Resources:

Continue ReadingScaling the logo of the generated events properly in Open Event Webapp

Writing Selenium Tests for Checking Bookmark Feature and Search functionality in Open Event Webapp

We integrated Selenium Testing in the Open Event Webapp and are in full swing in writing tests to check the major features of the webapp. Tests help us to fix the issues/bugs which have been solved earlier but keep on resurging when some new changes are incorporated in the repo. I describe the major features that we are testing in this.

Bookmark Feature
The first major feature that we want to test is the bookmark feature. It allows the users to mark a session they are interested in and view them all at once with a single click on the starred button. We want to ensure that the feature is working on all the pages.

Let us discuss the design of the test. First, we start with tracks page. We select few sessions (2 here) for test and note down their session_ids. Finding an element by its id is simple in Selenium can be done easily. After we find the session element, we then find the mark button inside it (with the help of its class name) and click on it to mark the session. After that, we click on the starred button to display only the marked sessions and proceed to count the number of visible elements on the page. If the number of visible session elements comes out to be 2 (the ones that we marked), it means that the feature is working. If the number deviates, it indicates that something is wrong and the test fails.

82080522-21e8-403d-9906-3b4f420720b9.png

Here is a part of the code implementing the above logic. The whole code can be seen here

// Returns the number of visible session elements on the tracks page
TrackPage.getNoOfVisibleSessionElems = function() {
 return this.findAll(By.className('room-filter')).then(this.getElemsDisplayStatus).then(function(displayArr) {
   return displayArr.reduce(function(counter, value) { return value == 1 ? counter + 1 : counter; }, 0);
 });
};
// Bookmark the sessions, scrolls down the page and then count the number of visible session elements
TrackPage.checkIsolatedBookmark = function() {
 // Sample sessions having ids of 3014 and 3015 being checked for the bookmark feature
 var sessionIdsArr = ['3014', '3015'];
 var self = this;
 return  self.toggleSessionBookmark(sessionIdsArr).then(self.toggleStarredButton.bind(self)).then(function() {
   return self.driver.executeScript('window.scrollTo(0, 400)').then(self.getNoOfVisibleSessionElems.bind(self));
 });
};

Here is the excerpt of code which matches the actual number of visible session elements to the expected number. You can view the whole test script here

//Test for checking the bookmark feature on the tracks page
it('Checking the bookmark toggle', function(done) {
 trackPage.checkIsolatedBookmark().then(function(num) {
   assert.equal(num, 2);
   done();
 }).catch(function(err) {
   done(err);
 });
});

Now, we want to test this feature on the other pages: schedule and rooms page. We can simply follow the same approach as done on the tracks page but it is time expensive. Checking the visibility of all the sessions elements present on the page takes quite some time due to a large number of sessions. We need to think of a different approach.We had already marked two elements on the tracks page. We then go to the schedule page and click on the starred mode. We calculate the current height of the page. We then unmark a session and then recalculate the height of the page again. If the bookmark feature is working, then the height should decrease. This determines the correctness of the test. We follow the same approach on the rooms pages too. While this is not absolutely correct, it is a good way to check the feature. We have already employed the perfect method on the tracks page so there was no need of applying it on the schedule and the rooms page since it would have increased the time of the testing by a quite large margin.

Here is an excerpt of the code. The whole work can be viewed here

RoomPage.checkIsolatedBookmark = function() {
 // We go into starred mode and unmark sessions having id 3015 which was marked previously on tracks pages. If the bookmark feature works, then length of the web page would decrease. Return true if that happens. False otherwise
 var getPageHeight = 'return document.body.scrollHeight';
 var sessionIdsArr = ['3015'];
 var self = this;
 var oldHeight, newHeight;
 return self.toggleStarredButton().then(function() {
   return self.driver.executeScript(getPageHeight).then(function(height) {
     oldHeight = height;
     return self.toggleSessionBookmark(sessionIdsArr).then(function() {
       return self.driver.executeScript(getPageHeight).then(function(height) {
         newHeight = height;
         return oldHeight > newHeight;
       });
     });
   });
 });
};

Search Feature
Now, let us go to the testing of the search feature in the webapp. The main object of focus is the Search bar. It is present on all the pages: tracks, rooms, schedule, and speakers page and allows the user to search for a particular session or a speaker and instantly fetches the result as he/she types.

We want to ensure that this feature works across all the pages. Tracks, Rooms and Schedule pages are similar in a way that they display all the session of the event albeit in a different manner. Any query made on any one of these pages should fetch the same number of session elements on the other pages too. The speaker page contains mostly information about the speakers only. So, we make a single common test for the former three pages and a little different test for the latter page.

Designing a test for this feature is interesting. We want it to be fast and accurate. A simple way to approach this is to think of the components involved. One is the query text which would be entered in the search input bar. Other is the list of the sessions which would match the text entered and will be visible on the page after the text has been entered. We decide upon a text string and a list containing session ids. This list contains the id of the sessions should be visible on the above query and also contain few id of the sessions which do not match the text entered. During the actual test, we enter the decided text string and check the visibility of the sessions which are present in the decided list. If the result matches the expected order, then it means that the feature is working well and the test passes. Otherwise, it means that there is some problem with the default implementation and the test fails.

For eg: We decide upon the search text ‘Mario’ and then note the ids of the sessions which should be visible in that search.

c0e4910f-cf69-4b2a-8cc1-233badb35eee.png

Suppose the list of the ids come out to be

['3017''3029''3013''3031']

We then add few more session ids which should not be visible on that search text. Like we add two extra false ids 3014, 3015. Modified list would be something like this

['3017''3029''3013''3031''3014''3015']

Now we run the test and determine the visibility of the sessions present in the above list, compare it to the expected output and accordingly determine the fate of the test.

Expected: [truetruetruetruefalsefalse]
Actual Output: [truetruetruetruetruetrue]

Then the test would fail since the last two sessions were not expected to be visible.

Here is some code related to it. The whole work can be seen here

function commonSearchTest(text, idList) {
 var self = this;
 var searchText = text || 'Mario';
 // First 4 session ids should show up on default search text and the last two not. If no idList provided for testing, use the idList for the default search text
 var arrId = idList || ['3017', '3029', '3013', '3031', '3014', '3015'];
 var promise = new Promise(function(resolve) {
   self.search(searchText).then(function() {
     var promiseArr = arrId.map(function(curElem) {
       return self.find(By.id(curElem)).isDisplayed();
     });

     self.resetSearchBar().then(function() {
       resolve(Promise.all(promiseArr));
     });
   });
 });
 return promise;
}

Here is the code for comparing the expected and the actual output. You can view the whole file here

it('Checking search functionality', function(done) {
 schedulePage.commonSearchTest().then(function(boolArr) {
   assert.deepEqual(boolArr, [true, true, true, true, false, false]);
   done();
 }).catch(function(err) {
   done(err);
 });
});

The search functionality test for the speaker’s page is done in the same style. Just instead of having the session ids, we work with speaker ids there. Rest everything is done in a similar manner.

Resources:

Continue ReadingWriting Selenium Tests for Checking Bookmark Feature and Search functionality in Open Event Webapp

Keep your node server running using PM2

The open event webapp generator is a node projects (using an express server), and a copy of it runs all the time on my personal DigitalOcean box (other than our heroku instance).

On a service like Heroku, the platform manages the task of bringing your server process up. But on, say a Linux distro on the DO box, you have to manually do

 npm run server

to be able to run the server.

While that is all good, it is a foreground shell process, which means, you will lose the node server, when you log out (or your internet connection into the ssh breaks).
So we need to be able to keep the process running in the background.

The way we do it in bash on Unix, is that we can do either of the following

 npm run server&

The “&” at the end means it will make a background fork of this task. Or if you’ve already started it without it, you can also do the following.

npm run server # starts in foreground
#Press Ctrl + Z, this pauses task and frees the shell
bg 1 # sends task no 1 to background thread.

Again, both these are hacky methods, will work only on Unix OSs, and are not really recommended for production.
For production, we need a Process Manager, for Node.js the best we can get is pm2 – purpose built process manager for node.

Install pm2 first

sudo npm install -g pm2

Using pm2, we can start any process that can be started with node. We can start the app.js script like this

pm2 start src/app.js

Also, pm2 can run npm tasks too like

pm2 start npm -- start

Pm2 has a pretty status message display window. And we can start, stop, pause, kill and/or restart any process.

 

Screenshot from 2016-08-28 01-19-29

Continue ReadingKeep your node server running using PM2

sTeam GSoC 2016 Windup

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST

An overview of the work done by ajinkya007 during Google Summer of code 2016 with FOSSASIA on its project sTeam.

The community bonding period saw the creation of a docker image and a debian package for the sTeam server. The integration of the sTeam shell into vi, improvements in the export and import to git scripts, user and group manipulation commands, sending mails through the commandline, viewing logs and the edit script modifications were done subsequently. The later part of GSOC saw that the sTeam-rest repository was restructured, unit and api-end point tests were performed. The new web interface developed was tested.
The code written during this period by me and siddhant was merged and the conflicts were resolved. The merged code was tested thoroughly as no automated test integration tool supports pike programming language. Documentation was generated using Doxygen and deployed in the gh-pages of the sTeam server repository.

A trello board was maintained throughout the course of GSOC 2016.

Trello Board: sTeam

Accomplishments

Issues Reported and Resolved

A list of tasks covered and all the Pull requests related to each:

Tasks Issue PR
Make changes in the Makefile for installation of sTeam. Issue-25 Issue-27 PR-66 PR-67
Edit script modifications Issue-20 Issue-29 Issue-43 PR-44 PR-48
Indentation of output in steal-shell. Issue-24 PR-42
Integrate steam-shell into vim or emacs. Issue-37 Issue-43 Issue-49 PR-41 PR-48 PR-51
Improve the import and export from git scripts. Issue-9 Issue-14 Issue-16 Issue-18 Issue-19 Issue-46 PR-45 PR-54 PR-55 PR-76
Create, Delete and List the user through commandline Issue-58 Issue-69 Issue-72 PR-59 PR-70 PR-78
Sending Mails through commandline Issue-74 PR-85
Generate error logs and display them in CLI Issue-83 PR-86
Create a file of any mime type from command line. Issue-79 PR-82
Add more commands for group operations. Issue-80 PR-84
Add more utility to the steam-shell Issue-56 Issue-71 Issue-73 PR-57 PR-75 PR-81
Restructure the sTeam-rest repository List of Issue’s List of PR’s
Write test cases to test sTeam-rest api List of Issue’s List of PR’s
Create a debian package and a docker image for easy deployment Create docker image Docker Image
Document the work done Issue 149 sTeam Server Structure, sTeam Server Documentation
Test the web-interface

Commits Merged

During the course of GSOC 2016, work was done on the sTeam and sTeam-rest repositories.

1. The work done on the sTeam repository.

We have combined all the work into two branches for the ease of creating a debian package. The commits made by me in each branch can be seen here.

2. The work done on the sTeam-rest repository

The push request’s sent for the issue’s are yet to be merged in the main repository. The list of PR’s for the sTeam-rest repository.

sTeam-rest PR’s

The weekly blogs

The blogs summarizing the work done during the week were published on my personal website. These can be found on Weekly Blogs
All the blogs can also be found on the Fossasia blog.
The list in reverse chronological order is as follows.

Scrums

Scrum reports were posted on the #steam-devel on irc.freenode.net and sTeam google group. The sTeam trello board also has everyday scrum reports.

Further Improvements

  1. sTeam command line lacks the functionality to read and set the object access permissions. sTeam function analogous to getfacl() to change the sTeam server object permisssions.
  2. sTeam debian package for easy installation of the sTeam server. The debian package is yet to be fully packaged.

Special Thanks

  • I would like to thank my mentors Mario Behling, Hong Phuc Dang, Martin Bahr, Trilok Tourani and my peers for being there to help me and guide me.
  • I would like to thank FOSSASIA, sTeam and Pike Community for giving me this opportunity and guiding me in this endeavour.
  • I would also like to thank Google Summer of Code for this experience.

Feel free to explore the repository. Suggestions for improvements are welcomed.

Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingsTeam GSoC 2016 Windup

File upload progress in a Node app using Socket.io

If you look at the webapp generator, you’ll see that there is an option to upload a zip file containing event data. We wanted to give visual cue to the user when he is uploading to see how much file has uploaded.

We are uploading the file, and giving the generate start command via socket.io events instead of POST requests here.

To observe file upload progress on socket (when sending file using a Buffer), there is an awesome node module available called socketio-upload-progress.

In our webapp you can see we implemented it on the frontend here in the form.js and here in the backend in app.js

Basically on the backend you should add the socketio-file-upload module as a middleware to express

var siofu = require("socketio-file-upload");
var app = express()
    .use(siofu.router)
    .listen(8000);

After a socket is opened, set up the upload directory and start listening for uploads

io.on("connection", function(socket){
    var uploader = new siofu();
    uploader.dir = "/path/to/save/uploads";
    uploader.listen(socket);
});

On the frontend, we’ll listen for an input change on an file input type element whose id is siofu_upload

var socket = io.connect();
var uploader = new SocketIOFileUpload(socket);
uploader.listenOnInput(document.getElementById("siofu_input"));

One thing to note here is that, if you observe percentage of upload on frontend, it’ll give you false values. The correct values of how much data is actually transferred can be found in the backend. So observe progress in backend, and send percentage to frontend using the same socket.

  uploader.on('progress', function(event) {
    console.log(event.file.bytesLoaded / event.file.size)
    socket.emit('upload.progress', {
      percentage:(event.file.bytesLoaded / event.file.size) * 100
    })
});

 

Continue ReadingFile upload progress in a Node app using Socket.io

Using compression middleware in Node/Express

If you’re using ExpressJS to host your server, there’s a small performance tweak that you can do (works on universally on any kind of server – HTML templates to JSON REST APIs to image servers) – which is to enable GZIP compression.

The changed required to do it should be very simple. First install the compression npm package

 npm install --save compression

In you app.js (or wherever you start your express server), modify the code to include the compression middleware like this

var express = require('express);
var app = express();

var compression = require('compression');

app.use(compression());

// Rest of your app code
// app.get (endpoint, callback) 
// app.listen(port, callback)

This should easily reduce page loads time to the order of 15-20%.

For example, page load before compression middleware added 72f7abf2-4899-11e6-9cd5-68a7addaf3a6

 

And Page load time after compression is added69e6fba8-4899-11e6-9626-a25c26ea7d2b

Continue ReadingUsing compression middleware in Node/Express