Continuous Integration in Yaydoc using GitHub webhook API

In Yaydoc,  Travis is used for pushing the documentation for each and every commit. But this leads us to rely on a third party to push the documentation and also in long run it won’t allow us to implement new features, so we decided to do the continuous documentation pushing on our own. In order to build the documentation for each and every commit we have to know when the user is pushing code. This can be achieved by using GitHub webhook API. Basically we have to register our api to specific GitHub repository, and then GitHub will send a POST request to our API on each and every commit. “auth/ci” handler is used to get access of the user. Here we request user to give access to Yaydoc such as accessing the public repositories , read organization details and write permission to write webhook to the repository and also I maintaining state by keeping the ci session as true so that I can know that this callback is for gh-pages deploy or ci deployOn On callback I’m keeping the necessary informations like username, access_token, id and email in session. Then based on ci session state, I’m redirecting to the appropriate handler. In this case I’m redirecting to “ci/register”.After redirecting to the “ci/register”, I’m getting all the public repositories using GitHub API and then I’m asking the users to choose the repository on which users want to integrate Yaydoc CI After redirecting to the “ci/register”, I’m getting all the public repositories using GitHub API and then I’m asking the users to choose the repository on which users want to integrate Yaydoc CI router.post('/register', function (req, res, next) { request({ url: `https://api.github.com/repos/${req.session.username}/${repositoryName}/hooks?access_token=${req.session.token}`, method: 'POST', json: { name: "web", active: true, events: [ "push" ], config: { url: process.env.HOSTNAME + '/ci/webhook', content_type: "json" } } }, function(error, response, body) { repositoryModel.newRepository(req.body.repository, req.session.username, req.session.githubId, crypter.encrypt(req.session.token), req.session.email) .then(function(result) { res.render("index", { showMessage: true, messages: `Thanks for registering with Yaydoc.Hereafter Documentation will be pushed to the GitHub pages on each commit.` }) }) }) } }) After user choose the repository, they will send a POST request to “ci/register” and then I’m registering the webhook to the repository and I’m saving the repository, user details in the database, so that it can be used when GitHub send request to push the documentation to the GitHub Pages. router.post('/webhook', function(req, res, next) { var event = req.get('X-GitHub-Event') if (event == 'Push') { repositoryModel.findOneRepository( { githubId: req.body.repository.owner.id, name: req.body.repository.name } ). then(function(result) { var data = { email: result.email, gitUrl: req.body.repository.clone_url, docTheme: "", } generator.executeScript({}, data, function(err, generatedData) { deploy.deployPages({}, { email: result.email, gitURL: req.body.repository.clone_url, username: result.username, uniqueId: generatedData.uniqueId, encryptedToken: result.accessToken }) }) }) res.json({ status: true }) } }) After you register on webhook, GitHub will send a request to the url which we registered on the repository. In our case “https:/yaydoc.herokuapp.com/ci/auth” is the url. The type of the event can be known by reading 'X-GitHub-Event' header. Right now I’m registering only for the push event. So…

Continue ReadingContinuous Integration in Yaydoc using GitHub webhook API

Using compression middleware in Node/Express

If you're using ExpressJS to host your server, there's a small performance tweak that you can do (works on universally on any kind of server - HTML templates to JSON REST APIs to image servers) - which is to enable GZIP compression. The changed required to do it should be very simple. First install the compression npm package npm install --save compression In you app.js (or wherever you start your express server), modify the code to include the compression middleware like this var express = require('express); var app = express(); var compression = require('compression'); app.use(compression()); // Rest of your app code // app.get (endpoint, callback) // app.listen(port, callback) This should easily reduce page loads time to the order of 15-20%. For example, page load before compression middleware added   And Page load time after compression is added

Continue ReadingUsing compression middleware in Node/Express