Status Badges for Repositories Registered to Yaydoc

Yaydoc, our automatic documentation generation and deployment project, generates and deploys documentation for each of its registered repository at every commit. It is possible that due to any misconfiguration in users’ project the build process may fail. Hence, it is vital for an improved user experience to store the build status for at least the most recent build process.

There are different ways with which a user can be notified about the build status after a commit. At Yaydoc, we chose to notify the user by emailing a status report. Since sending an email at each at every commit can be quite annoying, we chose to limit it to specific scenarios. We decided that we will send the mail

  • On the first build after the repository is registered to Yaydoc, irrespective of the status
  • On every failed build
  • On the change of build status (Success to Failed or vice versa)
  • To the user who registered the repository to Yaydoc
exports.updateBuildStatus = function (name, buildStatus) {
  async.waterfall([
    function (callback) {
      Repository.setBuildStatusToRepository(name, buildStatus, 
      function (error, repository) {
        callback(error, repository);
      });
    },
    function (repository, callback) {
      if (repository.mailService === true && 
          (repository.buildStatus === false || buildStatus === false || 
           repository.buildStatus === undefined)) {
        User.getUserByUsername(repository.registrant.login, 
        function (error, user)) {
          callback(null, user, repository);
        }
      }
    }
  ], function (error, user, repository) {
    mailer.sendMailOnBuild(buildStatus, user.email, repository);
  });
};

Considering the fact that the user may not wish to receive build emails and hence made them configurable by adding a mailService: Boolean  key in repository’s collection.

Taking this forward, we then decided to generate status badges similar to how Travis and other Continuous Integration platform do. Since we now store a `buildStatus` for each repository, we can use it to generate an svg image to be added to README files of repositories. We generated  the status badges using Shields.io and added them to the route /<owner>/<reponame>.svg.  The dynamicity of image generated is achieved by retrieving the value of buildStatus and render the images with different constructs based on its value.

router.get(‘/:owner/:reponame.svg’, function (req, res, next) {
  var fullName = req.params.owner + ‘/’ + req.params.reponame;
  Repository.getBuildStatusByRepositoryName(fullName, function(error, result)) {
    var buildStatus = ‘invalid’; var width =94’; 
    var color = ‘#9f9f9f’; var x =70.5’;
    
    if (result.buildStatus) {
      buildStatus = ‘success’; width =102’; color = ‘#97CA00’; x =74.5’;
    } else {
      buildStatus = ‘failed’; width =88’; color = ‘#E05d44’; x =67.5’;
    }

    res.set(‘Content-Type’, ‘image/svg+xml’);
    res.render(“badge”, {
      status: buildStatus,
      width: width,
      color: color,
      x: x,
    });
  }
});

The status tags generated can then be added as:

[![Yaydoc Status] (https://yaydoc.herokuapp.com/imujjwal96/prelimQuiz.svg)] (https://yaydoc.herokuapp.com/imujjwal96/prelimQuiz)

Resources:

  1. Shields.io: Quality metadata badges for open source projects – https://shields.io
  2. Async utilities for node and browser – https://caolan.github.io/async/

Display a List of Registered Repositories in User’s Dashboard

Yaydoc, our automatic documentation generation and deployment project was recently introduced with the dashboard for its registered users. Introducing a dashboard to Yaydoc enabled us to include a lot of new functionalities and processes associated with users and their registered repositories.

One of the main reasons of creating a dashboard for registered users was to enable them to access and edit configurations of the repositories they registered to Yaydoc. Hence, there was a need to display all of the registered repositories to users. These repositories include the public repositories owned by the users and organizations’ repositories the user has admin access to.

The organizations and users are retrieved separately in parallel using async.js. Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. The required functionality is achieved using async.parallel() method which runs the tasks of a collection of functions in parallel, without waiting until the previous function has completed.

async.parallel({
  organizations: function (callback) {
    github.retrieveOrganizations(user.token, function (error, organizations)) {
      callback(null, organizations);
    }
  },
  ownedRepositories: function (callback) {
    Repository.getRepositoriesByOwner(user.username, function (err, repositories)) {
      callback(null, repositories);
    }
  }, function (err, results) {
    res.render(‘dashboard’, {
      title: ‘Dashboard’,
      organizations: results.organizations,
      ownedRepositories: results.ownedRepositories
    });
  }
});

In order to show the organizations’ registered repositories  the github.retrieveOrganizations() function is updated, retrieving registered repositories from Yaydoc’s database. To perform the operation, we use async’s waterfall method. This method runs the tasks in series, passing the result of each function to the next function in that array. This function is needed since we are retrieving the organizations first and then for each organization, we retrieve all the repositories for each organization. To access each organization asynchronously, we use the async.forEachOf() method.

/**
 * Retrieve a list of organization the user has access to
 * Along with the organizations, also retrieve the registered repositories
 * @param accessToken: Access token of the user
 * @param callback
 */
exports.retrieveOrganizations = function (accessToken, callback) {
  async.waterfall([
    function (callback) {
      // Retrieve organizations
      ....
      organizations.push({
        id: organization.id,
        name: organization.login,
        avatar: organization.avatar_url
      });
      return callback(null, organizations);
    },
    function (organizations, callback) {
      async.forEachOf(organizations, function (value, key, callback)) {
        Repository.getRepositoriesByOwner(value.name, function (error, repositories) {
          value.repositories = repositories;
          update.push(value);
          callback()
        });
      }, function (err) {
        callback(null, update);
      }
    }
  ], function (error, result) {
    callback(error, result);
  });
};

Showing the list of repositories, we now plan to make these repositories configurable. Some of these configurations may include deleting the repository and enabling or disabling it from the build process.

Resources

  1. Async utilities for node and browser – https://caolan.github.io/async/
  2. Github’s organization API https://developer.github.com/v3/orgs/

Doing asynchronous tasks serially using ‘async’ in node.js

In the open-event-webapp generator we need to perform a lot of asynchronous tasks in the background like –

  • Downloading images and audio assets
  • Downloading the jsons from the endpoints
  • Generating the html from handelbar templates
  • and so on . .

Sometimes tasks depend on previous tasks, and in such cases we need to perform them serially. Also there are tasks like image downloads, that would be better if done parallelly.

To achieve both these purposes, there is an awesome node.js library called async that helps achieve this.

To perform asynchronous tasks serially (one task, then another task), we can use something like this –

 

async.series([
    (done) => {
       someAsyncFunction(function () { done () })
    },
    //(done) => {..}, (done) => {..} more tasks here
    (done) => {
       someAsyncFunction(function () { done () })
    });
      
    }
]);

Basically async takes an array of functions. Each function contains a callback that you need to call when the internal task is finished. The 2nd task starts, only after the done() callback of first task is executed.

An example of it’s usage can be seen in the open-event-webapp project here