In Open Event Webapp, we use Socket.IO library for real-time communication between client and the server. To be able to generate a site for an event, the user has to first upload the file to the generator. There are a lot of node libraries like multer and formidable which exists for this purpose. But they don’t support display of real-time stats regarding the progress of the uploaded file. To solve this issue, the project used socket-io-file-upload library which uploads the file to the Node.js server using Socket.IO and show live percentage denoting how much of the zip has been uploaded to the server.

It was working quite well until we discovered a major problem with the library. It didn’t support canceling the upload. If we clicked on the cancel button, it stopped showing the progress on the front end but actually continued to upload the file to the server on the back end. We had only two choices: Either to refresh the page or just wait for the existing zip file to upload completely. The former is a really bad solution and the latter result in wastage of time and bandwidth.

On investigating the issue, the problem was with how the library is designed. This was not a fault in our code or the library either. It was just that the library didn’t support this functionality.  After thoroughly searching for the solutions, we came across this library named socket-io-stream which met our requirements. It supports bidirectional binary data transfer with Stream API through Socket.IO. We can also cancel the upload in between.

For streaming between server and client, we will send stream instances first. To receive streams, we just wrap socket with socket-io-stream and then listen for any events as usual.

Client Side:

function uploadFile(file) {
  // Calculating size of the file and creating a stream to be sent to the server

  var size = (file.size/(1024*1024)).toString().substring(0, 3);
  var stream = ss.createStream();

  /* Creating a read stream and initializing variable to keep track of the size of the file uploaded so far */

  var blobStream = ss.createBlobReadStream(file);
  var fileUploadSize = 0;

  // Tracking upload progress and updating the variables

  blobStream.on('data', function(chunk) {
    fileUploadSize += chunk.length;
    var percent = (fileUploadSize / file.size * 100);

    /* isCancelling is a boolean which stores the status of the zip upload.That is, whether it is live or canceled */

      if (isCancelling) {
           var msg = 'File upload was cancelled by user';
           stream.destroy();
           socket.emit('cancelled', msg);
           isCancelling = false;
           console.log(msg);
       }

       updateUploadProgress(Math.floor(percent));
       if (percent === 100) {
           uploadFinished = true;
           enableGenerateButton(true);
       }
     });

  // Sending the stream to the server and transferring read data to it

   ss(socket).emit('file', stream, {size: file.size, name: file.name});
   blobStream.pipe(stream);
}

Server Side:

// Importing the library
const ss = require('socket.io-stream');

// Listening to the stream sent from the client
ss(socket).on('file', function(stream, file) {
  generator.startZipUpload(socket.connId);

  // Declare a filename and write the incoming data to it
  var filename = path.join(__dirname, '..', 'uploads/connection-' + id.toString()) + '/upload.zip';

  stream.pipe(fs.createWriteStream(filename));
});

That’s all we need to do to stream data from the client to the server. Head over here for more information regarding the library.