File upload progress in a Node app using Socket.io

If you look at the webapp generator, you’ll see that there is an option to upload a zip file containing event data. We wanted to give visual cue to the user when he is uploading to see how much file has uploaded.

We are uploading the file, and giving the generate start command via socket.io events instead of POST requests here.

To observe file upload progress on socket (when sending file using a Buffer), there is an awesome node module available called socketio-upload-progress.

In our webapp you can see we implemented it on the frontend here in the form.js and here in the backend in app.js

Basically on the backend you should add the socketio-file-upload module as a middleware to express

var siofu = require("socketio-file-upload");
var app = express()
    .use(siofu.router)
    .listen(8000);

After a socket is opened, set up the upload directory and start listening for uploads

io.on("connection", function(socket){
    var uploader = new siofu();
    uploader.dir = "/path/to/save/uploads";
    uploader.listen(socket);
});

On the frontend, we’ll listen for an input change on an file input type element whose id is siofu_upload

var socket = io.connect();
var uploader = new SocketIOFileUpload(socket);
uploader.listenOnInput(document.getElementById("siofu_input"));

One thing to note here is that, if you observe percentage of upload on frontend, it’ll give you false values. The correct values of how much data is actually transferred can be found in the backend. So observe progress in backend, and send percentage to frontend using the same socket.

  uploader.on('progress', function(event) {
    console.log(event.file.bytesLoaded / event.file.size)
    socket.emit('upload.progress', {
      percentage:(event.file.bytesLoaded / event.file.size) * 100
    })
});

 

Handling file uploading in HTML5 and Express

The Open Event Webapp Generator has a pure HTML front end form, and Express based backend.

 

We needed an option where users can upload their own JSON files to create a schedule page for it.

On the HTML form that’s easy. As you can see here we simple add input tags with type=”file”


<input type="file" name="sponsorfile" id="sponsorfile">

In our express app, we need to use multer to be able to handle file uploads.

We create a middleware called uploadedFiles, and pass the middleware to the get() block

 

var express = require('express');
var multer = require('multer');
var app = express();

var upload = multer({dest: 'uploads/'});

var uploadedFiles = upload.fields([
  {name: 'speakerfile', maxCount: 1},
  {name: 'sessionfile', maxCount: 1},
  {name: 'trackfile', maxCount: 1},
  {name: 'sponsorfile', maxCount: 1},
  {name: 'eventfile', maxCount: 1},
  {name: 'locationfile', maxCount: 1}
]);

app.post('/live', uploadedFiles, function(req, res) {
         // req.files has the files
         // req.body has the POST body params
  });
});

Now we can access the files inside req.files (that will have a path to the temporary location of the file, you’ll have to use some filesystem module to read the files)

File Uploading in Flask

Last week I took up an issue of adding upload functionality to the open-event server. I had to implement the upload in 3 places – one in the sponsor table to upload images, another in the user profile page and the third is to upload slides in the session form. However the basic function behind it remains the same. File upload makes use of the inbuilt werkzeug FileStorage class. A code snippet will help understand how it works:

Capture.PNG

So ¬†on selecting the file it gets stored in the request.files object. It is temporarily stored in the FileStorage in werkzeug. Now we access the file’s name using request.files[‘files’] and then using the inbuilt save() ¬†function in flask it gets saved to the folder specified by us. There are some frameworks available for file uploading in Flask but all this can be done using the standard libraries also and there is no such need of the frameworks.

However instead of storing just the direct name of the file we make use of the secure_filename of werkzeug to save it.Capture2.PNG

Thus the secure_filename stores the file in the format given in the image and makes uploading easier by converting the long url/image path to an easier one.