Express.js: Handling file uploads

How to handle file uploads in Express#

I briefly touched the subject of file uploads in Express in "Express.js: Handling / processing forms". Let's revisit that usecase and others in a more detailed manner.

We will be using the multer Node module for handling file uploads. Make sure to install it in your project directory.

$ npm i multer

1. A basic file uploader and handler#

Let's start with a very simple scenario where we want to upload a single file.

The form:

<form method="post" action="/upload" enctype="multipart/form-data">
  <input type="file" name="wallpaper" />
  <input type="submit" />
</form>

The app:

var path = require('path');
var express = require('express');
var app = express();
var multer  = require('multer');

var storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, './public/images/');
  },
  filename: function (req, file, cb) {
    cb(null, Date.now() + file.originalname);
  }
});

var upload = multer({ storage: storage });

app.use(express.static(path.join(__dirname, 'public')));

app.post('/upload', upload.single('wallpaper'), function (req, res) {
  var imagePath = req.file.path.replace(/^public\//, '');
  res.redirect(imagePath);
});

app.use(function (err, req, res, next) {
  if (err instanceof multer.MulterError) res.status(500).send(err.message);
  else next(err);
});

app.listen(5000);

The URL: http://localhost:5000/upload.html

On line number 6, we are defining the storage engine for multer, of the type multer.diskStorage.

One may wonder, "Shouldn't handling file uploads be just about defining a path for the uploaded files? Why do we need to define a storage? Isn't this obvious over-engineering?"

Contrary to what many may think about file uploads, handling file uploads is not a simple task. Consider the following:

• What if another file with the same name is uploaded?
• What if we want to rename the uploaded files?
• What if we want to detect the file type or the file extension?
• What if we want to save files to different directories based on some criteria?
• What if we want to ensure we allow uploads only with our predefined form field names?
• What if we want to reject an upload?
• What if we want to analyze the uploading file before saving it?
• What if we want to re-upload the file to a cloud storage service or a CDN?

The use of storage engines makes handling of all those scenario and more, very easy.

On line number 7, we are definiting the destination function of the storage. This defines where the uploaded file will be stored. In our case all the files will be uploaded to ./public/images/. It can be programmed to store the files to different directories conditionally, if required.

On line number 10, we are defining the filename function of the storage. This function takes care of renaming the uploaded file.

Then, on line number 19, we are plugging in the multer middleware for handling the file upload, which is configured to accept only one file using the form field named "wallpaper". If there are more than one file field or the file field is not named "wallpaper", multer will throw a MulterError: Unexpected field error. Text fields are exempted from these errors.

On line number 24, we define an error handling middleware to help us catch error thrown by multer.

multer allows you to predefine the names of the fields and how many files you are expecting in each upload. It may sound like an inconvenience, but it helps to prevent unmonitored and unwanted file uploads which can eat up your disk space.

2. Uploading multiple files#

There will be situations where you want to allow more than one file to be uploaded in a go. Let's see how we can take care of those requirements.

We will rewrite the /upload handler according to our needs.

When you want to allow more than one file to be uploaded using the same field:

app.post('/upload', upload.array('wallpaper', 3), function (req, res) {
  console.log(req.files);
  res.send(req.files);
});

We use the upload.array(fieldName[, maxCount]) middleware instead of upload.single(fieldName). The optional maxCount parameter determines the number of files that can be uploaded. If the number of files exceeds the maxCount value, multer will throw a MulterError: Unexpected field error.

When you want to allow more than one field to be used in one upload session:

var fields = [
  { name: 'cover', maxCount: 1 },
  { name: 'wallpaper', maxCount: 3 }
];

app.post('/upload', upload.fields(fields), function (req, res) {
  console.log(req.files);
  res.send(req.files);
});

We use the upload.fields(fields) middleware, where fields is an array of objects with name and optional maxCount property.

3. Creating a custom storage engine#

So far, we have been using the multer.diskStorage storage engine. It serves us well if we want to store the uploaded files on the same machine as the application. What if we want to upload the files somewhere else? Like a cloud storage service or a CDN?

We will have to write our own storage engine.

We won't be building a cloud storage engine today, due to their elaborate nature, instead we will create an example storage engine that writes all files to /dev/null, essentially deleting them on arrival.

var fs = require('fs');

function getDestination (req, file, cb) {
  cb(null, '/dev/null');
}

function NullStorage (opts) {
  this.getDestination = (opts.destination || getDestination);
}

NullStorage.prototype._handleFile = function _handleFile (req, file, cb) {
  this.getDestination(req, file, function (err, path) {
    if (err) return cb(err);

    var outStream = fs.createWriteStream(path);

    file.stream.pipe(outStream);
    outStream.on('error', cb);
    outStream.on('finish', function () {
      cb(null, {
        path: path,
        size: outStream.bytesWritten
      });
    });
  });
}

NullStorage.prototype._removeFile = function _removeFile (req, file, cb) {
  fs.unlink(file.path, cb);
}

module.exports = function (opts) {
  return new NullStorage(opts);
}

Now, app.js can be re-written to use our NullStorage storage engine.

var path = require('path');
var express = require('express');
var app = express();
var multer  = require('multer');
var NullStorage = require('./null-storage');

var storage = NullStorage({
  destination: function (req, file, cb) {
    cb(null, './public/images/');
  }
});

var upload = multer({ storage: storage });
...

4. Rejecting an upload#

A file upload can be rejected either by the filename function or the destination function of multer.diskStorage. Just call the callback function with an error message or object as the first parameter.

In this example we reject the upload if the file type if not JPEG.

var storage = multer.diskStorage({
  destination: function (req, file, cb) {
    if (file.mimetype !== 'image/jpeg') return cb('Invalid image format');
    cb(null, './public/images/');
  },
  filename: function (req, file, cb) {
    cb(null, Date.now() + file.originalname);
  }
});

5. Forms with text and images#

Handling forms with text and images require no additional configuration, if your app is already configured for handling images. The text data is available on the req.body object.

Here is an example of a form for submitting text data and uploading an image.

The form:

<form method="post" action="/upload" enctype="multipart/form-data">
  <input type="text" name="title" /><br />
  <input type="file" name="wallpaper" /><br />
  <input type="submit" />
</form>

The app:

var path = require('path');
var express = require('express');
var app = express();
var multer  = require('multer');

var storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, './public/images/');
  },
  filename: function (req, file, cb) {
    cb(null, Date.now() + file.originalname);
  }
});

var upload = multer({ storage: storage });

app.use(express.static(path.join(__dirname, 'public')));

app.post('/upload', upload.single('wallpaper'), function (req, res) {
  // Text data from the form
  console.log(req.body);
  // Details about the uploaded file
  console.log(req.file);
  var imagePath = req.file.path.replace(/^public\//, '');
  res.redirect(imagePath);
});

app.listen(5000);

The URL: http://localhost:5000/upload.html

Summary#

We can use the multer Node module for handling file uploads in Express. It comes with an in-build diskStorage engine which makes setting the upload destination, renaming the file, and rejecting the upload very easy. We can create custom storage engines for multer to re-upload the file to a remote services.

References#

1. multer
2. multer - Multer Storage Engine
3. W3C - multipart/form-data
4. W3C -Multipart Content-Type