Upload CSV to Heroku via AJAX - Node/Express

Problem

I have a database I would like users to be able to update using CSV files. I'm trying to use the Postgres COPY function to do this but haven't gotten it to work.

HTML:

<h2>Mass Update</h2>
  <form enctype="multipart/form-data" id="mass">
    <input type="file" id="file" name="upload"/><br/>
    Replace <input type="checkbox" name="replace"/>
  </form>
<button id="trigger-mass">Submit</button>

jQuery

$('#trigger-mass').on('click', function(){
var fd = new FormData(document.getElementById('mass'));
$.ajax({
    url: base + table,
    processData:false,
    data:fd,
    type:'POST'
});

Node

index.js

 app.use('/', express.static(__dirname + '/static'));
 app.use(express.bodyParser({uploadDir:__dirname +'/static/tmp', keepExtensions:true}));

This may not seem best practice, but it is ok in my case as I want these uploads to be available for download. I was receiving Error:ENOENT open"..." if the tmp dir was outside the static dir.

routes.js

 // Clear table and refill with data
 app.post('/data/:table', auth, function(req, res) {
    var oMsg = quickRes(res, req);
    console.log('REQ DATA')
    console.log(req.body);
    console.log(req.files);
...// Here is where I would issue the COPY command through pg
 });

Unfortunately, I haven't had anything show up in req.files. I've also tried just directly uploading the file in AJAX as data:document.getElementById('file').files[0] to no avail. Any suggestions?

Problem courtesy of: Eric H.

Solution

You need to explicitly set contentType to false in $.ajax():

$.ajax({
  url         : base + table,
  processData : false,
  contentType : false, // <-- required
  data        : fd,
  type        : 'POST'
});

Otherwise, jQuery will set it to application/x-www-form-urlencoded; charset=UTF-8, which isn't appropriate for file uploads.

Solution courtesy of: robertklep

Discussion

Could we get a look at the middleware you've included and what version of express you're using? express.multipart will be deprecated in the next version so you have to use an alternative solution. The general consensus thus far is to use the connect-busboy npm package as a middleware for file uploads.

Example index.js:

app.use(express.json()); // These replace the bodyParse middleware due to the tmp folder vulnerability
app.use(express.urlencoded());
app.use(busboy());

app.post('/data/:table', auth, function (req, res) {
    var fstream;
    req.pipe(req.busboy);
    req.busboy.on('file', function (fieldname, file, filename) {
        fstream = fs.createWriteStream(__dirname + '/tmp/' + filename);
        file.pipe(fstream);
        fstream.on('close', function () {
            res.redirect('/'); // Or wherever you want to go next
        });
    });
});

Included node packages would be express, fs, and connect-busboy. See this Github Issue to confirm the deprecation of multipart.

Discussion courtesy of: Deif

This recipe can be found in it's original form on Stack Over Flow.