How to prevent race conditions in a distributed nodejs web servers architecture backed by mongodb

Problem

Here' s the description of my problem:

  1. I have x web workers written in nodejs (express.js) behind a load ballancer. These workers write data into mongodb (mongoose.js)
  2. I've setup endpoint y such that at some point in the middleware chain of the handler I'm doing the following logic: If the requesting user exists in the database, then fetch it, update some fields then store it back. If it doesn't exist, insert it in mongo.

NOTE! I'm unable to use Mongoose's findAndUpdateOne() - which would be atomic - due to domain specific logic, ie. if the user already exists update it with a certain value, else insert it with another value.

  1. The problem is, quite often, two requests from the same user (who doesn't yet exist in the db) will hit two different workers. When user processing middleware kicks in, both workers will determine that the user doesn't exist and attempt to insert it then update it. Naturally this causes errors, eg. validation errors: I've setup a unique email per user validation and others.

How can I prevent this?

Problem courtesy of: alexandru.topliceanu

Solution

You can use $set and $setOnInsert update operators to achieve that.

From MongoDB $setOnInsert docs:

If an update operation with upsert: true results in an insert of a document, then $setOnInsert assigns the specified values to the fields in the document. If the update operation does not result in an insert, $setOnInsert does nothing.

and

If the db.collection.update() with upsert: true had found a matching document, then MongoDB performs an update, applying the $set operation but ignoring the $setOnInsert operation.

so, this should do the trick:

var now = Date.now();

Users.update({ _id: 1 }, {
  $set:         { updatedAt: now },
  $setOnInsert: { createdAt: now }
}, {upsert: true}, function(err, doc) {

  // resultant doc is available here

})
Solution courtesy of: Ezequias Dinella

Discussion

There is currently no discussion for this recipe.

This recipe can be found in it's original form on Stack Over Flow.