How to develop node.js run-time strategy?

Problem

Node.js approach is event driven and I was wondering how would you tackle the problem of when to fire off an event?

Lets say that we have some actions on a web application: create some data, serve pages, receive data etc.

How would you lay out these events? In a threaded system the design is rather "simple". You dedicated threads to specific set of tasks and you go down the road of thread synchronization. While these task are at low on demand the threads sit idle and do nothing. When they are needed they run their code. While this road has issues it's well documented and kind of solved.

I find it hard to wrap my head around the node.js event way of doing things. I have 10 request coming in, but I haven't created any data so I can't serve anying, creating data is a long action and another 5 client wants to send data. What now?

UPDATE: While trying to wrap my head around this issue I've created the following code, it's untested yet so bare with me here. That code is basically a pile of callbacks which get registered and should be executed. There will be some kind of a pile manager that will run and decide which code does it want to execute now. All the callback created by that callback can be added "naturally" to the even loop. It should also register it's self so the event loop could give the control back to it. Other things like static content and what ever can be binded differently.

  1. How can I register a call back to be the last call in the current event loop state?
  2. Is this a good way to solve this issue?

Correct Answer:

The answer chosen is the most correct, there is but one minor code change and that is:

Change this function from this:

getDetail : function (id, callback) {
    var data = makeMyData(id);
    callback(data)
}

To that:

getDetail : function (id, callback) {
    var data = makeMyData(id);
    setTimeout(callback,0,data);
}
Problem courtesy of: qballer

Solution

The most important thing to remember when coming from a threaded environment is that in node you don't wait for an action to finish happening, instead you tell it what to do when it is done. To do this you use a callback, this is a variable which contains a function to execute, or a pointer to a function if you like.

For example:

 app.get('/details/:id?', function (req, res) {
    var id = req.params.ucid,
        publish = function (data) {
            res.send(data);
        };
    service.getDetails(id, publish);
});

You can then invoke the publish method from within your get details method once you have created the required data.

getDetail : function (id, callback) {
    var data = makeMyData(id);
    callback(data)
}

Which will then publish your data back to the response object. Because of the event loop node will continue to serve requests to this url without interrupting the data generation from the first request

Solution courtesy of: Mark Taylor

Discussion

There is currently no discussion for this recipe.

This recipe can be found in it's original form on Stack Over Flow.