Node.js - Maximum call stack size exceeded

Problem

When I run my code, Node.js throw "RangeError: Maximum call stack size exceeded" exception caused by too much recursion calls. I tried to increase Node.js stack-size by sudo node --stack-size=16000 app, but Node.js crash without any error message. When I run this again without sudo, then Node.js print 'Segmentation fault: 11'. Is there a possibility to solve this without removing recursion call?

Thanks

Problem courtesy of: user1518183

Solution

You should wrap your recursive function call into a

  • setTimeout,
  • setImmediate or
  • process.nextTick

function to give node.js the chance to clear the stack. If you don't do that and there are many loops without any real async function call or if you do not wait for the callback, your RangeError: Maximum call stack size exceeded will be inevitable.

There are many articles concerning "Potential Async Loop". Here is one.

Now some more example code:

// ANTI-PATTERN
// THIS WILL CRASH

var condition = false, // potential means "maybe never"
    max = 1000000;

function potAsyncLoop( i, resume ) {
    if( i < max ) {
        if( condition ) { 
            someAsyncFunc( function( err, result ) { 
                potAsyncLoop( i+1, callback );
            });
        } else {
            // this will crash after some rounds with
            // "stack exceed", because control is never given back
            // to the browser 
            // -> no GC and browser "dead" ... "VERY BAD"
            potAsyncLoop( i+1, resume ); 
        }
    } else {
        resume();
    }
}
potAsyncLoop( 0, function() {
    // code after the loop
    ...
});

This is right:

var condition = false, // potential means "maybe never"
    max = 1000000;

function potAsyncLoop( i, resume ) {
    if( i < max ) {
        if( condition ) { 
            someAsyncFunc( function( err, result ) { 
                potAsyncLoop( i+1, callback );
            });
        } else {
            // Now the browser gets the chance to clear the stack
            // after every round by getting the control back.
            // Afterwards the loop continues
            setTimeout( function() {
                potAsyncLoop( i+1, resume ); 
            }, 0 );
        }
    } else {
        resume();
    }
}
potAsyncLoop( 0, function() {
    // code after the loop
    ...
});

Now your loop may become too slow, because we loose a little time (one browser roundtrip) per round. But you do not have to call setTimeout in every round. Normally it is o.k. to do it every 1000th time. But this may differ depending on your stack size:

var condition = false, // potential means "maybe never"
    max = 1000000;

function potAsyncLoop( i, resume ) {
    if( i < max ) {
        if( condition ) { 
            someAsyncFunc( function( err, result ) { 
                potAsyncLoop( i+1, callback );
            });
        } else {
            if( i % 1000 === 0 ) {
                setTimeout( function() {
                    potAsyncLoop( i+1, resume ); 
                }, 0 );
            } else {
                potAsyncLoop( i+1, resume ); 
            }
        }
    } else {
        resume();
    }
}
potAsyncLoop( 0, function() {
    // code after the loop
    ...
});
Solution courtesy of: heinob

Discussion

I found a dirty solution:

/bin/bash -c "ulimit -s 65500; exec /usr/local/bin/node --stack-size=65500 /path/to/app.js"

It just increase call stack limit. I think that this is not suitable for production code, but I needed it for script that run only once.

Discussion courtesy of: user1518183

In some languages this can be solved with tail call optimization, where the recursion call is transformed under the hood into a loop so no maximum stack size reached error exists.

But in javascript the current engines don't support this, it's foreseen for new version of the language Ecmascript 6.

Node.js has some flags to enable ES6 features but tail call is not yet available.

So you can refactor your code to implement a technique called trampolining, or refactor in order to transform recursion into a loop.

Discussion courtesy of: Angular University

If you don't want to implement your own wrapper, you can use a queue system, e.g. async.queue, queue.

Discussion courtesy of: weakish

Regarding increasing the max stack size, on 32 bit and 64 bit machines V8's memory allocation defaults are, respectively, 700 MB and 1400 MB. In newer versions of V8, memory limits on 64 bit systems are no longer set by V8, theoretically indicating no limit. However, the OS (Operating System) on which Node is running can always limit the amount of memory V8 can take, so the true limit of any given process cannot be generally stated.

Though V8 makes available the --max_old_space_size option, which allows control over the amount of memory available to a process, accepting a value in MB. Should you need to increase memory allocation, simply pass this option the desired value when spawning a Node process.

It is often an excellent strategy to reduce the available memory allocation for a given Node instance, especially when running many instances. As with stack limits, consider whether massive memory needs are better delegated to a dedicated storage layer, such as an in-memory database or similar.

Discussion courtesy of: serkandemirel0420

This recipe can be found in it's original form on Stack Over Flow.