Getting Started With Nodejs Cluster

If you are already using NodeJS for a while, you might already know that NodeJS is, by default, made up of single-threaded architecture. That means all your application logic will be run as a single thread for any amount of traffic requests. So, whenever it’s doing one operation, it won’t be doing anything else at that moment, even if there is a new request at the same time, which will have to wait. It may sound terrifying, but with its event-driven architecture, it’s doing the job quite well as soon as one golden rule is followed, which is ‘each operation in NodeJS should be done using minimum CPU time usage’. All CPU-heavy operations should be done somewhere else.

Well, that sounds fair. As soon as we know and follow this accordingly, our app should be fine doing its work in single-threaded mode. However, still, if we get very high traffic, where several requests per second need to be handled as fast as possible, single-threaded architecture won’t do much good at that point, restricting you from scaling your application. So, what do we do?

NodeJS Cluster To The rescue:

Well, there is no direct facility to create multiple threads in a single NodeJS application process. Still, there is a facility to create multiple processes to bind under the same server port and work in the same way independently. Nodejs ‘cluster’ module facilitates this in a very efficient manner. We can implement multi-process architecture with it easily.

Creating a simple worker process:

The cluster module follows the master-slave/parent-child paradigm. So, the first process that will be run is the master process, and all the other processes that it creates are workers. Let’s see a small code snippet:

var http = require("http");

var cluster    = require('cluster');
if (cluster.isMaster) {
    var worker = cluster.fork();
}
else {
    http.createServer(function(req, res){
        res.end("Hello World");
    }).listen(1337);
}
Code language: JavaScript (javascript)

As you can see here, we are checking if it’s the initial/master process by using ‘cluster.isMaster’ check. If so, we are forking a new process under it, which will in turn fall under ‘else’ block and create the child process, which does the actual work of server port binding. However, you may notice that the above code is going to create only two processes: one child and one master, where the master is doing nothing else other than creating the child.

Creating multiple child processes controlled by the master:

Let’s now move ahead a little to create more than one process and make sure all of them get terminated if the master is terminated.

var http = require("http");

var cluster    = require('cluster');
if (cluster.isMaster) {
    var workers = [];
    for(var i=0; i<5; i++) {
      workers.push(cluster.fork());
    }
    process.on('SIGINT', function(){
        console.log("exiting "+process.pid);
        for(var i=0; i<5; i++) {
           console.log("destroying "+worker.process.pid);
           workers[i].destroy();
        }
    });
}
else {
    console.log("Child process "+process.pid+" being created and listening to port 1337");
    http.createServer(function(req, res){
        res.end("Hello World");
    }).listen(1337);
}
Code language: JavaScript (javascript)

As you can see here, we are creating five child processes now, and all of them are listening to the same 1337 port. Besides, here our master process is also doing some significant tasks of making sure that all workers are terminated properly when the master process has to be terminated so that there are no orphan child processes going to be around in the system.

Now all of these processes will share the same server port and listen together, though the only process will be served with the request. If at some point 4 of them are already busy doing something, it still has room for getting another request by the fifth one.

Some more failure recovery tuning:

Let’s do some more awesome things. Let’s assume, for some reason, our NodeJS app is buggy and can crash anytime. Normally, you will consider something like supervisord to monitor such process so that they respawn/restarted again automatically on crash events to make sure your application is always alive. However, we can do that internally, taken care of by the master process. A small example to give you an idea of how to do this is here:

var child = cluster.fork();
cluster.on('exit', function (worker, code, signal) {
        //logging details about what happend
        child = cluster.fork();;
});
Code language: JavaScript (javascript)

Here, as soon as an exit event is observed, the master process creates another child process so that the number of available processes is always the same.

Making Your Life easier:

If you are wondering, what if you could just tell very easily how many child processes to create and where to bind, and all of this stuff happens automatically? Yes, it’s quite possible, leaving you doing all this stuff by yourself every time you create a NodeJS application; you can consider using the clustered-node library that I created a few days ago, which is both simple-to-use and eases migrating your existing NodeJS app to multi-process mode.

Things To Remember:

• Give Master Process Least Responsibility: It’s always recommended that you shouldn’t give the master process much responsibility as more responsibility means more risk to the erroneous process and more chance to crash the master itself, leaving all children processes as orphans.
• Don’t Just create many workers: You should consider an original number of CPUs/cores while creating child processes. You shouldn’t just create several child processes on a single CPU core, as they will share the same computing power by context switching, which might make performance worse. The general rule of thumb is the number of child processes=number of CPU cores available.

Final Words:

Let me know if something isn’t clear in this tutorial via comments. I hope this will help you to some extent to start with NodeJS cluster. Happy coding 🙂