Allow your Node.js workers to die honourably with Seppuku
A Node.js project I worked on recently required the ability to gracefully restart a cluster’s workers under a specific set of circumstances. The documentation gives a few hints on how to implement this functionality, and one can easily find many solutions with a simple Google search.
But I had a few specific requirements in mind:
- Restart gracefully in the event of an exception.
- Restart gracefully after a certain number of requests to avoid memory creep.
- Restart on request using a variety of methods.
- Allow me to perform custom steps as part of the shutdown process.
- Provide for a period of time during which additional shutdown ops can be performed.
- Randomize the shutdown timing so that the server doesn’t get overwhelmed by multiple workers restarting at the same time.
- Allow me to avoid having to build this functionality into every project. Laziness dictates the creation of a module!
A little coat of straw
These requirements led me to write Seppuku, a library that is simple to use and can handle a variety of restart conditions.
Seppuku should work well in a either an Express or Restify project, though I mainly test with the latter. It’s set up as a common handler, which means that you can instantiate very easily:
var seppuku = require('seppuku'); var server = restify.createServer(); server.use(seppuku(server, options));
By default, Seppuku just keeps an eye on uncaught exceptions; if it detects one, it shuts down the server and (if applicable) notifies the parent process that the worker is no longer active.
The server is then made to emit a
seppuku event, after which Seppuku waits a randomized amount of time for the existing event queue to deplete; if the depletion doesn’t complete, it then forces the worker to die.
You can also have Seppuku restart workers automatically after they have processed a certain number of requests—that’s handy if you have a leak somewhere and don’t have time to attend to it right away. There are provisions in the module to allow you exclude certain requests from the count, or give each request a custom weight.
Finally, you can initiate a seppuku manually—for example, in response to an signal being received from the operating system. And, of course, you can also cancel a seppuku if you need to.
Obviously, one ideally never wants a process to just terminate, but, if there is no other choice, it’s good to know that a termination will most likely be handled in an orderly fashion. In my particular case, I needed to shut down and restart a number of client-side streaming operations with minimal disruption whenever a worker reset occurs, and Seppuku lets me handle that process very easily.