How to manage multiple threads in Node JS

In this post, I’m going to show you how to potentially triple the performance of your application. Node by managing multiple threads. This is an essential tutorial in which the techniques and examples shown will give you everything you need to set up efficient flow control.

Child Processes, Clustering and Worker Threads

For a long time, Node has had the ability to be multithreaded using child processes, clustering, or the more recently preferred module method called Worker Threads…

Child processes were the original means of creating multiple threads for your application and have been available since version 0.10. This was achieved by creating a node process for each additional thread that you wanted to create.

Clustering, which has been stable since around version 4, allows us to simplify the creation and management of child processes. It works great in combination with PM2…

Now, before we move on to multithreading our application, there are a few points that need to be fully understood:

1. Multithreading already exists for I / O tasks

There is a layer in Node that is already multithreaded and this is the thread pool libuv… I / O tasks such as file and folder management, TCP / UDP transactions, compression and encryption are passed to libuv, and if they are not asynchronous in nature, they are processed in the libuv thread pool.

2. Child Processes / Worker Threads only work for synchronous JavaScript logic.

Implementing multithreading using child processes or Worker Threads will only be effective for synchronous JavaScript code that performs time-consuming operations such as loops, computations, etc. If you try to offload I / O tasks to Worker Threads as an example, you will see no performance improvement.

3. It is easy to create one thread. It is difficult to dynamically manage multiple threads

It is easy enough to create one additional thread in your application, as there are tons of tutorials on how to do it. However, creating threads equivalent to the number of logical cores of your or virtual machine and managing the distribution of work between these threads is more difficult, developing such logic is quite laborious.

It’s good that we live in an open source world and the brilliant contributions of the Node. This means that there is already a module that will give us the full ability to dynamically create and manage threads depending on the availability of the CPU of our or virtual machine.

Worker Pool

The module we will be working with today is called Worker Pool… Created Jos de JongThe Worker Pool offers an easy way to create a pool of workers to dynamically offload computations, as well as to manage a pool of dedicated workers. Basically, it is a thread pool manager for Node JS that supports Worker Threads, child processes, and Web Workers for browser implementations.

To use the Worker Pool module in our application, you need to complete the following tasks:

• Install Worker Pool

First we need to install the Worker Pool module – npm install workerpool

• Initialize Worker Pool

Next, we will need to initialize the Worker Pool when our application starts.

• Create Middleware Layer

Then we will need to create a Middleware Layer between our complex JavaScript logic and the Worker Pool that will manage it.

• Update existing logic

Finally, we need to update our application to hand over time-consuming Worker Pool tasks as needed.

Managing multiple threads with Worker Pool

At this point, you have 2 options: Use your own NodeJS app (and install the modules workerpool and bcryptjs), or download source from GitHub for this tutorial and my video series about optimizing NodeJS performance…

If you choose the latter option, the files for this guide will be located in the folder 06-multithreading… Once downloaded, go to your project root folder and run npm install. Then enter the folder 06-multithreadingto continue working.

In folder worker-pool we have 2 files: one is the controller logic for the Worker Pool (controller.js). The other (file) contains functions that will be run by threads … aka the so-called middle layer that I talked about earlier (thread-functions.js).

worker-pool / controller.js

'use strict'

const WorkerPool = require('workerpool')
const Path = require('path')

let poolProxy = null

// FUNCTIONS
const init = async (options) => {
  const pool = WorkerPool.pool(Path.join(__dirname, './thread-functions.js'), options)
  poolProxy = await pool.proxy()
  console.log(`Worker Threads Enabled - Min Workers: ${pool.minWorkers} - Max Workers: ${pool.maxWorkers} - Worker Type: ${pool.workerType}`)
}

const get = () => {
  return poolProxy
}

// EXPORTS
exports.init = init
exports.get = get

In the controller.js file, we use the module workerpool… We also have 2 exported functions called init and get… Function init will be executed once while loading our application. It will instantiate the Worker Pool with the options we provide and a link to thread-functions.js… It also creates a proxy that will be kept in memory as long as our application is running. Function get just returns the proxy in memory.

worker-pool / thread-functions.js

'use strict'

const WorkerPool = require('workerpool')
const Utilities = require('../2-utilities')

// MIDDLEWARE FUNCTIONS
const bcryptHash = (password) => {
  return Utilities.bcryptHash(password)
}

// CREATE WORKERS
WorkerPool.worker({
  bcryptHash
})

In file thread-functions.js Let’s create worker functions that will be managed by the Worker Pool. In our example, we will apply BcryptJS for hashing passwords. This usually takes about 10 milliseconds, depending on the speed of the machine being used, and is a good solution for tedious tasks. Inside the file utilities.js there is a function and logic that hashes the password. All we do in thread functions is to do this bcryptHash through the function workerpool… This way we keep the code centralized and avoid duplication or confusion about where certain operations exist.

2-utilities.js

'use strict'

const BCrypt = require('bcryptjs')

const bcryptHash = async (password) => {
  return await BCrypt.hash(password, 8)
}

exports.bcryptHash = bcryptHash

.env

NODE_ENV="production"
PORT=6000
WORKER_POOL_ENABLED="1"

File .env contains the port number and sets the variable NODE_ENV to “production”. Here we indicate whether we want to enable or disable the Worker Pool by setting WORKER_POOL_ENABLED to “1” or “0”.

1-app.js

'use strict'

require('dotenv').config()

const Express = require('express')
const App = Express()
const HTTP = require('http')
const Utilities = require('./2-utilities')
const WorkerCon = require('./worker-pool/controller')

// Router Setup
App.get('/bcrypt', async (req, res) => {
  const password = 'This is a long password'
  let result = null
  let workerPool = null

  if (process.env.WORKER_POOL_ENABLED === '1') {
    workerPool = WorkerCon.get()
    result = await workerPool.bcryptHash(password)
  } else {
    result = await Utilities.bcryptHash(password)
  }

  res.send(result)
})

// Server Setup
const port = process.env.PORT
const server = HTTP.createServer(App)

;(async () => {
  // Init Worker Pool
  if (process.env.WORKER_POOL_ENABLED === '1') {
    const options = { minWorkers: 'max' }
    await WorkerCon.init(options)
  }

  // Start Server
  server.listen(port, () => {
    console.log('NodeJS Performance Optimizations listening on: ', port)
  })
})()

Finally, the file 1-app.js contains the code that will be executed when our application starts. First, we initialize the variables in the file .env… Then let’s set up the server Express and create a route called /bcrypt… When starting this route, check if the Worker Pool is enabled. If yes, then we get the Worker Pool proxy control and execute the function bcryptHashwhich we declared in the file thread-functions.js… She, in turn, will perform the function bcryptHash in Utilities and will return the result to us. If the Worker Pool is disabled, then just execute the function bcryptHash directly to Utilities…

At the end of our file 1-app.js there is a function that calls itself. This is to support async / await, which we use when interacting with the Worker Pool. Next, we initialize the Worker Pool, if enabled. The only configuration that needs to be overridden is the installation minWorkers to “max”. This ensures that the Worker Pool spawns as many threads as there are logical cores on our machine, excluding 1 logical core, which is used for the main thread. In my case, I have 6 physical cores with hyper-threading, which means I have 12 logical cores at my disposal. Therefore, when the value minWorkers equal to “max”, the Worker Pool will create and manage 11 threads. Finally, the last piece of code is to start our server and listen on port 6000.

Worker Pool Testing

Testing the Worker Pool is very simple: launch the application and execute the request while it is running get on the http://localhost:6000/bcrypt… If you have a load testing toolkit such as AutoCannon, you can see the difference in performance by enabling / disabling Worker Pool. AutoCannon is very easy to use.

Conclusion

I hope this tutorial has given you an introduction to managing multiple threads in your Node. The attached video at the beginning of this article clearly demonstrates the process of testing a Node.

Translation prepared as part of the course “Node.js Developer”… If you are interested in learning more about the course, register at Open Day…