Your libuv Thread Pool Size Is Too Small


A micro-article about Node.js, libuv, how your application might be bottlenecked by libuv’s thread pool size, and how to fix it.

Potential Impact

For I/O-heavy applications that rely on libuv, the libuv thread pool size may become a critical bottleneck and one of the most impactful factors to increase total throughput.

“What’s a libuv?”

libuv
libuv logo

libuv is a multi-platform library for asynchronous I/O.

“Must Knows” for libuv?

  • Some of Node.js’ I/O methods rely on libuv.
    • Where possible, Node.js uses APIs that are already asynchronous/non-blocking.
    • In the other cases, libuv uses a thread pool to turn sync/blocking I/O into async/non-blocking.
  • libuv’s thread pool size is only 4 by default.

What’s the Problem?

In as few words as possible: libuv’s default thread pool size is only 4.

Do anything more than 4 concurrent I/O operations that rely on libuv, and libuv’s small thread pool size becomes a bottleneck, as each extra operation has to wait in queue.

Some I/O operations that use libuv’s thread pool:

  • All file system operations
  • DNS resolution
  • Lib & user code that calls uv_queue_work

Solving It

To solve it, increase the environment variable UV_THREADPOOL_SIZE before you do any I/O calls. Ideally, change it before starting Node.js.

libuv instantiates the thread pool when you first call an I/O method that relies on the thread pool. Once instantiated, changes to the thread pool size (UV_THREADPOOL_SIZE) will have no effect.

Examples

It’s recommended to change UV_THREADPOOL_SIZE before starting Node.js.

Your intentions are clearer if you set environment variables before launching Node.js, and it’s less likely that something will go wrong (e.g. an included library calls I/O as a side effect).

Via Bash:

UV_THREADPOOL_SIZE=64 node index.js

Via Windows CLI:

SET UV_THREADPOOL_SIZE=64 && node index.js

As first instruction in your Node.js app: (not recommended)

'use strict'
process.env.UV_THREADPOOL_SIZE=64;

How High Should I Set It?

Depends on a case-by-case basis.

  • The thread pool size is the exact amount of maximum concurrent I/O tasks.
  • Setting it a little too high has less impact on your throughput than setting it too low.
  • Setting it extremely high is unnecessary.
  • Memory overhead is ~1MB for 128 threads.
  • The maximum (since 1.30.0) is 1024.
  • Setting it to the number of CPU cores is not a good standard. These are I/O, not CPU-intensive, operations.

You can count threads in use by node on Linux with:

ps -Lef | grep  "\<node\>" | wc -l

Reading Material