The Thread Pool Nobody Reads About Until Their API Starts Queuing

You’ve read the event loop diagrams. You know Worker Threads exist for CPU-bound work, and Cluster exists to use more than one core. So why does a Node.js app with 2% CPU usage start queuing requests the moment fifty users hit an endpoint that reads a file and hashes a password?

Because none of that explains what’s actually happening. There’s a fourth thing under the hood, and it’s not the event loop, not Worker Threads, and not Cluster. It’s libuv’s thread pool, and it has a hard limit that most people never touch, because most people never learn it exists.

It’s not “async I/O” the way people assume

Node.js markets itself as non-blocking, and for network sockets that’s true at the OS level — TCP connections use epoll (Linux), kqueue (macOS), or IOCP (Windows), and the event loop handles thousands of them without spinning up a single extra thread. That part really is free concurrency.

But a handful of APIs can’t use that trick, because the underlying OS calls are blocking by nature. There’s no non-blocking version of “hash this password with scrypt” or “read this file from disk” at the syscall level on most platforms. So libuv fakes it: it hands the blocking work to a background thread, and tells your JavaScript callback about it once that thread is done. From your code’s perspective it looks async. Under the hood, a real OS thread just sat there blocking.

That background thread comes from a fixed-size pool. Not a growing pool, not one thread per request — a pool, and its default size is 4.

What actually goes through it

If you’ve never had a reason to look this up, the list is shorter than people expect:

Most of the fs module — readFile, writeFile, stat, readdir, and friends. dns.lookup() specifically (not dns.resolve() and its variants, which use c-ares and go through actual async DNS queries, not the thread pool). Password hashing functions in crypto — pbkdf2, scrypt, and randomBytes for larger byte counts. zlib compression and decompression.

Notice what’s missing: HTTP requests, database drivers using their own async protocol implementations, raw TCP sockets. Those go through the event loop’s native async I/O and don’t touch this pool at all. Which is exactly why so many people never hit the limit — plenty of Node backends are mostly HTTP handlers with a database call, and the database driver isn’t in that list above.

The apps that get bitten are the ones doing several of these at once under load: an upload endpoint writing files to disk, an auth endpoint hashing passwords with bcrypt (which is built on similar blocking primitives) or scrypt, a log ingestion service reading and parsing files. Four background threads, shared across every single one of those operations happening anywhere in your process.

Seeing it happen

This is easy to reproduce without guessing. Run four crypto.pbkdf2 calls, then run eight, and time both:

const crypto = require('crypto');

function hashOne(i) {
 return new Promise((resolve) => {
 const start = Date.now();
 crypto.pbkdf2('password', 'salt', 100000, 64, 'sha512', () => {
  resolve(Date.now() - start);
 });
 });
}

async function run(count) {
 const results = await Promise.all(
 Array.from({ length: count }, (_, i) => hashOne(i))
 );
 console.log(`count=${count}`, results);
}

run(4).then(() => run(8));

With the default pool size, the first four calls finish in roughly the same window — they each got their own thread. The next four in the batch of eight sit waiting for a thread to free up, so their individual completion times stretch out, even though total CPU usage on the machine barely moves. That “CPU is fine but things are slow” combination is the signature of this specific bottleneck, and it’s why so many people chase the wrong fix — they look at CPU graphs, see nothing alarming, and assume the problem is elsewhere.

The fix, and the gotcha that catches people

The pool size is controlled by an environment variable:

UV_THREADPOOL_SIZE=16 node server.js

The maximum used to be capped at 128; as of libuv 1.30.0 that ceiling was raised to 1024. But the number that matters in practice is usually much smaller — going from 4 to somewhere in the 8–16 range covers most real workloads, and pushing it far higher mostly adds thread management overhead without matching gains, since your CPU core count still limits how much of that work can actually run in parallel.

The gotcha: libuv allocates the pool lazily, the first time something actually needs it. If you try to set process.env.UV_THREADPOOL_SIZE inside your own JavaScript after some other module has already triggered a file read or a DNS lookup, it’s too late — the pool size for that process is already locked in. This is the actual reason “I set it in my code and it did nothing” is a common complaint. It needs to be set as an environment variable before the Node process starts, not from inside it.

Where this doesn’t help

If your service is mostly proxying HTTP requests and talking to Postgres or Redis through their own drivers, raising this number will do nothing measurable, because none of that traffic goes through the thread pool in the first place. This is worth checking before spending time tuning it — profile which of your operations actually fall into the fs / dns.lookup / crypto / zlib bucket before assuming this is your bottleneck.

It’s also not a substitute for Worker Threads if your actual problem is CPU-bound JavaScript computation blocking the event loop itself — that’s a different mechanism entirely, and no amount of thread pool tuning touches JavaScript execution on the main thread. This pool only helps with the specific handful of blocking syscalls listed above.