Why Your Node.js Stream Is Quietly Eating RAM and You Don’t See It

Most Node.js stream bugs don’t crash your app. They just slowly inflate RSS until someone notices the memory graph looks like a ski slope. The core of a Node.js streams memory leak is almost always the same story: a fast producer, a slow consumer, and a buffer that nobody is watching. The worst part is that your app keeps working — requests go through, data gets written — right up until it doesn’t.

TL;DR: Quick Takeaways

• .pipe() does not destroy streams on error — leaked file descriptors and growing buffers are the default behavior, not an edge case.
• highWaterMark is a backpressure threshold, not a memory cap — ignoring write()‘s return value overrides it completely.
• The 'data' event puts a readable stream into flowing mode and bypasses backpressure entirely — a very common cause of unbounded memory growth.
• Without respecting backpressure, RSS can grow to 17× what pipeline() uses for the same workload — documented in Node.js core benchmarks.

How Node.js Stream Buffering Actually Works

Every stream in Node.js has an internal buffer. For binary streams the default highWaterMark is 16KB. For object mode streams it’s 16 objects — and this is where people get burned, because 16 objects sounds small until each object is a fat JSON payload with nested arrays. The buffer is not a hard wall. When a writable stream’s buffer exceeds highWaterMark, write() returns false as a signal that the producer should back off. That’s the entire backpressure mechanism in one sentence. If you ignore that false, Node.js doesn’t throw. It just keeps buffering. Forever.

The Drain Event Nobody Listens For

The contract is straightforward: write() returns false → you stop writing → you wait for 'drain' → you resume. Most tutorial code skips the middle two steps and goes straight from “here’s how to create a WriteStream” to “ship it.” In production, under load, when your writable is doing anything remotely slow — a DB insert, a network call, even gzip compression — the write queue grows chunk by chunk and takes your heap with it.

// The pattern that causes the node.js streams memory leak
readable.on('data', (chunk) => {
 writable.write(chunk); // return value ignored — backpressure dead
});

// What you actually need
readable.on('data', (chunk) => {
 const ok = writable.write(chunk);
 if (!ok) {
 readable.pause();
 }
});

writable.on('drain', () => {
 readable.resume();
});

The first pattern is everywhere in blog posts and StackOverflow answers. It works fine with small files and fast consumers. Throw a 2GB CSV at it with a slow downstream and you’ll watch RSS climb until the OOM killer shows up. The drain pattern is the manual implementation of what pipe() does internally — which is why pipe() handles backpressure correctly for the happy path, but falls apart the moment an error enters the picture.

Object Mode and the highWaterMark Illusion

Object mode changes how highWaterMark is counted — from bytes to object count. A limit of 16 objects sounds reasonable until you stream database rows with 50 fields each. The buffer won’t tell you it’s holding 800KB of JSON; it’ll say “14 objects, no problem.” And unlike binary mode where you can sanity-check against expected file size, object mode gives you no intuitive way to estimate memory pressure without instrumenting writable.writableLength explicitly.

// Object mode: highWaterMark counts objects, not bytes
const { Transform } = require('node:stream');

const transform = new Transform({
 objectMode: true,
 highWaterMark: 16, // 16 objects — could be 16MB if objects are fat
 transform(chunk, encoding, callback) {
 // Slow async operation — DB write, HTTP call, etc.
 processSlowly(chunk).then(() => callback());
 }
});

// Monitor actual buffer pressure
setInterval(() => {
 console.log('Buffer depth:', transform.writableLength); // objects in queue
}, 1000);

Watching writableLength in production is the fastest way to spot a backpressure problem before it becomes a memory incident. If that number keeps climbing and never comes back down, the consumer is losing the race and your buffer is acting as a shock absorber for the entire speed delta.

Why .pipe() Silently Leaks — The Error Path Nobody Tested

Here’s the thing about .pipe() that trips up almost every developer at some point: it handles backpressure correctly, but it does not handle errors correctly. When an error fires on the source stream, pipe() unregisters the pipe but does not destroy the destination stream. When an error fires on the destination, same story — the source keeps running, the destination stays open, and both streams are now orphaned in memory. This is the canonical node.js pipe memory leak, and it’s been a known issue since at least Node.js v0.10. It didn’t get a proper fix — it got a new API instead: stream.pipeline().

The Resource Leak Pattern

Real production code usually looks something like this: read a large file, compress it, write to S3 or disk. Three streams chained. If the write fails halfway through, pipe() will disconnect the pipe, but the read stream keeps reading, the gzip stream keeps compressing, and the file descriptor stays open. You now have a zombie pipeline consuming CPU and holding an fd. If your server handles concurrent uploads, this multiplies fast.

// What most code looks like — leaks on error
const src = fs.createReadStream('./big-data.csv');
const gzip = zlib.createGzip();
const dest = fs.createWriteStream('./big-data.csv.gz');

src.pipe(gzip).pipe(dest);
// Error in dest? src and gzip stay alive.
// Error in src? dest stays open.
// No cleanup. No fd release. Silent.

// What it should look like
const { pipeline } = require('node:stream/promises');

await pipeline(
 fs.createReadStream('./big-data.csv'),
 zlib.createGzip(),
 fs.createWriteStream('./big-data.csv.gz')
);
// Error anywhere → all streams destroyed, fds closed, memory freed

stream.pipeline() — available since Node.js v10, promise-based via stream/promises since v15 — calls .destroy() on every stream in the chain when any one of them errors. That’s the behavior you actually want. The fact that pipe() shipped without this and became the default teaching example for a decade is one of those Node.js legacy decisions that keeps causing incidents.

Multiple Pipes to One Readable: The Speed-of-Fastest Problem

There’s a less obvious variant: piping one readable to two or more writables. Node.js will push data at the rate of the fastest consumer, not the slowest. The fast consumer (say, a checksum calculation) drains chunks immediately, which signals the readable to keep flowing — while the slow consumer (say, a network upload) can’t keep up. The readable happily buffers everything for the slow consumer. A GitHub issue in the Node.js core repo documented this producing 5GB of in-memory buffers for a 5GB file copy. The fix is a Transform stream in the chain so backpressure from the slow consumer actually propagates.

The ‘data’ Event Antipattern: Flowing Mode Bypasses Everything

Attaching a 'data' event listener on a readable stream flips it into flowing mode. In flowing mode the stream pushes data as fast as it can generate it. There is no built-in pause. If the downstream can’t keep up, the chunks pile up wherever you’re writing them. This is not a bug — it’s documented behavior. But it’s the kind of thing that makes a slow consumer problem invisible: the readable stream reports healthy, readableFlowing is true, event loop isn’t blocked — and your writable is drowning.

Flowing Mode vs Paused Mode: What Actually Changes

The state machine behind readable streams has three values for readableFlowing: null (initial, no consumer), true (flowing, emitting data events), false (explicitly paused). The critical detail is that transitioning from true to false requires an explicit readable.pause() call or an unpipe(). And here’s the trap: once you’ve attached a 'data' listener and then removed it, the stream doesn’t automatically return to null — it stays at false. Adding a new 'data' listener won’t restart the flow. You have to call resume() manually. This has caused dozens of bug reports over the years from devs who swore their stream “just stopped emitting.”

// Flowing mode — backpressure blind
readable.on('data', (chunk) => {
 writable.write(chunk); // writable drowning, nobody cares
});

// Paused mode with async iterator — backpressure baked in
async function process(readable, writable) {
 for await (const chunk of readable) {
 const ok = writable.write(chunk);
 if (!ok) {
 await new Promise(resolve => writable.once('drain', resolve));
 }
 }
 writable.end();
}

// Or just use pipeline — it handles all of this
await pipeline(readable, writable);

The async iterator approach is the cleanest manual solution. The for await...of loop naturally respects pauses because it awaits the next chunk — if the readable is paused, the loop waits. It also handles cleanup better than raw event listeners because a break or thrown error inside the loop calls readable.destroy() automatically in Node.js v14+.

Transform Streams: Two Buffers, One Headache

A Transform stream is both readable and writable, which means it has two internal buffers: one on the writable side (incoming data) and one on the readable side (outgoing data). The highWaterMark option in older Node.js versions applied to both sides equally. Since v10 you can set readableHighWaterMark and writableHighWaterMark separately — but almost nobody does, which means the default 16KB applies to both, and mismatched upstream/downstream speeds create double buffering pressure.

The Paused-by-Default Trap

Transform streams start paused. They won’t process data until something consumes their readable side — either a pipe, a 'data' listener, or an async iterator. If you write to a Transform but never read from it, the writable buffer fills up, backpressure kicks in, and all upstream writes stall. Silently. This is a common gotcha when Transform streams get created and configured but the pipeline assembly is delayed by async initialization code. You write three chunks, then await your DB connection, and suddenly the writable has buffered everything and the upstream is paused for no apparent reason.

Detecting Stream Memory Pressure Before Production Notices

The standard approach — wait for alerts, look at Datadog — is almost always too late. By the time RSS is visibly climbing in your APM, you’ve already been leaking for minutes. The right approach is instrumenting the streams themselves. writable.writableLength tells you how many bytes (or objects) are queued in the write buffer right now. If this value consistently exceeds highWaterMark, you’re not respecting backpressure somewhere. A periodic log of this metric in any high-throughput stream pipeline will surface problems in staging before they become incidents.

What process.memoryUsage() Actually Tells You

process.memoryUsage() returns rss, heapTotal, heapUsed, and external. Stream buffers mostly live in external — they’re allocated outside the V8 heap via Buffer which uses native memory. This means a stream-induced memory leak can be completely invisible in heapUsed. If your heap looks flat but RSS is growing, look at external. A growing external value with flat heapUsed is a strong signal that buffer data is accumulating in stream internals rather than in JavaScript objects.

// Stream health check — add to any critical pipeline
function monitorStream(writable, label) {
 const interval = setInterval(() => {
 const mem = process.memoryUsage();
 console.log({
 label,
 writableLength: writable.writableLength,
 highWaterMark: writable.writableHighWaterMark,
 bufferRatio: (writable.writableLength / writable.writableHighWaterMark).toFixed(2),
 externalMB: (mem.external / 1024 / 1024).toFixed(1),
 rssMB: (mem.rss / 1024 / 1024).toFixed(1)
 });
 }, 5000);

 writable.on('finish', () => clearInterval(interval));
 writable.on('error', () => clearInterval(interval));
}

A bufferRatio consistently above 1.0 means you’re above the backpressure threshold — the stream is holding more than its designed capacity. A ratio that keeps climbing without coming back down means the consumer is permanently losing the race. This snippet costs almost nothing to run and has saved multiple production debugging sessions.

FAQ

Why does a Node.js streams memory leak happen even when I use .pipe()?

.pipe() correctly implements backpressure for the happy path — it pauses the readable when the writable signals pressure via write() returning false. The leak happens on the error path: when any stream in the chain errors, pipe() disconnects the pipe but does not destroy the other streams. File descriptors stay open, buffers aren’t released, and if your app is handling concurrent requests, these zombie streams accumulate. The fix is stream.pipeline() which calls .destroy() on all streams in the chain when any one of them fails.

What does a slow consumer actually do to memory in Node.js?

When the writable consumer can’t process data fast enough, the write queue grows. Each queued chunk sits in memory — specifically in Buffer objects allocated outside the V8 heap, which is why heap profilers often miss stream-induced leaks. In the Node.js core docs, a benchmark comparing proper backpressure vs. ignoring it showed peak RSS of 87MB vs 1.52GB for the same workload — roughly a 17× difference. The slow consumer problem compounds under concurrent load because multiple pipelines can be building up buffers simultaneously.

Is highWaterMark a hard memory limit for Node.js streams?

No, and this misconception causes real problems. highWaterMark is a threshold that triggers the backpressure signal — it tells write() to return false. But Node.js doesn’t enforce it as a hard cap. If you keep calling write() after it returns false, Node will keep buffering. The data goes into the internal write queue without any error or exception. Raising highWaterMark just delays the backpressure signal — it doesn’t fix a fundamental producer/consumer speed mismatch, it just gives the problem more runway before it becomes visible.

Why does node.js pipe memory leak specifically happen with error events?

The design of pipe() separates data flow from error handling. An error event on the source causes unpipe() — the connection is severed, but both streams remain alive. An error on the destination also causes unpipe(), and again, both streams stay open. The source keeps reading (and buffering internally), the destination stays open consuming a file descriptor, and neither cleans up. This isn’t a bug that got patched — it’s intentional behavior from Node.js v0.x that was never changed for backwards compatibility. The explicit fix is attaching .on('error') to every stream in a pipe() chain, or switching to pipeline() which handles cleanup automatically.

When should I use async iterators vs pipeline for consuming streams?

Use pipeline() when you’re connecting discrete streams end-to-end and want automatic backpressure, error propagation, and cleanup with minimal code. Use async iterators with for await...of when you need to do something conditional with each chunk — inspect, filter, branch — that doesn’t fit cleanly into a Transform stream. Async iterators also integrate more naturally with async/await-heavy codebases and give you try/catch error handling without needing to attach listeners. Both approaches handle backpressure correctly; the 'data' event pattern handles neither backpressure nor errors correctly by default.

How do I detect a stream buffer accumulation problem before it causes an outage?

The key metric is writable.writableLength relative to writable.writableHighWaterMark. If the ratio consistently exceeds 1.0 and doesn’t recover, backpressure isn’t being respected somewhere in the chain. Second signal: watch process.memoryUsage().external rather than heapUsed — stream buffers are Buffer objects allocated in native memory, invisible to the V8 heap. A growing external value with a stable heap is almost always unmanaged stream buffer accumulation. Add these two metrics to any critical data pipeline and you’ll catch slow consumer problems in staging, not at 3am on call.