-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Unknown
-
None
-
Affects Version/s: None
-
Component/s: Backpressure, Core, Cursors
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description
Summary
After migrating from MongoDB Node.js Driver v3 to v7, a large CSV export began showing sustained memory growth while consuming a MongoDB cursor. Memory continues increasing with the number of processed documents until the container reaches its 2 GB memory limit and is restarted due to an OOM condition.
The issue is reproducible even when all CSV transformation, serialization, HTTP response handling, and Node.js stream adapters are removed.
The same behavior was reproduced with Driver v6, but it was not reproduced with Driver v5.x under the same application workload. With v5.x, memory stabilizes even when processing more than 47 million documents.
Application flow
The original application flow is:
{{MongoDB cursor
→ Transform
→ CSV stringify
→ HTTP response}}
Environment:
- Environment: CERT
- Container memory limit: 2 GB
- Application: Node.js
- MongoDB Node.js Driver affected versions: v6.x and v7.x
- Comparison version without sustained growth: v5.x
- Previous production version: v3.x
- Cursor is consumed sequentially
- Processed documents are not stored or accumulated by the application
Node.js: 24 LTS MongoDB Server: 8 mongodb: v7.x y v6.x mongodb v5: v5.9.2 Docker image: node:24-bookworm-slim
Actual behavior
Memory usage grows as documents are consumed from the cursor.
In the original export flow with Driver v7, the following memory behavior was observed:
| Processed documents | Pod memory |
|---|---|
| 0 | 430.6 MB |
| 100,000 | 754.4 MB |
| 300,000 | 1.1 GB |
| 500,000 | 1.2 GB |
| 700,000 | 1.5 GB |
| 800,000 | 1.5 GB |
The process eventually reaches the 2 GB container limit during larger exports, causing an OOM restart and producing incomplete or truncated export files.
Expected behavior
Previously consumed documents should become eligible for garbage collection when the application does not retain references to them.
Memory may increase initially because of connection pools, cursor batches, BSON deserialization, buffers, or V8 heap expansion, but it should eventually stabilize instead of continuing to grow proportionally with the number of consumed documents.
The cursor should be able to process datasets significantly larger than the available memory when documents are consumed sequentially and are not retained by the application.
Minimal reproduction
The behavior can be reproduced without CSV serialization, transformations, or HTTP output:
{{const cursor = collection.find(query); for await (const document of cursor) { // Consume each document without transforming it, // writing it to an output stream, or retaining a reference. }}}
{{}}
It was also reproduced using the stream API:
{{const { Writable } = require('node:stream'); const { pipeline } = require('node:stream/promises'); const objectSink = new Writable({ objectMode: true, write(document, encoding, callback)
{ // Discard the document without retaining it. callback(); }}); const inputStream = collection.find(query).stream(); await pipeline(inputStream, objectSink);}}
Isolation tests with Driver v7
Cursor consumed without transformation or output
The following components were removed:
- Application transformation logic
- Filtering and object flattening
- CSV serialization
- HTTP response writes
Only the cursor stream and a discard sink remained.
| Documents | Duration | RSS change | Heap change |
|---|---|---|---|
| 100,000 | 3.08 s | +336.7 MB | +293.5 MB |
| 150,000 | 4.44 s | +138.6 MB | +419.0 MB |
| 200,000 | 7.32 s | +471.8 MB | +523.1 MB |
The memory growth remained after removing the application transformation and serialization stages.
Minimal projection
The query was changed to retrieve only the username field.
| Documents | Duration | RSS change | Heap change | Final RSS |
|---|---|---|---|---|
| 200,000 | 6.99 s | +475.7 MB | +504.7 MB | 1.0 GB |
| 250,000 | 9.22 s | +588.9 MB | +418.9 MB | 1.1 GB |
| 300,000 | 11.97 s | +400.0 MB | +384.5 MB | 1.5 GB |
Reducing the document size did not eliminate the growth.
Manual APM segment removed
The manual recordAsyncSegment wrapper was removed and the cursor was consumed directly.
For 500,000 documents:
RSS: 284.2 MB → 1.1 GB Δ RSS: +814.2 MB heapUsed: 68.9 MB → 854.4 MB Δ heap: +785.4 MB
{{}}
This test did not disable all automatic APM instrumentation, but it excluded the application's manually created APM segment as the main cause.
for await...of instead of cursor.stream()
The cursor was returned directly and consumed using for await...of. No pipeline, Writable, CSV transformation, or HTTP output was used.
For 500,000 documents:
RSS: 282.1 MB → approximately 1.2 GB Δ RSS: approximately +925 MB heapUsed: 73.6 MB → approximately 1.0 GB Δ heap: approximately +951 MB
{{}}
The behavior therefore does not require cursor.stream(), pipeline(), or a Writable stream.
Version comparison
| Driver version | Test volume | Observed behavior |
|---|---|---|
| v7.x | 500,000 | RSS increased from 282.1 MB to approximately 1.2 GB |
| v6.x | Test run completed in 21.70 s | RSS increased from 1.0 GB to 1.8 GB; heap increased by 637.5 MB |
| v5.x | 2,000,000 | RSS increased by approximately 48.1 MB and stabilized near 595 MB |
| v5.x | 10,000,000 | Final RSS was 593.3 MB; RSS increased by approximately 32.9 MB from the test baseline |
| v5.x | 47,191,480 | Export completed with a final RSS of 632.7 MB and pod memory stabilized near 600 MiB |
With Driver v5.x, memory stabilized after the initial increase. Increasing the dataset from 2 million to more than 47 million documents did not produce the sustained growth observed with v6 and v7.
A production comparison using Driver v3.x also showed bounded memory growth for a 2-million-document export:
| Environment | Driver | Initial pod memory | Maximum pod memory | Increase |
|---|---|---|---|---|
| Production | v3.x | 121 MiB | 164 MiB | 43 MiB |
| CERT | v5.x | 522 MiB | 571 MiB | 49 MiB |
Absolute memory values differ between environments, but the increase during each export was similar and bounded.
Components excluded by the isolation tests
The sustained growth does not require:
- CSV serialization
- Application transformations
- HTTP response writes
- Full document projection
- The manual APM segment
- cursor.stream()
- pipeline()
- A Writable stream
- Application-level retention of processed documents
These results suggest that the behavior occurs within or below the cursor consumption path. However, the available evidence does not identify the exact internal component responsible for retaining or reserving the memory.
Impact
This issue prevents the application from safely upgrading to Driver v6 or v7 for high-volume cursor workloads.
The current operational workaround is to use Driver v5.x for the affected export process. This is not a desirable long-term solution because it prevents adoption of a current driver version.
Acceptance criteria
- The issue can be reproduced with an automated test that consumes a large cursor sequentially without retaining references to previously processed documents.
- The cause of the sustained heap or RSS growth in the affected cursor-consumption path is identified and documented.
- After the fix, memory reaches a bounded plateau instead of growing approximately with the total number of documents consumed.
- A cursor can process at least 2 million documents in a container limited to 2 GB without an OOM condition when the application does not retain the documents.
- Processing additional documents after the memory plateau does not cause continued monotonic growth attributable to already consumed cursor batches.
- The corrected behavior applies to both async iteration and the cursor stream API, or any API-specific limitation is documented.
- Existing cursor behavior, batching, backpressure, error handling, and cursor cleanup continue to work without regressions.
- A regression test covering large sequential cursor consumption is added to the MongoDB Node.js Driver test suite.
- The fix and affected versions are documented in the release notes or changelog.