Summary

Every response received on a long-lived exhaust command session (oplog fetcher getMore, awaitable hello, change-stream getMore) constructs a new child CancellationSource from the reader's long-lived parent token. The child's onCancel() continuation is registered into the parent token's SharedPromise children list and strongly holds the child CancellationState. When the child source is destroyed after the response completes, only its own promise is dismissed; the continuation entry in the parent's children list is never removed. Consequently each response permanently leaks one SharedStateImpl  CancellationState ( continuation) for the lifetime of the parent token (which equals the lifetime of the exhaust session — days to months). Growth is linear in the number of exhaust responses: measured 40–93 MB/day per node depending on response cadence.

Observed behavior

• `tcmalloc.generic.current_allocated_bytes` minus WiredTiger cache grows strictly linearly: secondary/hidden 40–93 MB/day (idle oplog, fetcher awaitTime ≈1 s ⇒ ≈86k responses/day); primary without change streams only single-digit MB/day (low-cadence awaitable-hello sessions); primary serving change streams matches secondary rates.
• Runtime heap profiler (`setParameter heapProfilingSampleIntervalBytes=65536`) shows monotonically ratcheting activeBytes (no pull-back) on exactly this family of stacks (≈82% of sampled growth on a hidden node):

• tcmalloc::tcmalloc_internal::SampleifyAllocation()
   mongo::future_details::FutureImpl<>::makeContinuation<>          (or SharedStateImpl::addChild)
   mongo::CancellationSource::CancellationSource()                 ← allocation site
   std::__shared_count<>::__shared_count<>()
   mongo::AsyncDBClient::_waitForResponse()
   mongo::AsyncDBClient::_continueReceiveExhaustResponse()
   mongo::AsyncDBClient::awaitExhaustCommand()
   mongo::executor::ExhaustResponseReaderTL::_read() / ::next()
   mongo::executor::ThreadPoolTaskExecutor::_continueExhaustCommand()
   ... NetworkInterfaceTL / asio reactor 

• • Closure check on a 75-day-uptime node: (39.7 + 5.5 [see Ticket 2]) MB/day × 75.4 d ≈ 3.4 GB ≈ measured non-WT heap 3.26 GB (<5% error).

    Root-cause analysis (source references per develop; identical in release trees and master)

• • src/mongo/executor/exhaust_response_reader_tl.cpp:67 — reader constructs _cancelSource from the request token (parent token lives as long as the reader).
  • src/mongo/executor/exhaust_response_reader_tl.cpp:119 — each response _read() reuses the same parent token.
  • src/mongo/db/client.h-side AsyncDBClient::_withCancellation(Message) (src/mongo/executor/async_client.h:164-167) — constructs a new child CancellationSource(parentToken) per response.
  • src/mongo/util/cancellation.h:243-255 — child ctor: token.onCancel().unsafeToInlineFuture().then([state]{ state->cancel(); }).getAsync(...). The then continuation is attached via SharedStateImpl::addChild (src/mongo/util/future_impl.h:538, children.emplace_front:573) and strongly holds the child CancellationState.
  • Child CancellationSource destruction only dismisses its own promise; no removal path exists for the parent-side child entry until the parent token is canceled/dismissed. The ctor comment claims "long-lived tokens can have many sub-sources for tasks which start and complete without worrying about too much memory build-up" — this claim does not hold for this path (measured).

    Carriers (any long-lived exhaust session)

• Oplog fetcher getMore (secondaries/hidden; dominant on idle replicas, ≈1 response/s).
• Awaitable hello streams (all nodes; low cadence floor).
• Change-stream getMore (primaries serving change streams; pins equal to open change streams).
• SD monitoring / any driver using exhaust commands. (Server-side awaitable-hello continuation loop, replication_info.cpp:632-660, is the same weak area; cf. official alert "Awaitable Hello Command in Exhaust Mode Unthrottled Response Loop", fixed 8.3.7.)

Impact

Unbounded, linear, restart-only-recoverable heap growth on any long-lived exhaust consumer: secondaries/hidden (fetcher) and primaries serving change streams. Production sharded clusters reach OOM/HA-switch risk after weeks of uptime (observed 84–87% memory on 6 GB nodes at 68 days). Upgrade within 8.x does not help (master affected).

Suggested fix (options)

• A (preferred, minimal): reuse a single per-reader child CancellationSource (_respCancelSource) created at reader construction; before each _read(), cancel+reconstruct (or add reset() clearing its onCancel registration chain) so continuations do not accumulate across responses; propagate cancel on reader destruction.
• B: make the parent children list removable: store the intrusive list hook in the continuation state; unlink (removeChild) when the child CancellationState's last reference dies; parent cancel skips unlinked entries.
• C: hold child state weakly in the parent's continuation; skip dead children on parent cancel. Acceptance: unit tests for cancel propagation semantics; resmoke exhaust/change-stream suites; 24 h replica+shard soak with heap profiler showing exhaust-family ratchet slope → 0 and node slope < 5 MB/day; feature-flag gated.