-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Fully Compatible
-
QE 2026-11-23
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Summary
Add two router-side server parameters that bound on-demand lookback ("backfill") for endOfTransaction change events, per the design doc "Emit End-of-Transaction Events with Shard Targeting Support" (sections "Maximum lookback distance" and "Rebuild cap"). Both are settable globally and overridable per change stream via query settings.
Parameters
| Server parameter | Query settings wire name | Bounds |
|---|---|---|
| internalChangeStreamEndOfTransactionMaxLookbackSeconds | changeStreamEndOfTransactionMaxLookbackSeconds | Length of a single lookback: the oplog timestamp distance (T0 - F) in seconds between the resume point T0 and the lookback floor F (commitFloor). |
| internalChangeStreamEndOfTransactionMaxRebuilds | changeStreamEndOfTransactionMaxRebuilds | Total number of kickback-driven rebuilds over a stream's lifetime. |
Define both in src/mongo/db/query/query_execution_knobs.idl as query-settings-settable, following
the pattern of existing query knobs. Default values and validation ranges are not specified in the
design (the rebuild cap default is "settled during implementation"); pick them here, with a positive
lower bound, and document the choice. Both values must be representable in query settings.
Behavior
Max lookback distance
- When the synthesis stage raises ChangeStreamEndOfTransactionBackfillRequired with floor F for a
stream whose resume point is T0, the execution layer checks T0 - F against the configured value.
If it exceeds the value, do not rebuild. - required streams: fail fast with a descriptive, non-resumable fatal error naming the lookback
distance and the configured bound. The error code is decided in this ticket (reuse
ChangeStreamHistoryLost or add a dedicated code). - whenAvailable streams: do not fail. The transactions that needed the refused lookback are
abandoned and handled by the aggregator's eviction disposition (omitted, logged, counted). - The stage records a refused floor and does not re-raise the kickback for any floor >= a previously
refused floor, so a refused lookback is not retried on every pass. - Increment metrics.changeStreams.endOfTransaction.lookbackDistanceExceeded.
Rebuild cap
- Each rebuild decrements the stream's remaining budget. When the cap is reached, the execution
layer stops catching the kickback and fails the stream with a descriptive fatal error. It must not
loop. - The cap is a backstop for the one-round-per-rebuild invariant and the bound on total rebuilds. It
does not bound an individual lookback (that is the max lookback distance). - Increment metrics.changeStreams.endOfTransaction.rebuilds on each rebuild. Reaching the cap should
also be recorded by a metric.