Overview

Measure the disagg flow-control standby lag as the age of the oldest oplog entry the standby has not applied: primaryTip.wallTime - wallTime(first entry after the standby's reported watermark), and 0 when the watermark is at or past the primary tip. This is the standby-leg counterpart of how the materialization leg already measures lag (SERVER-132979: lastQueued.wallTime - oldestUnmaterialized.wallTime, fed by PageLogProvider::_wallTimeQueue).

Background

SERVER-133624 (PR 63825) stops post-idle write gaps from reading as standby lag with a primary-side caught-up floor in DisaggFlowControlShim::_computeLags: _standbyCaughtUpTip / _standbyCaughtUpAt record the primary tip and refresh time from the last refresh at which the standby watermark was at or past the tip, and lag is measured from max(standbyLastApplied.wallTime, _standbyCaughtUpAt). The floor is a lower bound on the wall time of the oldest unapplied entry, so it leaves two cases that an exact measurement covers:

1. Residual of up to one poll interval. The floor only knows the pending entry was written after the last caught-up refresh, so a post-idle reading can overstate the true age by up to flowControlPollIntervalMs (default 1s, against the 1.2s DSC engage threshold).
2. A pause that starts while the standby is still behind. The floor is set only at a refresh that observes the standby at the primary tip. If the standby finishes the pre-pause entries only after writes resume, no refresh sees it caught up, and the pause is still added to the reading on top of real lag. This is relevant to the throttle-manufactured gap loop in HELP-99306 if refreshes do not land at caught-up moments.

Proposed approach

• OplogProvider::makeLogEntry already records (lsn, oplogWallTime) for every oplog entry in _wallTimeQueue, but trims it in markSent (and _committedWallTimeQueue in markCommitted), so nothing is retained until the standby applies an entry. Add a standby-retained queue, trimmed as the standby's reported watermark advances. The LSN is the oplog RecordId (the entry Timestamp), so it compares directly with the standby's reported OpTime.
• Expose "wall time of the oldest entry after timestamp T" to the shim along the same path as getLastSentOplogWallTime() (LogProvider, LogServerManager, ReplicationCoordinatorDisaggregatedStorage, plus mocks and test fixtures).
• In DisaggFlowControlShim::_computeLags, use the exact age when available and fall back to the SERVER-133624 caught-up floor (then the raw formula) when the queue cannot answer.

Constraints to handle

• Memory grows with entries times lag (about 4MB at 30k entries/s and 8s of lag) and is unbounded while the standby is unreachable: cap the queue and clear it while the standby-lag signal is ignored (SERVER-130171), falling back as above.
• Watermark regression (a restarted standby replaying its checkpoint), provider stop/start and the first-sight re-read rule in makeLogEntry, and step-up/step-down resets.
• Write-path cost: one more mutex-guarded push per oplog entry on the provider send thread; needs write-heavy sys-perf validation, not only the flow-control tasks.

Suggested first step

Before implementing, check primary FTDC from a throttled run like HELP-99306 for whether flow-control refreshes observe the standby at the primary tip between gaps. If they do, the SERVER-133624 floor already covers that loop and this ticket can be closed.

Acceptance criteria

• Unit tests for the retained queue (retention, trim on watermark advance, cap, clear while ignored, restart and re-read) and for the shim's exact path and fallback.
• jstests/disagg_storage/flow_control_post_idle_burst.js and the existing flow-control jstests still pass.
• sys-perf neutral on disagg-m8g-perf-11-node.arm.aws write-heavy tasks.

Estimated size

Roughly 150-250 production lines across about 12-15 files (mostly pali/oplog_provider and the provider/manager interfaces), plus 200-350 test lines. Builds on SERVER-133624 rather than replacing it; the caught-up floor stays as the fallback.