Update network interface mock test fixture to prevent missed wakeup

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Replication
    • Fully Compatible
    • ALL
    • Repl 2026-07-20, Repl 2026-08-03
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      BF-44190: _runReadyNetworkOperations_inlock drops canceled alarms after executor yield

      The race

      TransactionCoordinator completion follows this chain after the last commit response is delivered:

      1. deleteCoordinatorDoc() completes → outer .getAsync() lambda fires on the executor thread
      2. Lambda calls _scheduler->shutdown() → iterates pending alarms and calls .cancel() on each
      3. cancel() inserts the alarm ID into NetworkInterfaceMock::_canceledAlarms
      4. _canceledAlarms must be drained by the network thread: each entry calls alarm.cancel(), which resolves deadlineFuture, which unblocks _done(), which sets _completionPromise

      The problem is in _runReadyNetworkOperations_inlock ([network_interface_mock.cpp:765–775|vscode-webview://0a4n1vfqoj7somk3nd5dip8dksek0l7uaah3evkkq8spmbsgsi6m/src/mongo/executor/network_interface_mock.cpp]):

      {{Deliver response → set kExecutorThread → yield to executor → RETURN
      ↑ executor populates _canceledAlarms here
      but nobody drains them}}
      After yielding to the executor (line 769–774), the function returned unconditionally. The executor ran step 2–3 above during that yield, but _canceledAlarms was never checked again. Result: deadlineFuture never resolved, _completionPromise was never set, and anything awaiting onCompletion() hung forever.

      Log evidence (from CI failures, e.g. 537bc1963e9d27bced15afbea7447660.txt):

      {{"Deleted coordinator doc" ← log ID 22474, step 1 completed
      ← log ID 22447 "Two-phase commit completed" ABSENT}}
      The coordinator doc was deleted (chain reached the lambda) but _done() never fired.

      This is a race because in production the executor runs on a real thread — if it finishes populating _canceledAlarms before the test pumps again, the second pump drains them fine. Under resource pressure or timing variance, the pump runs first (finding _canceledAlarms empty), then the executor populates it, and nobody pumps again.

      The 10-day quiet period (July 3–13) correlated with the revert of SERVER-128711, which had widened the race window by adding extra work in the completion chain.

            Assignee:
            Indy Prentice
            Reporter:
            Indy Prentice
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: