-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Replication
-
ALL
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Symptoms
BF-45295: disagg_concurrency_simultaneous intermittently fails on enterprise-amazon-linux2023-arm64-disagg-no-feature-flags-extra-system-deps. The primary mongod exits abruptly with code 14 while the FSM batch is running, and the test then hangs for 600s in Finding primary before failing.
The mongod log shows the primary liveness watchdog firing:
- First interval: log 12984004, outstanding local writes uncommitted for a full interval.
- Second interval: log 12984000, followed by _unconditionalStepDown(kNoForwardProgress) and quickExit.
At the time of the step down, the written optime is well ahead while read/sent/acked and oplog visibility are stuck (e.g. written at :2175, everything else at :181 on the 2026-10-23 run), and WT logs show repeated 60s eviction assist waits that only start once the stall begins.
Root Cause
Under cache pressure, committing threads assist with WiredTiger eviction, but oplog visibility and lastApplied only advance after commit_transaction() returns.
With featureFlagPreparedTransactionsPreciseCheckpoints enabled, dirty content cannot be evicted until the stable timestamp moves, and stable is pinned by the committing transaction's durable timestamp, so the assist cannot make progress. WT-18210 bounds each wait at 60s (previously an unbounded
deadlock, SERVER-132391), but two back-to-back stalls exceed the watchdog's 120s window (primaryHealthIntervalMillis = 60000, two consecutive stalled intervals), so a primary that would have recovered is stepped down.
- related to
-
SERVER-138407 Re-exclude findAndModify_update_grow.js from disagg_concurrency_simultaneous until BF-45295 is fixed
-
- Closed
-