-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines - Transactions
-
ALL
-
v9.0
-
SE Transactions - 2026-09-25
-
200
-
5
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Summary
jstests/noPassthrough/txns_cache_errors/transaction_aborted_under_cache_pressure.js intermittently fails on mongodb-mongo-master at:
assert(numTemporarilyUnavailable > 0, "Expected TemporarilyUnavailable error but none occurred.");
The failure has been observed in BFG-3557872, BFG-3611646, and BFG-3638297, all on amazon-linux2023-arm64-static-compile. These failures were linked under BF-46354, but appear to be a separate, lower-rate master flake rather than the v9.0 backport-gap failure addressed by SERVER-132736.
Background
The test relies on a timing-sensitive race between the rollbackUnderCachePressure periodic thread and the drain loop, which calls abortTransaction_forTesting() for each session.
The background thread only runs while
WiredTigerCachePressureMonitor::isUnderCachePressure() is true. The thread-pressure portion of that condition is based on cumulative WT_STAT_CONN_APPLICATION_CACHE_TIME:
int64_t waitTimeThreshold =
gCachePressureEvictionStallThresholdProportion.load() * expectedThreadTime;
bool perThreadWaitTimeExceedsThreshold = cacheWaitTimeUsecs > waitTimeThreshold;
The test already lowers cachePressureEvictionStallThresholdProportion from the default 0.95 to 0.3, because the single-threaded workload otherwise rarely trips the condition. The test comments describe the remaining drain window as only a few hundred milliseconds.
Once the fill loop stops, no new writes generate cache-wait time. The drain loop therefore has only the residual pressure accumulated during the fill phase, leaving little margin for the pressure detector to fire before the sessions are drained.
Suspected interaction with WT-18210
WT-18210, “Bound the eviction assist at transaction resolution,” changed eviction-assist behavior on this path. When an insert fails under cache pressure and implicitly rolls back, the post-resolution assist that previously continued until the cache dropped below its trigger is now bounded by the operation timeout, with an additional stuck-cache escape.
This is the intended behavior for the problem addressed by WT-18210. However, it also means that application threads may accumulate less APPLICATION_CACHE_TIME per rollback. That further narrows the already-thin margin required for perThreadWaitTimeExceedsThreshold to become true during the drain loop.
This relationship is currently circumstantial and has not been confirmed by bisection.
Evidence
- All three failing master revisions—cb0fdd69, d87c40a1, and bae1b9ab—have af80d3c0 as an ancestor. The assertion also reports the post-fix line number, so these failures are not the leaked-session issue addressed by
SERVER-132736.
WT-18210landed approximately 12 days before the first post-fix master failure. This is consistent with the suspected interaction but does not establish causality.
- The failure rate is low—a handful of BFGs over approximately three weeks—compared with the roughly 20% failure rate seen in the v9.0 backport-gap failures. This is consistent with a narrow timing window rather than a consistently broken condition.
Proposed fix
Rather than continuing to tune cachePressureEvictionStallThresholdProportion or cachePressureStallDetectionWindowSeconds around changing WiredTiger eviction behavior, keep the drain loop supplied with active cache pressure.
Run a filler thread that continues inserting large documents concurrently with the abort/drain loop. Stop the filler once either of the following occurs:
- numTemporarilyUnavailable > 0; or
- all sessions have been drained.
This would decouple the test from the amount of time WiredTiger's eviction assist happens to block during any individual rollback and should make the test's intended cache-pressure condition deterministic.
- is related to
-
SERVER-132736 transaction_aborted_under_cache_pressure.js leaks sessions at teardown and the cache-pressure detector rarely fires under WT-18120's improved eviction
-
- Closed
-
-
WT-18210 Bound the eviction assist at transaction resolution
-
- Closed
-