-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Storage Engines - Transactions
-
Fully Compatible
-
ALL
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Summary
jstests/noPassthrough/txns_cache_errors/transaction_aborted_under_cache_pressure.js has been flaky/failing on amazon-linux2023-arm64-static-compile and commit-queue since WT-18120 landed (tracked as BF-45315). Root cause is two long-standing test defects that WT-18120 pushed from rare to frequent — not a WiredTiger bug.
What fails in CI
On the mainline failure the cache-pressure mechanism worked correctly (successfulKills > 0), the fill loop exited normally, and the TemporarilyUnavailable assertion passed. The test then died in teardown:
- The fill loop opens one session per iteration and never calls endSession().
- After stopSet() kills the server, the shell's GC finalizer retries endSessions against a dead server once per leaked session per second (920 lines of log id 22791 on the failing run).
- resmoke's hang analyzer SIGABRTs the shell. Task wall clock 1h53m vs ~38s normally.
Why lowering cache_size does not help
isUnderCachePressure() (wiredtiger_cache_pressure_monitor.cpp:69) is threadPressureResult && cacheRatioResult. cache_size only affects the cacheRatioResult half; that half was never the blocker — measured at 8M it was satisfied 12x more often than in a successful 16M run, yet the detector never fired in 12,615 fill-loop iterations (vs firing after 1,689 at 16M).
The binding condition is threadPressureResult, which requires an application thread to be blocked on eviction for ≥95% of every sampling interval (cachePressureEvictionStallThresholdProportion, default 0.95). That is a time condition; cache size cannot produce it.
Where WT-18120 comes in
Instrumented before/after runs (same test, same 16M config, only the WT-18120 import commit differs):
| metric | before |
after |
|---|---|---|
| iterations to trigger (N) | 6 | 14,289 |
| test duration | 12.0 s | 357.9 s |
| app-thread eviction stall | ≈0.73 s/s | ≈0.44 s/s |
| detector trigger rate (passes/s) | 0.334 | 0.0056 (59x lower) |
| observed/threshold ratio | max 0.45 | p50 0.49, max 0.88 (bar is 0.95) |
WT-18120 is a genuine eviction improvement — stall time per second dropped. That is exactly why the test breaks: its trigger condition is "an application thread is stalled," and the improvement removes the stall.
Test-side fix (being landed via the linked PR)
- sessions[i].endSession() in the drain loop — removes the teardown storm.
- cachePressureEvictionStallThresholdProportion 0.95→0.3 — measured: 100% of samples sit above 0.4, so 0.95 is unreachable for this single-threaded workload.
- cachePressureQueryPeriodMilliseconds 1000→20 and CachePressureAbortSessionKillLimitPerBatch 20→1 — the drain loop only lasts a few hundred ms, so at the default period/batch the background killer thread either never completes a cycle or clears the whole backlog before the client's own abort is in flight, so the client never observes TemporarilyUnavailable. Verified: client now observes TemporarilyUnavailable: ... caused by :: Transaction aborted due to cache pressure on 2 of 4 sessions with zero NoSuchTransaction, up from 2 of 1,689 (0.12%) on the mainline run — the assertion passes on its own merits, not by luck. cache_size stays at 16M and the assertion is unchanged (deliberately not relaxed to accept NoSuchTransaction, since that would let a run where cache pressure never fired at all pass silently).
- is related to
-
SERVER-127492 Fix transaction_aborted_under_cache_pressure.js test hang caused by WT-17598 eviction efficiency improvement
-
- Closed
-
-
WT-18120 Prevent eviction walk starvation for a tree dominating cache usage
-
- Closed
-