ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • None
    • Storage Engines - Transactions
    • SE Transactions - 2026-12-04
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Background

      jstests/noPassthrough/txns_cache_errors/transaction_aborted_under_cache_pressure.js has been patched three times in a row. Each patch followed a WiredTiger eviction change and tuned the test to the new timing.

      Ticket Trigger What it changed
      SERVER-132736 WT-18120 improved eviction, so the app-thread stall dropped (about 0.73 s/s to 0.44 s/s) and the detector rarely fired. The test also leaked sessions (BF-45315). endSession() in the drain loop; cachePressureEvictionStallThresholdProportion 0.95 to 0.3; cachePressureQueryPeriodMilliseconds 1000 to 20; CachePressureAbortSessionKillLimitPerBatch 20 to 1. Assertion kept strict.
      SERVER-135094 WT-18210 shrank the cache-pressure window left in the drain loop. A filler thread keeps writing through the drain loop; the drain waits for live pressure.
      SERVER-136123 Still flaky on master and v9.0 (BF-46462): every client abort returned NoSuchTransaction because the killer had already aborted the sessions. Assertion now accepts TemporarilyUnavailable or NoSuchTransaction.

      Remaining problems

      1. The test still depends on real eviction timing. isUnderCachePressure() is threadPressureResult && cacheRatioResult. The first half needs an application thread stalled on eviction for a proportion of each sampling window. Any future eviction improvement can shrink that stall again. Every round so far added another knob (threshold, period, batch size, filler thread), so the next eviction change will likely need another one.
      2. SERVER-136123 weakened the test. The fill loop only exits when successfulKills > 0 && sessions.length >= 20, so at least one session is already killed before the drain loop starts. A client abort on that session always returns NoSuchTransaction, regardless of what happens in the drain phase. The widened assertion is therefore almost always true and no longer proves that a client abort raced an in-flight kill, which was the point of the original TemporarilyUnavailable check. NoSuchTransaction can also come from other paths (for example a txn rolled back after a failed insert; the fill loop swallows all insert errors with catch (e) {}).
      3. Coverage is now unclear. The test is meant to cover four behaviours, and the last two patches blurred them: (a) the killer aborts the oldest unprepared txn under pressure; (b) the killer never aborts a prepared txn; (c) a client abort racing an in-flight kill sees TemporarilyUnavailable; (d) a client abort after the kill completed sees NoSuchTransaction.

      Proposal

      Add failpoints so the test controls what it currently hopes for. There is no cache-pressure failpoint today (WiredTigerKVEngine::underCachePressure and WiredTigerCachePressureMonitor have none; killOldestTransaction has none).

      1. WTForceUnderCachePressure in WiredTigerKVEngine::underCachePressure(): while on, return true regardless of WT stats. This removes the dependency on eviction timing, so the stall-proportion tuning, the filler thread and the small cache can go.
      2. A pause failpoint in killOldestTransaction() (kill_sessions_local.cpp) between SessionCatalog::killSession(..., TemporarilyUnavailable) and checkOutSessionForKill(). That window is the race the test wants. The design should confirm that a client abortTransaction that checks out the session in this window returns TemporarilyUnavailable.

      Rewrite the test to cover each outcome separately with exact assertions:

      • Prepared txn is never aborted (keep totalPreparedThenAborted == 0; also check skippedSessions increased).
      • Oldest unprepared txn is killed first (successfulKills +1 per pass; order follows txn duration).
      • Failpoint paused: the client abort returns exactly TemporarilyUnavailable.
      • Failpoint released and the kill completed: the client abort returns exactly NoSuchTransaction, with the kill counter showing the killer did it.
      • Keep endSession() for every session before teardown.
      • Do not widen any assertion to "either code is fine".

      Acceptance criteria

      • The test passes with default WT eviction settings: no cachePressureEvictionStallThresholdProportion override, no filler thread, no dependence on cache size.
      • Each of the four behaviours above has its own assertion.
      • 100 consecutive passes on amazon-linux2023-arm64-static-compile, including builds with WT-18120 and WT-18210.
      • Backport to v9.0 and v8.3 (where the earlier fixes were backported or requested).
      • Keep one separate end-to-end test with real cache pressure so the underCachePressure() monitor path stays covered, with no flaky timing assertions.

      Related

      SERVER-132736, SERVER-135094, SERVER-136123, BF-45315, BF-46462, WT-18120, WT-18210

            Assignee:
            Unassigned
            Reporter:
            Shoufu Du
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: