-
Type:
Task
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Cache and Eviction
-
None
-
Storage Engines
-
133.227
-
None
-
None
Objective
Reproduce the large-cache, update-trigger-only failure mode in standalone WiredTiger: update pressure reaches its trigger, but eviction does not drain update bytes effectively, leaving application threads waiting. This is a symptom-reproduction task; the underlying cause is not established.
Target signature
- Large cache: approximately 77 GiB.
- Total cache occupancy remains around 75–85%; dirty content remains at or below 3%; update bytes rise toward the 20% update trigger (accept 18–22%).
- The update trigger advances and the updates-hard state is present; dirty and full-cache triggers do not advance during the same interval.
- Empty score remains at least 95, cache-wait time advances, and application write latency is elevated.
- Require the signature for a 60-second detection window and a following non-overlapping 60-second confirmation window. Do not count windows across process/configuration changes, missing samples, counter resets, or gaps.
Customer reference at peak: updates 20.0%, dirty 1.70%, total occupancy 79.66%, empty score 99–100, and update-hard eviction state. The candidate queue was empty for about 98% of top-ups; roughly 275k pages/s were seen versus 3k queued/s, with about 32 unmodified-page evictions/s. These are mechanism fingerprints to investigate, not independent pass criteria.
Current evidence and investigation
Prior standalone WT 8.3.2 runs did not reproduce the signature: update bytes stayed around 2.3–2.5%, the update trigger did not fire, and cache wait did not increase. The customer workload is primarily in-place modifies plus substantial index insert/remove churn (about 460 document modifies/s within about 1.2k transactions/s), retryable-write activity on a small hot transactions table, and an approximately 300-second oldest-timestamp window. Customer FTDC shows a large internal-page share (about 29%) and poor candidate-walk yield. Tree size, internal-page residency, and candidate-supply limitations remain hypotheses to test, not confirmed causes.
Use the existing Alibaba reproduction investigation as background. Preserve the distinction between customer-measured values, synthetic workload assumptions, and locally reproduced results. Do not claim an exact root cause from matching the symptom alone.
Definition of done
- Produce a repeatable standalone WT workload/configuration and document the exact WT revision, effective settings, workload, and run procedure.
- Demonstrate the update-only trigger signature and cache-wait/latency impact over both required windows, with dirty/full triggers remaining inactive.
- Retain bounded measurements sufficient to show update-byte trajectory, eviction candidate supply and outcomes, and relevant workload rates; explain any remaining difference from the customer signature.
- If the signature cannot be reproduced, document the tested conditions and the strongest remaining gap/hypothesis rather than reporting a match.
Implementation of a long-term eviction redesign is out of scope for this task.
- related to
-
WT-19002 Reproduce the large-cache update-trigger failure mode where update bytes cannot be evicted effectively
-
- Investigating
-