-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Cache and Eviction, Statistics
-
None
-
Storage Engines - Transactions
-
186.217
-
None
-
None
Background
FTDC only has global cache numbers. It cannot show whether cache content is concentrated in a few large trees or spread thinly over many trees. WT-18400 (develop only) reports the share of the top 5 and top 32 tables, which shows the head of the distribution. This ticket adds the tail: how many trees hold anything evictable, and how many have only a root page in cache.
Statistics to add
For each tree, take three quantities, using the same getters as __evict_walk_target:
- evictable bytes, excluding the root page;
- dirty leaf bytes;
- update bytes.
New gauges:
- Number of trees with a non-zero value, one gauge for each of the three quantities.
- Number of trees with no evictable bytes (root page only). This is the population the eviction handle scan skips without result.
- Number of trees, and the bytes they hold, in a small fixed set of size buckets. Suggested boundaries: 0, up to 64 KB, 1 MB, 16 MB, 256 MB, above. Final boundaries are left to the implementer; the gauge count follows the bucket count.
Metadata and history store btrees are excluded.
Collection
- A pass over the handle list from the sweep server, next to the existing cache-top maintenance call (which skips sweeps while a checkpoint is gathering handles), only when statistics are enabled.
- The sweep walk (__sweep_mark) already walks the list forward without the handle list lock, so this pass needs none.
- No change to the per-page byte accounting hot path, and no new work in the eviction server.
- Gauges use no_clear,no_scale.
Open points for the implementer
- The evictable-bytes quantity reads the root page footprint. The sweep walk holds neither the walk lock nor a page reference, so the root page may be discarded concurrently. This is the same root page lifetime question as WT-18986 (skip idle trees in the eviction handle scan); use the protection it settles on, or define the root-only test from counters that do not need the page.
- The cost of one pass is not measured. It should not exceed the existing time-of-death marking pass, which walks the same list.
Definition of done
- Consistency: the tree counts of all size buckets add up to the number of open btrees (excluding metadata and history store).
- A Python test checks that the non-zero counts, the root-only count and the bucket counts move when tables are created, written and evicted.
- Cost of one pass measured with tens of thousands to about 150k open handles. Update throughput shows no measurable change with statistics enabled versus disabled.
- Statistics described in stat_data.py (s_all clean).
- Backport to 8.0. It needs only the sweep server, which exists in 8.0; the
WT-18400cache-top code is not required.
Related: WT-18829
- related to
-
WT-18829 Eviction candidate supply is capped when dirty content is spread across many trees (high data handle count)
-
- Open
-