ExportXMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • None
    • Storage Engines - Transactions
    • 186.217
    • None
    • None

      Background

      FTDC only has global cache numbers. It cannot show whether cache content is concentrated in a few large trees or spread thinly over many trees. WT-18400 (develop only) reports the share of the top 5 and top 32 tables, which shows the head of the distribution. This ticket adds the tail: how many trees hold anything evictable, and how many have only a root page in cache.

      Statistics to add

      For each tree, take three quantities, using the same getters as __evict_walk_target:

      • evictable bytes, excluding the root page;
      • dirty leaf bytes;
      • update bytes.

      New gauges:

      1. Number of trees with a non-zero value, one gauge for each of the three quantities.
      2. Number of trees with no evictable bytes (root page only). This is the population the eviction handle scan skips without result.
      3. Number of trees, and the bytes they hold, in a small fixed set of size buckets. Suggested boundaries: 0, up to 64 KB, 1 MB, 16 MB, 256 MB, above. Final boundaries are left to the implementer; the gauge count follows the bucket count.

      Metadata and history store btrees are excluded.

      Collection

      • A pass over the handle list from the sweep server, next to the existing cache-top maintenance call (which skips sweeps while a checkpoint is gathering handles), only when statistics are enabled.
      • The sweep walk (__sweep_mark) already walks the list forward without the handle list lock, so this pass needs none.
      • No change to the per-page byte accounting hot path, and no new work in the eviction server.
      • Gauges use no_clear,no_scale.

      Open points for the implementer

      • The evictable-bytes quantity reads the root page footprint. The sweep walk holds neither the walk lock nor a page reference, so the root page may be discarded concurrently. This is the same root page lifetime question as WT-18986 (skip idle trees in the eviction handle scan); use the protection it settles on, or define the root-only test from counters that do not need the page.
      • The cost of one pass is not measured. It should not exceed the existing time-of-death marking pass, which walks the same list.

      Definition of done

      • Consistency: the tree counts of all size buckets add up to the number of open btrees (excluding metadata and history store).
      • A Python test checks that the non-zero counts, the root-only count and the bucket counts move when tables are created, written and evicted.
      • Cost of one pass measured with tens of thousands to about 150k open handles. Update throughput shows no measurable change with statistics enabled versus disabled.
      • Statistics described in stat_data.py (s_all clean).
      • Backport to 8.0. It needs only the sweep server, which exists in 8.0; the WT-18400 cache-top code is not required.

      Related: WT-18829

            Assignee:
            Unassigned
            Reporter:
            Shoufu Du
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: