Label size-stats histogram from WT-published hist_buckets and hist_ceiling

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines - Persistence
    • Fully Compatible
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Problem

      WiredTigerUtil::logStorageSizeStats rebuilds leafPageSizeHistogram bucket maxBytes from WT_STAT_DSRC_BTREE_MAXLEAFPAGE and a hardcoded 9. Size stats measure uncompressed leaf images (dsk->mem_size). With compression adjust, WiredTiger budgets against maxleafpage_precomp (often ~4x maxleafpage, e.g. 128 KiB for 32 KiB collection leaves) then compresses toward maxleafpage on disk.

      After [WT-18417|https://jira.mongodb.org/browse/WT-18417], WT buckets and publishes geometry against that pre-compression ceiling (btree_size_leaf_hist_buckets / btree_size_leaf_hist_ceiling). Mongo still labeling from on-disk maxleafpage makes histogram maxBytes misleading: well-compressing collection leaves look "full" in the open-ended bucket.

      Change

      Mongo-only. Do not vendor WiredTiger.

      In WiredTigerUtil::logStorageSizeStats (src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp, LOGV2 12951900):
      * Prefer WT_STAT_DSRC_BTREE_SIZE_LEAF_HIST_BUCKETS and WT_STAT_DSRC_BTREE_SIZE_LEAF_HIST_CEILING for labels.
      * Loop buckets; finite maxBytes = (ceiling / (buckets - 1)) * (i + 1); last bucket maxBytes: null.
      * Fallback if buckets/ceiling are absent or 0: today's maxleafpage / 9 (old WT binaries, and this tree until WT-18417 is imported).
      * If the new stat macros are not in this tree's wiredtiger.h, compile against a fallback of 0 so the runtime path above is used. Do not vendor src/third_party/wiredtiger/**.

      Keep the log shape {{leafPageSizeHistogram: [

      {maxBytes, count}

      , ...]}}.

      Test

      Extend jstests/noPassthrough/validate/collhash_size_metrics.js:
      * Histogram length is 9.
      * Last finite maxBytes matches the published ceiling when the new stats are present (collection leaves 128 KiB with leafpage_compadjust; indexes still 16 KiB).
      * Until WT-18417 is in the binary, collection last finite maxBytes remains 32 KiB (maxleafpage fallback); the test must pass in both geometries.

      Out of scope

      * Vendoring WT-18417
      * Dynamic bucket count beyond fixed hist_0..8
      * Parser script / overhead formula / history-store metrics

            Assignee:
            Luke Pearson
            Reporter:
            Luke Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: