Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-11413

Add histogram stats for compression ratio of blocks written to disk

XMLWordPrintableJSON

    • Storage Engines
    • ASeasonTooMany-2023-08-22, BermudaTriangle- 2023-09-05, TheMoon-StorEng - 2023-09-19, NachoCheese - 2023-10-03, Joker - StorEng - 2023-10-17, Asparagus-StorEng - 2023-10-31, c(3x10^8)-StorEng - 2023-11-14, 2023-11-28 - Anthill Tiger, 2023-12-12 - Heisenbug
    • 3
    • Triage and Release
    • Hide
      This ticket changes some T2 stats and adds a few more.

      The following descriptions are longer present
      'number of blocks with compress ratio smaller than 2'
      'number of blocks with compress ratio smaller than 4'
      'number of blocks with compress ratio smaller than 8'
      'number of blocks with compress ratio smaller than 16'
      'number of blocks with compress ratio smaller than 32'
      'number of blocks with compress ratio smaller than 64'
      'number of blocks with compress ratio greater than 64'
      'compressed pages read'
      'compressed pages written'
      'page written failed to compress'
      'page written was too small to compress'

      and have been replaced with the following
      'pages read from disk with compression ratio smaller than 2'
      'pages read from disk with compression ratio smaller than 4'
      'pages read from disk with compression ratio smaller than 8'
      'pages read from disk with compression ratio smaller than 16'
      'pages read from disk with compression ratio smaller than 32'
      'pages read from disk with compression ratio smaller than 64'
      'pages read from disk with compression ratio greater than 64'
      'pages written to disk'
      'pages written to disk with compression ratio smaller than 2'
      'pages written to disk with compression ratio smaller than 4'
      'pages written to disk with compression ratio smaller than 8'
      'pages written to disk with compression ratio smaller than 16'
      'pages written to disk with compression ratio smaller than 32'
      'pages written to disk with compression ratio smaller than 64'
      'pages written to disk with compression ratio greater than 64'
      'page written to disk failed to compress'
      'page written to disk was too small to compress'
      Show
      This ticket changes some T2 stats and adds a few more. The following descriptions are longer present 'number of blocks with compress ratio smaller than 2' 'number of blocks with compress ratio smaller than 4' 'number of blocks with compress ratio smaller than 8' 'number of blocks with compress ratio smaller than 16' 'number of blocks with compress ratio smaller than 32' 'number of blocks with compress ratio smaller than 64' 'number of blocks with compress ratio greater than 64' 'compressed pages read' 'compressed pages written' 'page written failed to compress' 'page written was too small to compress' and have been replaced with the following 'pages read from disk with compression ratio smaller than 2' 'pages read from disk with compression ratio smaller than 4' 'pages read from disk with compression ratio smaller than 8' 'pages read from disk with compression ratio smaller than 16' 'pages read from disk with compression ratio smaller than 32' 'pages read from disk with compression ratio smaller than 64' 'pages read from disk with compression ratio greater than 64' 'pages written to disk' 'pages written to disk with compression ratio smaller than 2' 'pages written to disk with compression ratio smaller than 4' 'pages written to disk with compression ratio smaller than 8' 'pages written to disk with compression ratio smaller than 16' 'pages written to disk with compression ratio smaller than 32' 'pages written to disk with compression ratio smaller than 64' 'pages written to disk with compression ratio greater than 64' 'page written to disk failed to compress' 'page written to disk was too small to compress'

      There are a set of statistics that track block compression ratio - it was observed that they are always set to 0, even when pages are clearly being compressed. For example:

                              "compressed pages written" : 110204,
                        "number of blocks with compress ratio greater than 64" : 0,
                        "number of blocks with compress ratio smaller than 16" : 0,
                        "number of blocks with compress ratio smaller than 2" : 0,
                        "number of blocks with compress ratio smaller than 32" : 0,
                        "number of blocks with compress ratio smaller than 4" : 0,
                        "number of blocks with compress ratio smaller than 64" : 0,
                        "number of blocks with compress ratio smaller than 8" : 0,
                        "page written failed to compress" : 1581,
                        "page written was too small to compress" : 1077

      It appears as though the change for WT-6022, where a call to wt_stat_compr_ratio_hist_incr was replaced with a macro (WT_STAT_COMPR_RATIO_HIST_INCR_FUNC) that doesn't appear to be used.

      We should fix the code to increment the statistics again, and also add a test to ensure we don't break them again.

        1. stats_all.png
          stats_all.png
          506 kB
        2. stats_sample.png
          stats_sample.png
          252 kB

            Assignee:
            andrew.morton@mongodb.com Andrew Morton
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: