Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4016

Measure and improve lookaside performance with stable_timestamp set

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.6, 4.0.0-rc0, WT3.1.0
    • Component/s: None
    • Labels:
    • Sprint:
      Storage Non-NYC 2018-04-23, Storage Non-NYC 2018-05-07, Storage Non-NYC 2018-05-21

      Description

      When WiredTiger starts evicting using the lookaside table, it makes a choice about whether to "skew newest" or "skew oldest". That determines whether the versions of data written to data files reflects the most recent commit or only stable data (older than the oldest timestamp).

      Checkpoints test each page reference in cache with lookaside history to determine whether the version written to disk matches what checkpoint would write. If not, checkpoint has to reinstantiate the page and write the correct version.

      The skew decision depends on whether a "stable_timestamp" has been set (among other things): the reasoning was that if the stable timestamp is lagging, checkpoint is likely to choose the oldest versions of data when writing pages. Now that MongoDB master is setting a stable timestamp, revisit performance tests that stress lookaside eviction and determine whether the skew choice is optimal.

        Attachments

        1. repro-repl-lag2.sh
          2 kB
        2. Screen Shot 2018-05-09 at 4.00.48 pm.png
          Screen Shot 2018-05-09 at 4.00.48 pm.png
          111 kB
        3. Screen Shot 2018-05-09 at 4.49.07 pm.png
          Screen Shot 2018-05-09 at 4.49.07 pm.png
          135 kB

          Issue Links

            Activity

              People

              Assignee:
              vamsi.krishna Vamsi Boyapati
              Reporter:
              michael.cahill Michael Cahill
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: