Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4016

Measure and improve lookaside performance with stable_timestamp set

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.6.6, 4.0.0-rc0, WT3.1.0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage Non-NYC 2018-04-23, Storage Non-NYC 2018-05-07, Storage Non-NYC 2018-05-21

      When WiredTiger starts evicting using the lookaside table, it makes a choice about whether to "skew newest" or "skew oldest". That determines whether the versions of data written to data files reflects the most recent commit or only stable data (older than the oldest timestamp).

      Checkpoints test each page reference in cache with lookaside history to determine whether the version written to disk matches what checkpoint would write. If not, checkpoint has to reinstantiate the page and write the correct version.

      The skew decision depends on whether a "stable_timestamp" has been set (among other things): the reasoning was that if the stable timestamp is lagging, checkpoint is likely to choose the oldest versions of data when writing pages. Now that MongoDB master is setting a stable timestamp, revisit performance tests that stress lookaside eviction and determine whether the skew choice is optimal.

        1. Screen Shot 2018-05-09 at 4.49.07 pm.png
          Screen Shot 2018-05-09 at 4.49.07 pm.png
          135 kB
        2. Screen Shot 2018-05-09 at 4.00.48 pm.png
          Screen Shot 2018-05-09 at 4.00.48 pm.png
          111 kB
        3. repro-repl-lag2.sh
          2 kB

            Assignee:
            vamsi.krishna@mongodb.com Vamsi Boyapati
            Reporter:
            michael.cahill@mongodb.com Michael Cahill (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: