Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1157

LSM merges in update-heavy workloads

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: WT2.4.0
    • Component/s: None
    • Labels:

      Description

      The voxer-130k-short wtperf run dropped between 10-15% in performance between builds [287] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/287/) and [288] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/288/) (here's the plot, it's the drop at the [end of July] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/plot/).

      The change that's responsible is number 4 in build 288 (839742b, "Switch from using the checkpoint lock to serialize bulk-load close and database checkpoints, to using a new WT_DATA_HANDLE lock").

      Here are the runs:

      build 287:
      read: avg: 21239788
      insert: update: avg: 317920
       
      build 287 + change 1
      read: avg: 21346948
      insert: update: avg: 320332
       
      build 287 + change 4 (skipping changes 2 & 3, that was a change and its reversion)
      read: avg: 18351485
      insert: update: avg: 314058
      

      I've been looking at this, and I'm suspicious there's an open/close pattern in LSM that I don't understand, where the close handle spinlock is being held by checkpoint (implying a bulk-loaded file, I think), and is blocking LSM from making forward progress? It could be the reverse, but that should just slow down a checkpoint, it shouldn't affect the read performance of the run.

      @michaelcahill, any thoughts?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                keith.bostic Keith Bostic
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: