Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2902

Enhance eviction to work better with update heavy YCSB load

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.10, 3.3.14
    • Labels:
      None
    • # Replies:
      15
    • Last comment by Customer:
      true

      Description

      The recent eviction and checkpoint enhancements have led to a performance degradation in a particular YCSB workload. We need to investigate improving the performance for that workload.

      The workload in question is a 50% read 50% update workload. The workload fits entirely in cache, and no eviction is needed in the steady state. When a checkpoint starts, the checkpoint scrub phase causes application threads to become involved in eviction which leads to a drop in throughput until the checkpoint dirty target is reached.

      1. 2902ycsb.40.tar.gz
        603 kB
        Sue LoVerso
      2. 2hr.tar.gz
        1.84 MB
        Sue LoVerso
      3. enospc.tar.gz
        571 kB
        Sue LoVerso
      1. 2902.tcmalloc.40min.png
        153 kB
      2. 2902.ycsb.40min.png
        188 kB

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036)

          Some workloads create "deserts" within a tree where no good eviction
          candidates can be found (e.g., zipfian distributions, append-only
          workloads, etc.). Rather than walking all the way through such
          regions, give up and try in the opposite direction next time.

          Also add a statistic for the eviction empty score.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036) Some workloads create "deserts" within a tree where no good eviction candidates can be found (e.g., zipfian distributions, append-only workloads, etc.). Rather than walking all the way through such regions, give up and try in the opposite direction next time. Also add a statistic for the eviction empty score. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036)

          Some workloads create "deserts" within a tree where no good eviction
          candidates can be found (e.g., zipfian distributions, append-only
          workloads, etc.). Rather than walking all the way through such
          regions, give up and try in the opposite direction next time.

          Also add a statistic for the eviction empty score.
          Branch: mongodb-3.4
          https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036) Some workloads create "deserts" within a tree where no good eviction candidates can be found (e.g., zipfian distributions, append-only workloads, etc.). Rather than walking all the way through such regions, give up and try in the opposite direction next time. Also add a statistic for the eviction empty score. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger: 5bc03723a7e77c96b1d5e45a97173654872c727a from branch mongodb-3.4

          ref: deeb0f589a..5bc03723a7
          for: 3.3.13

          WT-2859 Improve statistics macros to not collect stats when statistics=none is set
          WT-2894 Create workload that shows negative scaling when overwhelmed
          WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags
          WT-2902 Enhance eviction to work better with update heavy YCSB load
          WT-2905 dead code: MongoDB Coverity #99881
          WT-2908 Add dry-run support to python unit-test
          WT-2910 test_inmem01 can abort due to stuck cache
          WT-2911 add support for gcc6
          WT-2912 make --enable-strict run on the zseries test box
          WT-2913 stuck cache after latest merge
          WT-2914 Fix link error on OS/X for checksum_init call
          WT-2916 whitespace check not working
          Branch: master
          https://github.com/mongodb/mongo/commit/7693fa59c4470db729d85af99fb9cc3b264fa8c8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 5bc03723a7e77c96b1d5e45a97173654872c727a from branch mongodb-3.4 ref: deeb0f589a..5bc03723a7 for: 3.3.13 WT-2859 Improve statistics macros to not collect stats when statistics=none is set WT-2894 Create workload that shows negative scaling when overwhelmed WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags WT-2902 Enhance eviction to work better with update heavy YCSB load WT-2905 dead code: MongoDB Coverity #99881 WT-2908 Add dry-run support to python unit-test WT-2910 test_inmem01 can abort due to stuck cache WT-2911 add support for gcc6 WT-2912 make --enable-strict run on the zseries test box WT-2913 stuck cache after latest merge WT-2914 Fix link error on OS/X for checksum_init call WT-2916 whitespace check not working Branch: master https://github.com/mongodb/mongo/commit/7693fa59c4470db729d85af99fb9cc3b264fa8c8
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036)

          Some workloads create "deserts" within a tree where no good eviction
          candidates can be found (e.g., zipfian distributions, append-only
          workloads, etc.). Rather than walking all the way through such
          regions, give up and try in the opposite direction next time.

          Also add a statistic for the eviction empty score.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2902 Track per-tree the efficiency of eviction walks. (#3036) Some workloads create "deserts" within a tree where no good eviction candidates can be found (e.g., zipfian distributions, append-only workloads, etc.). Rather than walking all the way through such regions, give up and try in the opposite direction next time. Also add a statistic for the eviction empty score. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/71c8bd736c888228d42d9a2ee478380d2224c984
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Import wiredtiger: bb18c439152379fb528af217345be40819b01fc4 from branch mongodb-3.2

          ref: 911c940ada..bb18c43915
          for: 3.2.10

          WT-2859 Improve statistics macros to not collect stats when statistics=none is set
          WT-2894 Create workload that shows negative scaling when overwhelmed
          WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags
          WT-2902 Enhance eviction to work better with update heavy YCSB load
          WT-2905 dead code: MongoDB Coverity #99881
          WT-2908 Add dry-run support to python unit-test
          WT-2910 test_inmem01 can abort due to stuck cache
          WT-2911 add support for gcc6
          WT-2912 make --enable-strict run on the zseries test box
          WT-2913 stuck cache after latest merge
          WT-2914 Fix link error on OS/X for checksum_init call
          WT-2916 whitespace check not working
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/dfca65ea3a32b1e0c832ff7ee2d7b667819ec64d

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Import wiredtiger: bb18c439152379fb528af217345be40819b01fc4 from branch mongodb-3.2 ref: 911c940ada..bb18c43915 for: 3.2.10 WT-2859 Improve statistics macros to not collect stats when statistics=none is set WT-2894 Create workload that shows negative scaling when overwhelmed WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags WT-2902 Enhance eviction to work better with update heavy YCSB load WT-2905 dead code: MongoDB Coverity #99881 WT-2908 Add dry-run support to python unit-test WT-2910 test_inmem01 can abort due to stuck cache WT-2911 add support for gcc6 WT-2912 make --enable-strict run on the zseries test box WT-2913 stuck cache after latest merge WT-2914 Fix link error on OS/X for checksum_init call WT-2916 whitespace check not working Branch: v3.2 https://github.com/mongodb/mongo/commit/dfca65ea3a32b1e0c832ff7ee2d7b667819ec64d

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                26 weeks, 4 days ago
                Date of 1st Reply: