Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6664

Cache eviction causes high latency during py-tpcc.

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Duplicate
    • None
    • None
    • None
    • None
    • 8
    • Storage - Ra 2020-10-05, Storage - Ra 2020-10-19, Storage - Ra 2020-11-02, Storage - Ra 2020-11-16, Storage - Ra 2020-11-30, Storage - Ra 2021-01-11, Storage - Ra 2021-11-15

    Description

      SERVER-50749 identifies some performance problems when loading data in py-tpcc.

      In attempting to reproduce the problem, I saw some high latency  insert operations that seem to be caused by WT cache eviction pressure.

      In particular, during a 25 minute test run, I saw 15 operations of 16+ seconds. All of these stalls correlated with aggressive eviction, high numbers of pages evicted by application threads, and evictions requiring restoration.

      Attachments

        1. 4-2-10-vs-master-closeup.png
          4-2-10-vs-master-closeup.png
          420 kB
        2. 4-2-10-vs-master-flow.png
          4-2-10-vs-master-flow.png
          232 kB
        3. compare_different_versions.png
          compare_different_versions.png
          301 kB
        4. compare-4.2and5.png
          compare-4.2and5.png
          71 kB
        5. compare-all.png
          compare-all.png
          257 kB
        6. flowcontrol-compare.png
          flowcontrol-compare.png
          216 kB
        7. flow-on-vs-flow-off.png
          flow-on-vs-flow-off.png
          194 kB
        8. metrics.2020-09-09T20-02-23Z-00000
          1.56 MB
        9. Screen Shot 2020-09-09 at 6.16.19 PM.png
          Screen Shot 2020-09-09 at 6.16.19 PM.png
          157 kB
        10. Screen Shot 2021-10-11 at 12.03.47 pm.png
          Screen Shot 2021-10-11 at 12.03.47 pm.png
          47 kB
        11. Screen Shot 2021-10-11 at 2.30.05 pm.png
          Screen Shot 2021-10-11 at 2.30.05 pm.png
          54 kB
        12. Screen Shot 2021-10-11 at 6.15.25 pm.png
          Screen Shot 2021-10-11 at 6.15.25 pm.png
          78 kB
        13. Screen Shot 2021-10-12 at 11.16.57 am.png
          Screen Shot 2021-10-12 at 11.16.57 am.png
          56 kB
        14. Screen Shot 2021-11-12 at 4.29.02 pm.png
          Screen Shot 2021-11-12 at 4.29.02 pm.png
          301 kB

        Issue Links

          Activity

            People

              sulabh.mahajan@mongodb.com Sulabh Mahajan
              keith.smith@mongodb.com Keith Smith
              Haribabu Kommi
              Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: