Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16977

Memory increase trend when running hammar.mongo with WT

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical - P2
    • Resolution: Incomplete
    • 2.8.0-rc5
    • None
    • Storage, WiredTiger
    • Topology: 3 replica set
      OS: Windows 2012
      Traffic:
      Hammar.mongo

      Command line below:
      -initdb=false –server ip:port –profile=INSERT –totaltime=52000 –worker=90 –rps=0
    • Fully Compatible
    • ALL

    Description

      Running on pre RC6 build from (1/20/2015)

      We investigated the followings:

      1) We used a private build w/ separated heap for WT, On this investigation VMMAP indicates that majority of memory increase happen inside of WT code.
      2) We tried to use WT stats, but we failed as the tool doesn’t seems working / usable for this workload.
      3) We reduced the WT cache size to 1 GB and observe the execution can pass this threshold.

      Next steps:

      1) Reduce the amount of load and verify if the memory/resources stabilize (means that evict process is able to keep up 4 threads instead of 90 threads)
      2) If the above steps will stabilize, we will consider to find way to tune the amount of evict threads in WT

      Attachments

        1. WT-HammarMongo.jpg
          WT-HammarMongo.jpg
          195 kB
        2. 34844-diff.txt
          2.00 MB
        3. rs3a-slow100ms.png
          rs3a-slow100ms.png
          56 kB
        4. Report150131.vspx
          5.24 MB
        5. WT_Load1.jpg
          WT_Load1.jpg
          162 kB
        6. CompareBetweenFrashToLoaded.png
          CompareBetweenFrashToLoaded.png
          22 kB
        7. WT_Insert_AfterLoad.vsps
          1.03 MB
        8. Cache_overhead=30%.png
          Cache_overhead=30%.png
          31 kB
        9. Defualt.png
          Defualt.png
          33 kB
        10. 304-Insertonly.png
          304-Insertonly.png
          19 kB
        11. ts1.html
          3.71 MB
        12. iLnx.html
          3.96 MB

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-17456 Mongodb 3.0 wiredTiger storage engine memory usage too high.

          • Critical - P2 - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-16902 wt cache: maximum page size at eviction (MB) exceeds memory_page_max by order of magnitude

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-16941 Cache can grow indefinitely under WiredTiger

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-17421 WiredTiger b-tree uses much more memory than wiredTigerCacheSizeGB

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-17424 WiredTiger uses substantially more memory than accounted for by cache

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              eitan.klein Eitan Klein
              eitan.klein Eitan Klein
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: