Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16977

Memory increase trend when running hammar.mongo with WT

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Incomplete
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.8.0-rc5
    • Component/s: Storage, WiredTiger
    • Environment:
      Topology: 3 replica set
      OS: Windows 2012
      Traffic:
      Hammar.mongo

      Command line below:
      -initdb=false –server ip:port –profile=INSERT –totaltime=52000 –worker=90 –rps=0
    • Fully Compatible
    • ALL

      Running on pre RC6 build from (1/20/2015)

      We investigated the followings:

      1) We used a private build w/ separated heap for WT, On this investigation VMMAP indicates that majority of memory increase happen inside of WT code.
      2) We tried to use WT stats, but we failed as the tool doesn’t seems working / usable for this workload.
      3) We reduced the WT cache size to 1 GB and observe the execution can pass this threshold.

      Next steps:

      1) Reduce the amount of load and verify if the memory/resources stabilize (means that evict process is able to keep up 4 threads instead of 90 threads)
      2) If the above steps will stabilize, we will consider to find way to tune the amount of evict threads in WT

        1. WT-HammarMongo.jpg
          WT-HammarMongo.jpg
          195 kB
        2. 34844-diff.txt
          2.00 MB
        3. rs3a-slow100ms.png
          rs3a-slow100ms.png
          56 kB
        4. Report150131.vspx
          5.24 MB
        5. WT_Load1.jpg
          WT_Load1.jpg
          162 kB
        6. CompareBetweenFrashToLoaded.png
          CompareBetweenFrashToLoaded.png
          22 kB
        7. WT_Insert_AfterLoad.vsps
          1.03 MB
        8. Cache_overhead=30%.png
          Cache_overhead=30%.png
          31 kB
        9. Defualt.png
          Defualt.png
          33 kB
        10. 304-Insertonly.png
          304-Insertonly.png
          19 kB
        11. ts1.html
          3.71 MB
        12. iLnx.html
          3.96 MB

            Assignee:
            eitan.klein Eitan Klein
            Reporter:
            eitan.klein Eitan Klein
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: