Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16902

wt cache: maximum page size at eviction (MB) exceeds memory_page_max by order of magnitude

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:

      Description

      With rc5, WT b-tree and the zlib block compressor there is either a memory leak or WT uses much more memory compared to rc4.

      My 144G test server gets OOM after insert 37M docs and vsz, rss for mongod are 175590976 142382556 (175B, 142B) at that point. With rc4 vsz, rss were 100881768 92328172 (100B, 92B) after 400M inserts.

      The test is run as described in SERVER-16886

      Looking at /proc/smaps there is one huge allocation:

      7f0fcd400000-7f297c000000 rw-p 00000000 00:00 0 
      Size:           107720704 kB
      Rss:            90840720 kB
      Pss:            90840720 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:         0 kB
      Private_Dirty:  90840720 kB
      Referenced:     90840080 kB
      Anonymous:      90840720 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      Locked:                0 kB
      VmFlags: rd wr mr mw me ac ?? 
      

      And one other large allocation

      7f2990000000-7f3999800000 rw-p 00000000 00:00 0 
      Size:           67264512 kB
      Rss:              207524 kB
      Pss:              207524 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:         0 kB
      Private_Dirty:    207524 kB
      Referenced:       190240 kB
      Anonymous:        207524 kB
      AnonHugePages:         0 kB
      Swap:             106344 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      Locked:                0 kB
      VmFlags: rd wr mr mw me ac ?? 
      

      From log, I don't set the cache size, but it used 70G

      2015-01-16T09:06:10.560-0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=70G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      

        Attachments

        1. jmongoiibench.class
          11 kB
        2. jmongoiibench.java
          35 kB
        3. jmongoiibench.java
          35 kB
        4. jmongoiibench.java
          35 kB
        5. jmongoiibench.java
          35 kB
        6. jmongoiibench.java
          35 kB
        7. jmongoiibench$MyQuery.class
          4 kB
        8. jmongoiibench$MyReporter.class
          4 kB
        9. jmongoiibench$MyWriter.class
          4 kB
        10. mongo.conf
          0.8 kB
        11. run.simple.bash
          4 kB
        12. run.simple.bash
          4 kB
        13. run.simple.bash
          4 kB
        14. run.simple.bash
          4 kB
        15. run.simple.bash.L1.Q10
          4 kB
        16. run.simple.bash.L10.Q1
          4 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: