Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16902

wt cache: maximum page size at eviction (MB) exceeds memory_page_max by order of magnitude

    • Fully Compatible
    • ALL

      With rc5, WT b-tree and the zlib block compressor there is either a memory leak or WT uses much more memory compared to rc4.

      My 144G test server gets OOM after insert 37M docs and vsz, rss for mongod are 175590976 142382556 (175B, 142B) at that point. With rc4 vsz, rss were 100881768 92328172 (100B, 92B) after 400M inserts.

      The test is run as described in SERVER-16886

      Looking at /proc/smaps there is one huge allocation:

      7f0fcd400000-7f297c000000 rw-p 00000000 00:00 0 
      Size:           107720704 kB
      Rss:            90840720 kB
      Pss:            90840720 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:         0 kB
      Private_Dirty:  90840720 kB
      Referenced:     90840080 kB
      Anonymous:      90840720 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      Locked:                0 kB
      VmFlags: rd wr mr mw me ac ?? 
      

      And one other large allocation

      7f2990000000-7f3999800000 rw-p 00000000 00:00 0 
      Size:           67264512 kB
      Rss:              207524 kB
      Pss:              207524 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:         0 kB
      Private_Dirty:    207524 kB
      Referenced:       190240 kB
      Anonymous:        207524 kB
      AnonHugePages:         0 kB
      Swap:             106344 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      Locked:                0 kB
      VmFlags: rd wr mr mw me ac ?? 
      

      From log, I don't set the cache size, but it used 70G

      2015-01-16T09:06:10.560-0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=70G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      

        1. run.simple.bash.L10.Q1
          4 kB
        2. run.simple.bash.L1.Q10
          4 kB
        3. run.simple.bash
          4 kB
        4. run.simple.bash
          4 kB
        5. run.simple.bash
          4 kB
        6. run.simple.bash
          4 kB
        7. mongo.conf
          0.8 kB
        8. jmongoiibench$MyWriter.class
          4 kB
        9. jmongoiibench$MyReporter.class
          4 kB
        10. jmongoiibench$MyQuery.class
          4 kB
        11. jmongoiibench.java
          35 kB
        12. jmongoiibench.java
          35 kB
        13. jmongoiibench.java
          35 kB
        14. jmongoiibench.java
          35 kB
        15. jmongoiibench.java
          35 kB
        16. jmongoiibench.class
          11 kB

            Assignee:
            david.hows David Hows
            Reporter:
            mdcallag Mark Callaghan
            Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated:
              Resolved: