Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17616

Removing or inserting documents with large indexed arrays consumes excessive memory

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.0.2, 3.1.1
    • Affects Version/s: 3.0.0
    • Component/s: MMAPv1
    • None
    • Fully Compatible
    • ALL
    • Quint Iteration 3.1.1

      Issue Status as of Apr 02, 2015

      ISSUE SUMMARY
      A mongod using the MMAPv1 storage engine may use too much memory to insert and remove documents with large indexed arrays.

      USER IMPACT
      MongoDB consumes an unnecessarily large amount of memory.

      WORKAROUNDS
      None.

      AFFECTED VERSIONS
      MongoDB 3.0.0 and 3.0.1 are affected by this issue.

      FIX VERSION
      The fix is included in the 3.0.2 production release.

      Original description

      Reproduce as follows:

          db.c.ensureIndex({a:1})
          doc = {a:[]}
          for (var i=0; i<100000; i++)
              doc.a.push(i)
          db.c.insert(doc)
          print('before:', db.serverStatus({tcmalloc:1}).tcmalloc.generic.heap_size/1024/1024, 'MB')
          db.c.remove({})
          print('after:', db.serverStatus({tcmalloc:1}).tcmalloc.generic.heap_size/1024/1024, 'MB')
      

      Results:

      3.0, mmapv1, 100k entries:
      before: 124.9453125 MB
      after: 3148.9453125 MB <===
      
      3.0, mmapv1, 50k entries:
      before: 93.234375 MB
      after: 1605.234375 MB <===
      
      3.0, wiredTiger, 100k entries:
      before: 72.28125 MB
      after: 98.53125 MB
      
      2.6.8, mmapv1, 100k entries
      before: 79.4453125 MB
      after: 79.4453125 MB
      
      • memory consumed is proportional to number of index entries, about 30 kB per entry
      • issue is specific to mmapv1, does not occur with WT - so maybe issue is in the journal?
      • problem does not occur in 2.6

            Assignee:
            geert.bosch@mongodb.com Geert Bosch
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: