shell leaks memory proportional to size of inserted documents

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Shell
    • None
    • Fully Compatible
    • ALL
    • Platforms 17 (07/15/16)
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Use the shell to naively insert 1gb of data:

      var s = (new Array(1048576)).join('x');
      db.test.drop();
      for (i=0; i<1000;i++) db.test.insert({s:s})
      

      While this is running watch the vmem and rss of the shell (eg. in top hit 'm' to sort by memory and 's0.3' to get fast update) — resident will grow to 1gb.

      • Running gc() doesn't help
      • Only affects write commands, does not happen if legacy writes are forced, so highly likely to be in the shell's Bulk API.
      • Seems to affect all shells since 2.6 (write commands), though worse now on mozjs
      • Same for ordered true and false
      • Same for single inserts and multi inserts
      • Variants which also exhibit the problem:
        var foo = null
        for (i=0; i<1000;i++) foo = db.test.insert({s:s})
        foo = null
        gc()
        
        var foo = []
        for (i=0; i<1000;i++) foo.push(db.test.insert({s:s}))
        foo = null
        gc()
        

              Assignee:
              Matt Cotter (Inactive)
              Reporter:
              Kevin Pulo
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

                Created:
                Updated:
                Resolved: