[SERVER-23421] shell leaks memory proportional to size of inserted documents Created: 30/Mar/16  Updated: 08/Aug/16  Resolved: 18/Jul/16

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kevin Pulo Assignee: Matt Cotter
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-25477 JS does not GC properly Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Platforms 17 (07/15/16)
Participants:

 Description   

Use the shell to naively insert 1gb of data:

var s = (new Array(1048576)).join('x');
db.test.drop();
for (i=0; i<1000;i++) db.test.insert({s:s})

While this is running watch the vmem and rss of the shell (eg. in top hit 'm' to sort by memory and 's0.3' to get fast update) — resident will grow to 1gb.

  • Running gc() doesn't help
  • Only affects write commands, does not happen if legacy writes are forced, so highly likely to be in the shell's Bulk API.
  • Seems to affect all shells since 2.6 (write commands), though worse now on mozjs
  • Same for ordered true and false
  • Same for single inserts and multi inserts
  • Variants which also exhibit the problem:

    var foo = null
    for (i=0; i<1000;i++) foo = db.test.insert({s:s})
    foo = null
    gc()
    

    var foo = []
    for (i=0; i<1000;i++) foo.push(db.test.insert({s:s}))
    foo = null
    gc()
    



 Comments   
Comment by Matt Cotter [ 18/Jul/16 ]

My suspicion is that gc() was actually reclaiming memory, but the resident size might have not shrunk due to SpiderMonkey's internal data structures. Turning verbose on (i.e. mongo --verbose), shows that the interpreter does indeed think it is reclaiming memory when gc() is called. I've confirmed with one poster that this doesn't reproduce any longer.

Generated at Thu Feb 08 04:03:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.