-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: 2.0.37
-
Component/s: None
-
Environment:Ubuntu 12.04, Node 0.10.38
-
Empty show more show less
I came across this issue in production, after migrating from v1.4.
Using the old driver we would create a connection to a server, then repeatedly switch databases on the same server.
e.g. I have a list of items, each goes to a collection in a different DB:
var db; // Initial Connect Using MongoClient, keep a reference to the original DB MongoClient.connect(url, options, function(err, initialDB){ db = initialDB; // For each item, change the DB and collection db = db.db(dbNameForThisItem); var collection = db.collection(collectionNameForThisItem); // And insert the item collection.insert(item);
This works fine for version 1.4.x, but in 2.0.x the memory usage increases over time (and fairly rapidly).
I've created a repo with a test app, which switches DB in this way, dumping out the heap usage and RSS (and deltas from a baseline) every 10k insertions and dumps the heap at the end.
https://github.com/dhendo/node-mongodb-memorytest
To use:
make v1
to run against v1.4.37
make v2
to run against v2.0.39
Sample output from v2:
50.00k Collections 201.09MB Heap 225.48MB RSS ------------------ 177.56MB Heap Delta 3.55MB Heap / 1000 collections ------------------ 193.36MB RSS Delta 3.87MB RSS / 1000 collections ------------------ { rss: 236437504, heapTotal: 210854144, heapUsed: 166121040 }
v1:
50.00k Collections 29.69MB Heap 52.50MB RSS ------------------ 5.09MB Heap Delta 0.10MB Heap / 1000 collections ------------------ 20.89MB RSS Delta 0.42MB RSS / 1000 collections ------------------ { rss: 55046144, heapTotal: 31127296, heapUsed: 10873576 }
It appears that this is due to the way that DB.db() now works - each call to it adds a reference to the "parentDb", and returns a new db object. This results in a long chain of parent/child relationships when .db() is used in this way. There isn't a leak as such, as the parentDb reference keeps all of them in scope.
Possible Options:
- Change the way the referencing works, so that the new parentDb is always set as the "root" db e.g.
options.parentDb = this.parentDb || this;
This will probably only flatten out the structure, not simplify it.
- Introduce some level of caching on the db instances (and introduce an option to return a non-cached instance.
- Kill the parent/child link so the GC can clean up unused db instances
- Document this behaviour of Db.db() with a warning so that heavy users can code around it