-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: Legacy C++ Implementation
-
Component/s: mongodump
-
None
mongodump seems to leave open noTimeout cursors. The number of cursors left open seems to correlate to the number of collections with more than a small number of documents. Here's a repro below.
Adding a collection with 1000 docs
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> var i = 0
> for (i = 0; i < 1000; i++) { db.test1.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()
The total number of cursors open is 64 (all noTimeout)
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(64),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
After mongodump the number of open cursors goes up by one to 65
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:46:13.014+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
2015-02-18T11:46:13.018+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:46:13.019+0000 1 documents
2015-02-18T11:46:13.019+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:46:13.024+0000 1000 documents
2015-02-18T11:46:13.024+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(65),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
Creating another collection with 1000 documents, now the number of cursors after mongodump goes up by two (65 to 67)
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
Server has startup warnings:
2015-02-17T19:11:17.118+0000 [initandlisten]
2015-02-17T19:11:17.118+0000 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2015-02-17T19:11:17.161+0000 [initandlisten]
2015-02-17T19:11:17.161+0000 [initandlisten] ** WARNING: mongod started without --replSet yet 1 documents are present in local.system.replset
2015-02-17T19:11:17.161+0000 [initandlisten] ** Restart with --replSet unless you are doing maintenance and no other clients are connected.
2015-02-17T19:11:17.161+0000 [initandlisten] ** The TTL collection monitor will not start because of this.
2015-02-17T19:11:17.162+0000 [initandlisten] ** For more info see http://dochub.mongodb.org/core/ttlcollections
2015-02-17T19:11:17.162+0000 [initandlisten]
> use mongodumpTest
switched to db mongodumpTest
> var i = 0
> for (i = 0; i < 1000; i++) { db.test2.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(65),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:47:23.804+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
2015-02-18T11:47:23.804+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:47:23.805+0000 2 documents
2015-02-18T11:47:23.805+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:47:23.807+0000 1000 documents
2015-02-18T11:47:23.807+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
2015-02-18T11:47:23.807+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:47:23.809+0000 1000 documents
2015-02-18T11:47:23.809+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(67),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
After adding another smaller collection (5 documents) running mongodump still causes the number of cursors to increase by two (67 to 69).
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> for (var i = 0; i < 5; i++) { db.test3.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(67),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:55:11.042+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
2015-02-18T11:55:11.082+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:55:11.083+0000 3 documents
2015-02-18T11:55:11.083+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:55:11.086+0000 1000 documents
2015-02-18T11:55:11.086+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
2015-02-18T11:55:11.086+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:55:11.088+0000 1000 documents
2015-02-18T11:55:11.089+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
2015-02-18T11:55:11.089+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:55:11.089+0000 5 documents
2015-02-18T11:55:11.089+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(69),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
If one of the collections is dropped and another mongodump is invoked it causes the number of cursors to increase by one (69 to 70).
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> db.test1.drop()
true
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(69),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:55:53.758+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
2015-02-18T11:55:53.759+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:55:53.760+0000 2 documents
2015-02-18T11:55:53.761+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:55:53.763+0000 1000 documents
2015-02-18T11:55:53.763+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
2015-02-18T11:55:53.763+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:55:53.764+0000 5 documents
2015-02-18T11:55:53.764+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(70),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
After dropping the other larger collection, now mongodump doesn't increase the number of open cursors.
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> db.test2.drop()
true
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(70),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:56:46.209+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
2015-02-18T11:56:46.209+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:56:46.210+0000 1 documents
2015-02-18T11:56:46.210+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:56:46.210+0000 5 documents
2015-02-18T11:56:46.210+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
"timedOut" : NumberLong(0),
"open" : {
"noTimeout" : NumberLong(70),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
}
In summary: mongodump seems to leave open noTimeout cursors for each collection with a significant number of documents.