Details
-
Bug
-
Resolution: Done
-
Major - P3
-
None
-
Legacy C++ Implementation
-
None
Description
mongodump seems to leave open noTimeout cursors. The number of cursors left open seems to correlate to the number of collections with more than a small number of documents. Here's a repro below.
Adding a collection with 1000 docs
$ mongo
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
> use mongodumpTest
|
switched to db mongodumpTest
|
> var i = 0
|
> for (i = 0; i < 1000; i++) { db.test1.insert({test:i}) }
|
WriteResult({ "nInserted" : 1 })
|
> quit()
|
The total number of cursors open is 64 (all noTimeout)
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(64),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
After mongodump the number of open cursors goes up by one to 65
$ mongodump -db mongodumpTest -o mongodumpTest
|
connected to: 127.0.0.1
|
2015-02-18T11:46:13.014+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
|
2015-02-18T11:46:13.018+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
|
2015-02-18T11:46:13.019+0000 1 documents
|
2015-02-18T11:46:13.019+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
|
2015-02-18T11:46:13.024+0000 1000 documents
|
2015-02-18T11:46:13.024+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(65),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
Creating another collection with 1000 documents, now the number of cursors after mongodump goes up by two (65 to 67)
$ mongo
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
Server has startup warnings:
|
2015-02-17T19:11:17.118+0000 [initandlisten]
|
2015-02-17T19:11:17.118+0000 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
|
2015-02-17T19:11:17.161+0000 [initandlisten]
|
2015-02-17T19:11:17.161+0000 [initandlisten] ** WARNING: mongod started without --replSet yet 1 documents are present in local.system.replset
|
2015-02-17T19:11:17.161+0000 [initandlisten] ** Restart with --replSet unless you are doing maintenance and no other clients are connected.
|
2015-02-17T19:11:17.161+0000 [initandlisten] ** The TTL collection monitor will not start because of this.
|
2015-02-17T19:11:17.162+0000 [initandlisten] ** For more info see http://dochub.mongodb.org/core/ttlcollections
|
2015-02-17T19:11:17.162+0000 [initandlisten]
|
> use mongodumpTest
|
switched to db mongodumpTest
|
> var i = 0
|
> for (i = 0; i < 1000; i++) { db.test2.insert({test:i}) }
|
WriteResult({ "nInserted" : 1 })
|
> quit()
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(65),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
$ mongodump -db mongodumpTest -o mongodumpTest
|
connected to: 127.0.0.1
|
2015-02-18T11:47:23.804+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
|
2015-02-18T11:47:23.804+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
|
2015-02-18T11:47:23.805+0000 2 documents
|
2015-02-18T11:47:23.805+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
|
2015-02-18T11:47:23.807+0000 1000 documents
|
2015-02-18T11:47:23.807+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
|
2015-02-18T11:47:23.807+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
|
2015-02-18T11:47:23.809+0000 1000 documents
|
2015-02-18T11:47:23.809+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(67),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
After adding another smaller collection (5 documents) running mongodump still causes the number of cursors to increase by two (67 to 69).
$ mongo
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
> use mongodumpTest
|
switched to db mongodumpTest
|
> for (var i = 0; i < 5; i++) { db.test3.insert({test:i}) }
|
WriteResult({ "nInserted" : 1 })
|
> quit()
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(67),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
$ mongodump -db mongodumpTest -o mongodumpTest
|
connected to: 127.0.0.1
|
2015-02-18T11:55:11.042+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
|
2015-02-18T11:55:11.082+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
|
2015-02-18T11:55:11.083+0000 3 documents
|
2015-02-18T11:55:11.083+0000 mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
|
2015-02-18T11:55:11.086+0000 1000 documents
|
2015-02-18T11:55:11.086+0000 Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
|
2015-02-18T11:55:11.086+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
|
2015-02-18T11:55:11.088+0000 1000 documents
|
2015-02-18T11:55:11.089+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
|
2015-02-18T11:55:11.089+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
|
2015-02-18T11:55:11.089+0000 5 documents
|
2015-02-18T11:55:11.089+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(69),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
If one of the collections is dropped and another mongodump is invoked it causes the number of cursors to increase by one (69 to 70).
$ mongo
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
> use mongodumpTest
|
switched to db mongodumpTest
|
> db.test1.drop()
|
true
|
> quit()
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(69),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
$ mongodump -db mongodumpTest -o mongodumpTest
|
connected to: 127.0.0.1
|
2015-02-18T11:55:53.758+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
|
2015-02-18T11:55:53.759+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
|
2015-02-18T11:55:53.760+0000 2 documents
|
2015-02-18T11:55:53.761+0000 mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
|
2015-02-18T11:55:53.763+0000 1000 documents
|
2015-02-18T11:55:53.763+0000 Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
|
2015-02-18T11:55:53.763+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
|
2015-02-18T11:55:53.764+0000 5 documents
|
2015-02-18T11:55:53.764+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(70),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
After dropping the other larger collection, now mongodump doesn't increase the number of open cursors.
$ mongo
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
> use mongodumpTest
|
switched to db mongodumpTest
|
> db.test2.drop()
|
true
|
> quit()
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(70),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
$ mongodump -db mongodumpTest -o mongodumpTest
|
connected to: 127.0.0.1
|
2015-02-18T11:56:46.209+0000 DATABASE: mongodumpTest to mongodumpTest/mongodumpTest
|
2015-02-18T11:56:46.209+0000 mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
|
2015-02-18T11:56:46.210+0000 1 documents
|
2015-02-18T11:56:46.210+0000 mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
|
2015-02-18T11:56:46.210+0000 5 documents
|
2015-02-18T11:56:46.210+0000 Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
|
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
|
MongoDB shell version: 2.6.5
|
connecting to: test
|
{
|
"timedOut" : NumberLong(0),
|
"open" : {
|
"noTimeout" : NumberLong(70),
|
"pinned" : NumberLong(0),
|
"total" : NumberLong(0)
|
}
|
}
|
In summary: mongodump seems to leave open noTimeout cursors for each collection with a significant number of documents.