Loading...

XML

Word

Printable

JSON

Type: Bug
Resolution: Done
Priority: Major - P3
Fix Version/s: None
Affects Version/s: Legacy C++ Implementation
Component/s: mongodump
Labels:
None

mongodump seems to leave open noTimeout cursors. The number of cursors left open seems to correlate to the number of collections with more than a small number of documents. Here's a repro below.

Adding a collection with 1000 docs

$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> var i = 0
> for (i = 0; i < 1000; i++) { db.test1.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()

The total number of cursors open is 64 (all noTimeout)

$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(64),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

After mongodump the number of open cursors goes up by one to 65

$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:46:13.014+0000 DATABASE: mongodumpTest	 to 	mongodumpTest/mongodumpTest
2015-02-18T11:46:13.018+0000 	mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:46:13.019+0000 		 1 documents
2015-02-18T11:46:13.019+0000 	mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:46:13.024+0000 		 1000 documents
2015-02-18T11:46:13.024+0000 	Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(65),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

Creating another collection with 1000 documents, now the number of cursors after mongodump goes up by two (65 to 67)

$ mongo
MongoDB shell version: 2.6.5
connecting to: test
Server has startup warnings: 
2015-02-17T19:11:17.118+0000 [initandlisten] 
2015-02-17T19:11:17.118+0000 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2015-02-17T19:11:17.161+0000 [initandlisten] 
2015-02-17T19:11:17.161+0000 [initandlisten] ** WARNING: mongod started without --replSet yet 1 documents are present in local.system.replset
2015-02-17T19:11:17.161+0000 [initandlisten] **          Restart with --replSet unless you are doing maintenance and no other clients are connected.
2015-02-17T19:11:17.161+0000 [initandlisten] **          The TTL collection monitor will not start because of this.
2015-02-17T19:11:17.162+0000 [initandlisten] **          For more info see http://dochub.mongodb.org/core/ttlcollections
2015-02-17T19:11:17.162+0000 [initandlisten] 
> use mongodumpTest
switched to db mongodumpTest
> var i = 0
> for (i = 0; i < 1000; i++) { db.test2.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(65),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:47:23.804+0000 DATABASE: mongodumpTest	 to 	mongodumpTest/mongodumpTest
2015-02-18T11:47:23.804+0000 	mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:47:23.805+0000 		 2 documents
2015-02-18T11:47:23.805+0000 	mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:47:23.807+0000 		 1000 documents
2015-02-18T11:47:23.807+0000 	Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
2015-02-18T11:47:23.807+0000 	mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:47:23.809+0000 		 1000 documents
2015-02-18T11:47:23.809+0000 	Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(67),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

After adding another smaller collection (5 documents) running mongodump still causes the number of cursors to increase by two (67 to 69).

$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> for (var i = 0; i < 5; i++) { db.test3.insert({test:i}) }
WriteResult({ "nInserted" : 1 })
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(67),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:55:11.042+0000 DATABASE: mongodumpTest	 to 	mongodumpTest/mongodumpTest
2015-02-18T11:55:11.082+0000 	mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:55:11.083+0000 		 3 documents
2015-02-18T11:55:11.083+0000 	mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.bson
2015-02-18T11:55:11.086+0000 		 1000 documents
2015-02-18T11:55:11.086+0000 	Metadata for mongodumpTest.test1 to mongodumpTest/mongodumpTest/test1.metadata.json
2015-02-18T11:55:11.086+0000 	mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:55:11.088+0000 		 1000 documents
2015-02-18T11:55:11.089+0000 	Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
2015-02-18T11:55:11.089+0000 	mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:55:11.089+0000 		 5 documents
2015-02-18T11:55:11.089+0000 	Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(69),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

If one of the collections is dropped and another mongodump is invoked it causes the number of cursors to increase by one (69 to 70).

$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> db.test1.drop()
true
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(69),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:55:53.758+0000 DATABASE: mongodumpTest	 to 	mongodumpTest/mongodumpTest
2015-02-18T11:55:53.759+0000 	mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:55:53.760+0000 		 2 documents
2015-02-18T11:55:53.761+0000 	mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.bson
2015-02-18T11:55:53.763+0000 		 1000 documents
2015-02-18T11:55:53.763+0000 	Metadata for mongodumpTest.test2 to mongodumpTest/mongodumpTest/test2.metadata.json
2015-02-18T11:55:53.763+0000 	mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:55:53.764+0000 		 5 documents
2015-02-18T11:55:53.764+0000 	Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(70),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

After dropping the other larger collection, now mongodump doesn't increase the number of open cursors.

$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use mongodumpTest
switched to db mongodumpTest
> db.test2.drop()
true
> quit()
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(70),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}
$ mongodump -db mongodumpTest -o mongodumpTest
connected to: 127.0.0.1
2015-02-18T11:56:46.209+0000 DATABASE: mongodumpTest	 to 	mongodumpTest/mongodumpTest
2015-02-18T11:56:46.209+0000 	mongodumpTest.system.indexes to mongodumpTest/mongodumpTest/system.indexes.bson
2015-02-18T11:56:46.210+0000 		 1 documents
2015-02-18T11:56:46.210+0000 	mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.bson
2015-02-18T11:56:46.210+0000 		 5 documents
2015-02-18T11:56:46.210+0000 	Metadata for mongodumpTest.test3 to mongodumpTest/mongodumpTest/test3.metadata.json
$ mongo --eval 'printjson(db.serverStatus().metrics.cursor)'
MongoDB shell version: 2.6.5
connecting to: test
{
	"timedOut" : NumberLong(0),
	"open" : {
		"noTimeout" : NumberLong(70),
		"pinned" : NumberLong(0),
		"total" : NumberLong(0)
	}
}

In summary: mongodump seems to leave open noTimeout cursors for each collection with a significant number of documents.

Assignee:: Kyle Erf (Inactive)

Reporter:: Ger Hartnett

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: Feb 18 2015 12:24:08 PM UTC

Updated:: Sep 15 2015 09:27:14 PM UTC

Resolved:: Sep 15 2015 09:27:14 PM UTC

Details

Description

Attachments

Activity

People

Dates