[SERVER-16353] listDatabases is very slow on wiredtiger primary Created: 30/Nov/14  Updated: 14/May/15  Resolved: 12/Dec/14

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.8.0-rc1
Fix Version/s: 2.8.0-rc3

Type: Bug Priority: Major - P3
Reporter: Cailin Nelson Assignee: Matt Kangas
Resolution: Done Votes: 0
Labels: wiredtiger
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File testListDatabases.js    
Issue Links:
Related
is related to SERVER-17078 show databases taking extraordinarily... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

On a process with 67 databases, the following takes approximately 2 seconds on a wiredTiger primary:

db.runCommand({listDatabases:1});

If I switch the primary to an mmapv1 node, the command returns within a few milliseconds.



 Comments   
Comment by Asya Kamsky [ 14/May/15 ]

paul.reed since this ticket is closed and you would like a different variant of list databases command I would recommend opening a new ticket asking for additional command (or option to command) to return "fast list" of DB names.

Comment by Paul Reed [ 13/May/15 ]

Have to say that this just isn't fixed - or has regressed. Sometimes it will take my server over 2 minutes to return. Especially if I have multiple clients running.
Any chance of providing a listDatabases command which does not get statistics at all. All I want is the names - and possibly a indicator as to whether it has any collections internal.

Comment by Matt Kangas [ 12/Dec/14 ]

Resolved as of d3877ce. With the attached testListDatabases.js, runCommand listDatabases takes:

  • rc2: elapsed 107.555 sec
  • d3877ce: elapsed: 0.222 sec
Comment by Githook User [ 05/Dec/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-16353 cleaned up storage and ident size functions to query individual statistics key instead of entire statistics table
Branch: master
https://github.com/mongodb/mongo/commit/dfbe7eb6f1c92e063f92c4588a334c708b2a8b63

Comment by Matt Kangas [ 04/Dec/14 ]

Simple jstest

Comment by Steve Briskin (Inactive) [ 30/Nov/14 ]

I don't think it's proportional to the number of collections. I have 4 DBs with 1 collection in each and "show dbs" took over 20s to come back.
I had an insert-only workload of ~1M documents into each collection, I stopped the inserts, and did "show dbs" from the shell. It took at least 20s for it to come back. I restarted mongod, which sped up "show dbs" to 5-10s, which is still slow.

Example:

> print(new Date()); printjson(db.adminCommand({listDatabases: 1})); print(new Date())
Sun Nov 30 2014 16:04:55 GMT-0500 (EST)
{
	"databases" : [
		{
			"name" : "db_0",
			"sizeOnDisk" : 26731806720,
			"empty" : false
		},
		{
			"name" : "db_1",
			"sizeOnDisk" : 27244208128,
			"empty" : false
		},
		{
			"name" : "db_2",
			"sizeOnDisk" : 27243417600,
			"empty" : false
		},
		{
			"name" : "db_3",
			"sizeOnDisk" : 27243876352,
			"empty" : false
		},
		{
			"name" : "local",
			"sizeOnDisk" : 32768,
			"empty" : false
		}
	],
	"totalSize" : 108463341568,
	"ok" : 1
}
Sun Nov 30 2014 16:05:23 GMT-0500 (EST)

Comment by Anil Kumar [ 30/Nov/14 ]

Apparently this is the behaviour proportional to number of collections. Even on a single database, larger number of collections will cause slow listDatabases, although there is only one database.

Generated at Thu Feb 08 03:40:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.