[SERVER-4023] improve db.stats performance Created: 05/Oct/11  Updated: 06/Dec/22  Resolved: 14/Sep/18

Status: Closed
Project: Core Server
Component/s: MMAPv1, Performance
Affects Version/s: 2.0.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Dominik Gehl Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 5
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux


Issue Links:
Depends
Duplicate
Assigned Teams:
Storage Execution
Participants:
Case:

 Description   

db.stats takes an enormous amount of time to execute. I have around 5000 collections and db.stats takes approx. 30min to execute.

Any reason why it was way faster in 1.8 ?



 Comments   
Comment by Eliot Horowitz (Inactive) [ 21/Nov/11 ]

cache extent meta data in NamespaceDetails

Comment by Dominik Gehl [ 18/Nov/11 ]

Using 8GB of RAM, approx 15 minutes, using 16GB of RAM, still approx 7 minutes

Comment by Scott Hernandez (Inactive) [ 18/Nov/11 ]

How long did those stats take to return?

Comment by Dominik Gehl [ 18/Nov/11 ]

PRIMARY> db.stats()
{
"db" : "db0",
"collections" : 961,
"objects" : 29347980,
"avgObjSize" : 1396.874217441882,
"dataSize" : 40995436596,
"storageSize" : 63500390400,
"numExtents" : 6831,
"indexes" : 9431,
"indexSize" : 19347514704,
"fileSize" : 100816388096,
"nsSizeMB" : 64,
"ok" : 1
}

PRIMARY> db.stats()
{
"db" : "db1",
"collections" : 1852,
"objects" : 29690902,
"avgObjSize" : 1903.0136255207067,
"dataSize" : 56502191060,
"storageSize" : 92368850944,
"numExtents" : 13411,
"indexes" : 30379,
"indexSize" : 31860081456,
"fileSize" : 156623699968,
"nsSizeMB" : 64,
"ok" : 1
}

and two other databases in the same size range

Comment by Scott Hernandez (Inactive) [ 14/Nov/11 ]

Can you post the response from db.stats() so we can see how large and how many collections your db is?

Comment by Dominik Gehl [ 26/Oct/11 ]

In 1.8, I noticed that the 'first' db.stats calls took some time (approx 5-10 min), but that after a few of them, response time became to be only a few seconds. So, it was possible to run monitoring agents in 1.8 (with db.stats), since after a few runs, the response time was 'fast enough' to not generate any impact.

In 2.0, response time does not seem to be related any more to the number of runs: it seems to always take the same amount of time (30min approx)

Comment by Eliot Horowitz (Inactive) [ 26/Oct/11 ]

How much faster in 1.8?
In 2.0 its designed to unlock a lot so as not to impact other operations.

Generated at Thu Feb 08 03:04:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.