[SERVER-43902] Apply scaling for totalIndexSize after summing the sizes from individual shards Created: 08/Oct/19  Updated: 29/Oct/23  Resolved: 27/Aug/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Improvement Priority: Major - P3
Reporter: Harshad Dhavale Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-13852 Investigate changes in SERVER-43902: ... Closed
Related
related to SERVER-56780 [4.4] mongod collStats does not handl... Closed
Backwards Compatibility: Minor Change
Sprint: Execution Team 2020-09-07
Participants:
Case:

 Description   

Currently, when we run db.collection.stats() on a sharded cluster, it appears that the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize. For example, as we can observe below,

1) The "totalIndexSize" = 0 = 0 + 0 + 0, which is the sum of the totalIndexSize from each shard.

 

mongos> db.DOB_2565354.stats({scale:1024*1024*1024}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354",... "totalIndexSize" : 0,... "shards" : { "shard0" : {... "totalIndexSize" : 0,... "shard1" : {... "totalIndexSize" : 0,... "shard2" : {... "totalIndexSize" : 0,...

2) The "totalIndexSize" = 73838592 = 49078272 + 16392192 + 8368128, which is the sum of the totalIndexSize from each shard.

 

mongos> db.DOB_2565354.stats({}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354", ..."totalIndexSize" : 73838592,... "shards" : { "shard0" : {... "totalIndexSize" : 49078272,... "shard1" : {... "totalIndexSize" : 16392192,... "shard2" : {... "totalIndexSize" : 8368128,...

 Therefore, the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize.

However, instead of calculating the totalIndexSize this way, it would be more accurate if the totalIndexSize returned was scaled/rounded after summing the size from the individual shards. For example, if there are 3 shards with each shard having an index sized 0.99 GB, then the scale 1024*1024*1024 would round each shard's index size to 0, and so the totalIndexSize reported will be 0 as well. However, if the individual shards' index sizes were to be added first (to yield 2.997 GB) and then the scaling factor was applied to the totalIndexSize, then the totalIndexSize reported will be 2GB (and not 0 GB).

Therefore, instead of applying the scaling factor to the index sizes retrieved for each shards, this improvement request is to scale/round the totalIndexSize only after summing the index sizes from the individual shards.



 Comments   
Comment by Githook User [ 27/Aug/20 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-43902 Apply scaling for collection statistics after summing statistics from individual shards
Branch: master
https://github.com/mongodb/mongo/commit/cefce38b2c914aa2167943e92fb4fe23a1cb9c3b

Generated at Thu Feb 08 05:04:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.