[DOCS-13852] Investigate changes in SERVER-43902: Apply scaling for totalIndexSize after summing the sizes from individual shards Created: 27/Aug/20  Updated: 13/Nov/23  Resolved: 16/Jun/21

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 4.7.0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Andrew Feierabend (Inactive)
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-43902 Apply scaling for totalIndexSize afte... Closed
Participants:
Days since reply: 3 years, 24 weeks ago
Epic Link: DOCSP-9747

 Description   

Description

Downstream Change Summary

Scaling for collection statistics is applied after summing statistics from individual shards as opposed to summing the scaled statistics of each individual shard.

Description of Linked Ticket

Currently, when we run db.collection.stats() on a sharded cluster, it appears that the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize. For example, as we can observe below,

1) The "totalIndexSize" = 0 = 0 + 0 + 0, which is the sum of the totalIndexSize from each shard.

 

mongos> db.DOB_2565354.stats({scale:1024*1024*1024}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354",... "totalIndexSize" : 0,... "shards" : { "shard0" : {... "totalIndexSize" : 0,... "shard1" : {... "totalIndexSize" : 0,... "shard2" : {... "totalIndexSize" : 0,...

2) The "totalIndexSize" = 73838592 = 49078272 + 16392192 + 8368128, which is the sum of the totalIndexSize from each shard.

 

mongos> db.DOB_2565354.stats({}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354", ..."totalIndexSize" : 73838592,... "shards" : { "shard0" : {... "totalIndexSize" : 49078272,... "shard1" : {... "totalIndexSize" : 16392192,... "shard2" : {... "totalIndexSize" : 8368128,...

 Therefore, the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize.

However, instead of calculating the totalIndexSize this way, it would be more accurate if the totalIndexSize returned was scaled/rounded after summing the size from the individual shards. For example, if there are 3 shards with each shard having an index sized 0.99 GB, then the scale 1024*1024*1024 would round each shard's index size to 0, and so the totalIndexSize reported will be 0 as well. However, if the individual shards' index sizes were to be added first (to yield 2.997 GB) and then the scaling factor was applied to the totalIndexSize, then the totalIndexSize reported will be 2GB (and not 0 GB).

Therefore, instead of applying the scaling factor to the index sizes retrieved for each shards, this improvement request is to scale/round the totalIndexSize only after summing the index sizes from the individual shards.

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)


Generated at Thu Feb 08 08:08:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.