Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-13852

Investigate changes in SERVER-43902: Apply scaling for totalIndexSize after summing the sizes from individual shards

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.7.0
    • Component/s: manual, Server
    • Labels:
      None

      Description

      Description

      Downstream Change Summary

      Scaling for collection statistics is applied after summing statistics from individual shards as opposed to summing the scaled statistics of each individual shard.

      Description of Linked Ticket

      Currently, when we run db.collection.stats() on a sharded cluster, it appears that the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize. For example, as we can observe below,

      1) The "totalIndexSize" = 0 = 0 + 0 + 0, which is the sum of the totalIndexSize from each shard.

       

      mongos> db.DOB_2565354.stats({scale:1024*1024*1024}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354",... "totalIndexSize" : 0,... "shards" : { "shard0" : {... "totalIndexSize" : 0,... "shard1" : {... "totalIndexSize" : 0,... "shard2" : {... "totalIndexSize" : 0,...

      2) The "totalIndexSize" = 73838592 = 49078272 + 16392192 + 8368128, which is the sum of the totalIndexSize from each shard.

       

      mongos> db.DOB_2565354.stats({}){ "sharded" : true, "capped" : false, "ns" : "test.DOB_2565354", ..."totalIndexSize" : 73838592,... "shards" : { "shard0" : {... "totalIndexSize" : 49078272,... "shard1" : {... "totalIndexSize" : 16392192,... "shard2" : {... "totalIndexSize" : 8368128,...

       Therefore, the scale factor gets "passed down" to calculate each shards' index size, and then those individual sizes get summed up to give the totalIndexSize.

      However, instead of calculating the totalIndexSize this way, it would be more accurate if the totalIndexSize returned was scaled/rounded after summing the size from the individual shards. For example, if there are 3 shards with each shard having an index sized 0.99 GB, then the scale 1024*1024*1024 would round each shard's index size to 0, and so the totalIndexSize reported will be 0 as well. However, if the individual shards' index sizes were to be added first (to yield 2.997 GB) and then the scaling factor was applied to the totalIndexSize, then the totalIndexSize reported will be 2GB (and not 0 GB).

      Therefore, instead of applying the scaling factor to the index sizes retrieved for each shards, this improvement request is to scale/round the totalIndexSize only after summing the index sizes from the individual shards.

      Scope of changes

      Impact to Other Docs

      MVP (Work and Date)

      Resources (Scope or Design Docs, Invision, etc.)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              backlog-server-pm Backlog - DB Eng Program Management Team
              Participants:
              Last commenter:
              Backlog - DB Eng Program Management Team
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Days since reply:
                7 weeks, 6 days ago