[SERVER-43689] getShardDistribution() incorrectly shows 0 chunks Created: 27/Sep/19  Updated: 29/Oct/23  Resolved: 22/Oct/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 4.3.1

Type: Bug Priority: Major - P3
Reporter: Vlad Rachev (Inactive) Assignee: Alexander Taskov (Inactive)
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File query_fuzzer-524659-1569598253645-0.js     Text File shardDist1.log    
Issue Links:
Related
is related to SERVER-44051 getShardDistribution() does not repor... Closed
is related to SERVER-44052 Inconsistencies in sharded collections Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

python3 buildscripts/resmoke.py --suites=generational_fuzzer --storageEngineCacheSizeGB=1 path/to/query_fuzzer-524659-1569598253645-0.js

Sprint: Sharding 2019-10-21
Participants:

 Description   

Output of getShardDistribution():

data : 63KiB docs : 108 chunks : 0
estimated data per chunk : 0B
estimated docs per chunk : 0
 
data : 53KiB docs : 92 chunks : 0
estimated data per chunk : 0B
estimated docs per chunk : 0
 
Totals
 data : 116KiB docs : 200 chunks : 0
 Shard query_fuzzer-524659-1569598253645-0-rs1 contains 53.86% data, 54% docs in cluster, avg obj size on shard : 597B
 Shard query_fuzzer-524659-1569598253645-0-rs0 contains 46.13% data, 46% docs in cluster, avg obj size on shard : 600B

config.chunks.find({ns: 'fuzzer.fuzzer_coll'}).itcount() = 4 // so there are actually 4, not 0 chunks



 Comments   
Comment by Githook User [ 22/Oct/19 ]

Author:

{'username': 'alextaskov', 'email': 'alex.taskov@mongodb.com', 'name': 'Alex Taskov'}

Message: SERVER-43689 getShardDistribution() incorrectly shows 0 chunks
Branch: master
https://github.com/mongodb/mongo/commit/563dc7451690efa475db5feda913098e777471da

Comment by Matthew Saltz (Inactive) [ 27/Sep/19 ]

We should also check other shell helpers to make sure they conform to the correct schema, as well as jstests which query config.chunks manually, e.g. getSplitKeysForChunks(), and jstests/noPassthroughWithMongod/no_balance_collection.js

Comment by Matthew Saltz (Inactive) [ 27/Sep/19 ]

The query issued by getShardDistribution() is

COMMAND  [conn5] command config.chunks appName: "MongoDB Shell" command: find { find: "chunks", filter: { _id: /^fuzzer\.fuzzer_coll-.*/, shard: "query_fuzzer-524659-1569598253645-0-rs1" }, lsid: { id: UUID("5ff2244a-cba2-4daa-8542-3e127733da82") }, $clusterTime: { clusterTime: Timestamp(1569600184, 224), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "config" } nShards:1 cursorExhausted:1 numYields:0 nreturned:0 reslen:227 protocol:op_msg 0ms

Notice that the filter is

{ _id: /^fuzzer\.fuzzer_coll-.*/, shard: "query_fuzzer-524659-1569598253645-0-rs1" }

Now, compare that to the document in config.chunks for the namespace we're interested in:

{
     "_id" : "5d8e32b2ac1435db017391b4",
     "ns" : "fuzzer.fuzzer_coll",
     "min" : {
         "_id" : NumberLong(0)
     },
     "max" : {
         "_id" : NumberLong("4611686018427387902")
     },
     "shard" : "query_fuzzer-524659-1569598253645-0-rs1",
     "lastmod" : Timestamp(1, 2),
     "lastmodEpoch" : ObjectId("5d8e32b2ac1435db017391b1"),
     "history" : [
         {
             "validAfter" : Timestamp(1569600178, 44),
             "shard" : "query_fuzzer-524659-1569598253645-0-rs1"
         }
     ]
 },

The _id is a normal object id! It turns out we changed the format of config.chunks for the refine shard key project. So, we need to update getShardDistribution to match the new schema.

Generated at Thu Feb 08 05:03:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.