[SERVER-34904] recipient shard may be unable to accept versioned requests if it doesn't receive setShardVersion for drop and collection is recreated Created: 08/May/18  Updated: 27/Oct/23  Resolved: 26/Jul/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.6.4, 3.7.9
Fix Version/s: 4.1 Desired, 5.0.0

Type: Bug Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: [DO NOT USE] Backlog - Sharding EMEA
Resolution: Gone away Votes: 0
Labels: sharding-DDL-bugs, sharding-causes-bfs-hard
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-34878 if setShardVersion for dropCollection... Closed
Assigned Teams:
Sharding EMEA
Operating System: ALL
Steps To Reproduce:

Apply this diff and run jstests/sharding/upsert_sharded.js. The diff removes the setShardVersion from dropCollection, but ensures just the primary shard refreshes after the drop in the test.

diff --git a/jstests/sharding/upsert_sharded.js b/jstests/sharding/upsert_sharded.js
index 5d3dde7..b279b02 100644
--- a/jstests/sharding/upsert_sharded.js
+++ b/jstests/sharding/upsert_sharded.js
@@ -68,6 +68,7 @@
     assert.writeError(upsertedResult({"x.x": {$eq: 1}}, {$set: {a: 1}}));     coll.drop();
+    assert.commandWorked(st.shard0.adminCommand({_flushRoutingTableCacheUpdates: coll + ""}));     st.ensurePrimaryShard(coll.getDB() + "", st.shard0.shardName);
     assert.commandWorked(admin.runCommand({shardCollection: coll + "", key: {'x.x': 1}}));
diff --git a/src/mongo/db/s/config/sharding_catalog_manager_collection_operations.cpp b/src/mongo/db/s/config/sharding_catalog_manager_collection_operations.cpp
index 59d2511..b3191d1 100644
--- a/src/mongo/db/s/config/sharding_catalog_manager_collection_operations.cpp
+++ b/src/mongo/db/s/config/sharding_catalog_manager_collection_operations.cpp
@@ -429,57 +429,6 @@ Status ShardingCatalogManager::dropCollection(OperationContext* opCtx, const Nam     LOG(1) << "dropCollection " << nss.ns() << " collection marked as dropped";-    for (const auto& shardEntry : allShards) {
-        auto swShard = shardRegistry->getShard(opCtx, shardEntry.getName());
-        if (!swShard.isOK()) {
-            return swShard.getStatus();
-        }
-
-        const auto& shard = swShard.getValue();
-
-        SetShardVersionRequest ssv = SetShardVersionRequest::makeForVersioningNoPersist(
-            shardRegistry->getConfigServerConnectionString(),
-            shardEntry.getName(),
-            fassert(28781, ConnectionString::parse(shardEntry.getHost())),
-            nss,
-            ChunkVersion::DROPPED(),
-            true);
-
-        auto ssvResult = shard->runCommandWithFixedRetryAttempts(
-            opCtx,
-            ReadPreferenceSetting{ReadPreference::PrimaryOnly},
-            "admin",
-            ssv.toBSON(),
-            Shard::RetryPolicy::kIdempotent);
-
-        if (!ssvResult.isOK()) {
-            return ssvResult.getStatus();
-        }
-
-        auto ssvStatus = std::move(ssvResult.getValue().commandStatus);
-        if (!ssvStatus.isOK()) {
-            return ssvStatus;
-        }
-
-        auto unsetShardingStatus = shard->runCommandWithFixedRetryAttempts(
-            opCtx,
-            ReadPreferenceSetting{ReadPreference::PrimaryOnly},
-            "admin",
-            BSON("unsetSharding" << 1),
-            Shard::RetryPolicy::kIdempotent);
-
-        if (!unsetShardingStatus.isOK()) {
-            return unsetShardingStatus.getStatus();
-        }
-
-        auto unsetShardingResult = std::move(unsetShardingStatus.getValue().commandStatus);
-        if (!unsetShardingResult.isOK()) {
-            return unsetShardingResult;
-        }
-    }
-
-    LOG(1) << "dropCollection " << nss.ns() << " completed";
-
     catalogClient
         ->logChange(opCtx,
                     "dropCollection", 

Participants:
Linked BF Score: 41

 Description   

Let's say a (non-primary shard) recipient shard does not receive setShardVersion on dropCollection and it had cached chunks for the old collection in its config.cache.chunks.<ns>.

The first time it will refresh after the drop and recreate is when a chunk is being migrated to it (because recipient shards refresh at the beginning of _recvChunkStart).

On this refresh, it will see an epoch mismatch and query config.chunks with $gte 0|0. If this is the first chunk to be migrated in the new collection, the version of the chunk with the highest version in config.chunks will be 1|x. The recipient shard will persist this new chunk, but may not delete the "overlapping" (garbage) chunks from the previous collection if the shard key is different (see SERVER-34856).

The next time the recipient will refresh will be on a versioned request. Since its config.cache.chunks.<ns> contains garbage chunks, the one with the highest version will be used for querying config.chuns.

Since the control chunk is the one whose minor version is set to 1 (the migrated chunk's minor version is set to 0), the recipient shard may see only chunks that don't belong to it.

So, in all shardVersion checks, the recipient shard' its CSS will report that it does not own any chunks for the collection.

So, no versioned requests from a fresh mongos will succeed against the recipient shard.



 Comments   
Comment by Kaloian Manassiev [ 26/Jul/21 ]

As of FCV 5.0, the new DDL code paths no longer use setShardVersion so closing this as 'Gone Away'.

Comment by Kaloian Manassiev [ 29/Jun/21 ]

With the new DDL coordinators framework, this will no longer be a problem in 5.0 and later, running on FCV 5.0.

Generated at Thu Feb 08 04:38:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.