-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Catalog and Routing
-
ALL
-
3
-
🟥 DDL, 🟦 Shard Catalog
-
None
-
None
-
None
-
None
-
None
-
None
Currently the critical section is not held which leads to a existence of a time window, that allows the following scenario to appear:
- In the resharding coordinator state machine as part of it's initialization phase it commits the update in the global catalog about the ownership of the new collection by resharding recipient
- Beginning of the time window: If in this time window router would ask for the collection (Note: only the temporary resharding collection) we're currently resharding, it can target the resharding recipient and despite routing information pointing at the resharding recipient as the owner of the collection we seek, it won't itself recognise the collection, as it's own authoritative information wasn't update (it hasn't pulled it from config). Due to this the recipient would respond with sharding state kUnknown for this collection, raising StaleConfig in the router. After the retry, the router would still continue targeting the recipient repeatedly, as the routing infromation keeps pointing at it as the owner, until all of the retries would be exhausted
- Only after moving on to the different phase, "kReshardingCoordinatorRunUntilReadyToCommit", the coordinator sends a command to make a flush of the routing information on the recipient through a noop write, which has no guarantees to complete
- End of the time window: Only after the recipient applied the command and received the request from the global catalog, the error stops from occurring.
Example of hitting similar issue in production was noticed in SERVER-134127 which led to exclusion of certain checks from jstests to avoid the problem causing BFs.
Note, that this can be abused only for temporary resharding collections, which are not directly visible by users. Therefore it doesn't produce a direct impact exposed to the users. Additionally, the issue does resolve after eventually the recipient fetches the data from the global catalog.
Therefore we should seek to obtain a critical section during this time window and ensure this problem won't surface again.
- is related to
-
SERVER-134127 Exclude temporary resharding collections from the cluster index consistency checks
-
- Closed
-