[SERVER-73848] Hashed shard keys with zones can cause issues with resharding Created: 09/Feb/23 Updated: 29/Oct/23 Resolved: 08/May/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 5.0.0, 6.0.0, 6.3.0-rc0 |
| Fix Version/s: | 7.1.0-rc0, 6.0.7, 5.0.19, 7.0.0-rc3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kshitij Gupta | Assignee: | Kruti Shah |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | neweng, sharding-nyc-subteam3 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Sharding NYC
|
||||||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Backport Requested: |
v7.0, v6.0, v5.0
|
||||||||||||||||||||||||
| Sprint: | Sharding NYC 2023-05-15 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Story Points: | 3 | ||||||||||||||||||||||||
| Description |
|
1. The new zone range can conflict with the old zone range. If a user reshards from a hashed shard key to the same shard key but unhashed, there could be extraneous conflicting documents in the config.tags collection. This could cause issues with the balancer. To reproduce:
^This will cause the old zone range and new zone range to both still exist post-resharding. We don't delete the previous zones because the hashed and unhashed shard key have the same shard key shape.
2. If the zone ranges are the same for the new and old shard key, we will see a duplicate key error like |
| Comments |
| Comment by Githook User [ 06/Jun/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 74383d5529c531973e6a53dbe248b0b67823da14) |
| Comment by Githook User [ 06/Jun/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 5ddce0f5ecb5a199fc3e9ba3b94c6b630a5b7b6d) |
| Comment by Githook User [ 31/May/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 74383d5529c531973e6a53dbe248b0b67823da14) |
| Comment by Githook User [ 31/May/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 5ddce0f5ecb5a199fc3e9ba3b94c6b630a5b7b6d) |
| Comment by Githook User [ 30/May/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 74383d5529c531973e6a53dbe248b0b67823da14) |
| Comment by Githook User [ 30/May/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: (cherry picked from commit 5ddce0f5ecb5a199fc3e9ba3b94c6b630a5b7b6d) |
| Comment by Githook User [ 08/May/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: |
| Comment by Githook User [ 14/Apr/23 ] |
|
Author: {'name': 'Kruti Shah', 'email': 'kruti139@gmail.com', 'username': 'krutishah139'}Message: |
| Comment by Max Hirschhorn [ 29/Mar/23 ] |
|
The core idea of
(There are technically some other writes to config.csrs.indexes and config.placementHistory which also happen in this same replica set transaction but I'm going to gloss over those.) The cleanupSourceConfigCollections() function which runs after the resharding operation has committed would be changed to only delete the config.chunks entries for the original collection because the config.tags entries for the original collection would already have been deleted by #4 and the config.tags entries for the new collection are in their place. The updateTagsDocsForTempNss() function would be changed to run as part of the replica set transaction as #5. Now I'll summarize some additional context on it is expected to be safe to make these changes described above given the work which originally transpired in
|