[SERVER-76779] moveChunk should wait until in-progress index builds are finished on recipient shard Created: 03/May/23 Updated: 29/Oct/23 Resolved: 22/Jun/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.1.0-rc0, 7.0.0-rc6 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Pol Pinol | Assignee: | Pol Pinol |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | sharding-wfbf-day | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Backport Requested: |
v7.0
|
||||||||
| Sprint: | Sharding EMEA 2023-05-15, Sharding EMEA 2023-05-29, Sharding EMEA 2023-06-12, Sharding EMEA 2023-06-26 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 14 | ||||||||
| Description |
|
When there is a moveChunk from one shard to another, in the migration_destination_manager.cpp (cloning indexes phase), the recipient shard is not waiting for in-progress index builds to be finished. More in particular, the logic of filtering the existing indexes on the destination shard, is using removeExistingIndexesNoChecks that filters out read and in-progress indexes. If the recipient shard assumes that in-progress indexes are already created indexes, we could potentially end up in a situation where the index is not finally created in that shard. There is no guarantee that in-progress index build to succeed. The goal of this ticket is to fix that behaviour. In order to make that, we could make moveChunk wait for in-progress index build to be finished on the recipient shard. |
| Comments |
| Comment by Githook User [ 22/Jun/23 ] |
|
Author: {'name': 'Pol Pinol Castuera', 'email': 'pol.pinol@mongodb.com', 'username': 'PolPinol'}Message: (cherry picked from commit 09884999cd6ed51aacf98c9b59851f225d1b8968) |
| Comment by Githook User [ 21/Jun/23 ] |
|
Author: {'name': 'Pol Pinol Castuera', 'email': 'pol.pinol@mongodb.com', 'username': 'PolPinol'}Message: |
| Comment by Max Hirschhorn [ 03/May/23 ] |
|
There is no guarantee for in-progress index builds to succeed in being built on the donor shard. It is always possible for the index build to hit a duplicate key error or some other form of index key constraint. Having the recipient shard copy over in-progress index builds can still lead to index inconsistencies between shards in the other direction. Speculating the index build will end up succeeding may be preferable from a user perspective because probably the most likely outcome for the index build. However I do feel like the ultimate fix for shards having consistent indexes must be in the createIndexes command to have explicit orchestration for index builds across all shards. Also note that refineCollectionShardKey followed by chunk migration leading to a missing shard key index has other impacts tracked in |