[SERVER-52906] moveChunk after failed migration that rolled back cloning indexes can hang indefinitely due to missing shard key index Created: 17/Nov/20  Updated: 29/Oct/23  Resolved: 22/Jul/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.4.2
Fix Version/s: 4.4.8, 5.0.2, 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Paolo Polato
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-sprint
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-55557 Range deletion of aborted migration c... Closed
Problem/Incident
causes SERVER-60142 Shard can migrate on top of orphans a... Closed
Related
related to SERVER-47025 moveChunk after refine shard key can ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0, v4.4
Sprint: Sharding 2020-12-28, Sharding 2021-01-11, Sharding EMEA 2021-06-14
Participants:
Linked BF Score: 140

 Description   

A recipient can end up being unable to process a range deletion task if:

  1. The recipient persists a "pending" range deletion doc for a migration.
  2. The recipient fails over after cloning indexes, but before majority committing that index creation.
  3. On step up, the index creation gets rolled back.
  4. The donor marks the recipient's range deletion doc as no longer pending
  5. The recipient submits the range for deletion from the op observer
  6. The range deletion infinitely fails because the shard key index doesn't exist.

Until the range deletion task is processed, the recipient will be unable to re-receive a chunk that overlaps the range deletion task's range.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 21/Jul/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Delete orphaned document in the recipient before starting the migration.
Branch: v5.0
https://github.com/mongodb/mongo/commit/4a399e6d1f7da8022e7d8d489bb9500ffded6d58

Comment by Githook User [ 21/Jul/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 adapt assert condition to new migrateSteps order

(cherry picked from commit ba9ccb3c9ef2e5322daee08274f713df74524761)
Branch: v5.0
https://github.com/mongodb/mongo/commit/fc7bbb2c39bee08dbb01e978b4aa7d90be2b729e

Comment by Githook User [ 21/Jul/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Applying metadata update before dispatching rollback task in moveChunk

(cherry picked from commit e40a2e909716ecaa82a440e9b7831eae17140212)
Branch: v5.0
https://github.com/mongodb/mongo/commit/fbf4388d075884269706ade02ebecec9ac1ef6df

Comment by Githook User [ 21/Jul/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Applying metadata update before dispatching rollback task in moveChunk (BACKPORT-9612)
Branch: v4.4
https://github.com/mongodb/mongo/commit/d735ffcac376e514f266a315f6405976d603aef6

Comment by Githook User [ 25/Jun/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: Revert "SERVER-52906 Applying metadata update before dispatching rollback task in moveChunk"

This reverts commit 51b58488decbaa166b3b8069cc2b18288899f904.
Branch: v5.0
https://github.com/mongodb/mongo/commit/996f8eb8f5540143790df5cff7f761f82e30444d

Comment by Githook User [ 25/Jun/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: Revert "SERVER-52906 adapt assert condition to new migrateSteps order"

This reverts commit e11f0fe41e64b0274d94c7dfbb45046d4261d379.
Branch: v5.0
https://github.com/mongodb/mongo/commit/4470421a5cb13051cfd5d0fccd48cd0cff72aa98

Comment by Githook User [ 25/Jun/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Delete orphaned document in the recipient before starting the migration.
Branch: master
https://github.com/mongodb/mongo/commit/6ea1a17fd8d1d0f512cfc08750ac19aaaae047aa

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 adapt assert condition to new migrateSteps order

(cherry picked from commit ba9ccb3c9ef2e5322daee08274f713df74524761)
Branch: v5.0
https://github.com/mongodb/mongo/commit/e11f0fe41e64b0274d94c7dfbb45046d4261d379

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 adapt assert condition to new migrateSteps order
Branch: master
https://github.com/mongodb/mongo/commit/ba9ccb3c9ef2e5322daee08274f713df74524761

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Applying metadata update before dispatching rollback task in moveChunk

(cherry picked from commit e40a2e909716ecaa82a440e9b7831eae17140212)
Branch: v5.0
https://github.com/mongodb/mongo/commit/51b58488decbaa166b3b8069cc2b18288899f904

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-52906 Applying metadata update before dispatching rollback task in moveChunk
Branch: master
https://github.com/mongodb/mongo/commit/e40a2e909716ecaa82a440e9b7831eae17140212

Generated at Thu Feb 08 05:29:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.