[SERVER-70437] Lost writes to unsharded collection during movePrimary Created: 11/Oct/22  Updated: 29/Oct/23  Resolved: 03/Jan/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.4.0, 5.0.0, 6.0.0
Fix Version/s: 6.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: Jordi Serra Torrens Assignee: Daniel Gomez Ferro
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File 0001-Repro-SERVER-70437.patch    
Issue Links:
Problem/Incident
causes SERVER-80463 MigrationChunkClonerSourceOpObserver:... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Execution Team 2022-11-14, Execution Team 2022-12-12, Execution Team 2022-11-28, Execution Team 2022-12-26, Execution Team 2023-01-09
Participants:
Linked BF Score: 35

 Description   

Consider the following interleaving:

1.  [th1] Starts a multi write operation on an unsharded collection and passes the dbVersion check successfully.
2. [th1] The multi write yields.
3. [th2] A movePrimary starts and sets the 'move primary in progress' flag on the DatabaseShardingState.
4. [th2] MovePrimary commits.
5. [th2] MovePrimary unsets the 'move primary in progress'
6. [th2] MovePrimary hangs before dropping the old collection from the former primary.
7. [th1] The multi write now resumes from the yield. Note how the writes never recheck the dbVersion on the op_observers (like we do for the 'shardVersion'). Therefore, the writes don't fail, but are lost because they happened on the old db-primary shard after the ownership change had already committed.

Note that on the op_observer we fail writes if there's a move primary operation in progress. This is what typically prevents losing writes. However, this is only hit if the write restores from yield while the movePrimary is still in progress. In the interleaving above, this does not happen. I don't think this interleaving is very likely, since it requires that the write was yielded for a long time (from the moment the cloning started until after the commit happened).



 Comments   
Comment by Githook User [ 23/Dec/22 ]

Author:

{'name': 'Daniel Gómez Ferro', 'email': 'daniel.gomezferro@mongodb.com', 'username': 'dgomezferro'}

Message: SERVER-70437 Check dbVersion on sharding opobservers
Branch: master
https://github.com/mongodb/mongo/commit/0323cbd6a51391da9f78a43831a78185027fc12c

Generated at Thu Feb 08 06:16:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.