[SERVER-39021] Switch migrations to observe multi-statement transaction CRUD statements onCommit instead of onCRUD Created: 15/Jan/19 Updated: 29/Oct/23 Resolved: 21/Feb/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 4.1.6 |
| Fix Version/s: | 4.1.9 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Blake Oler | Assignee: | Blake Oler |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Sharding 2019-01-28, Sharding 2019-02-11, Sharding 2019-02-25 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
Problem SummaryWe are proposing removing a collection X-lock in the MigrationSourceManager. This X-lock creates a barrier after which all active transactions on the collection have been committed. This barrier ensures that the migration's chunk cloner will not miss writes that started before the migration. Without this barrier, the cloner will miss transaction writes that start before a migration starts. These writes will not be registered under the LogOpForShardingHandler. Proposed ApproachBecause of this inconsistency, we would like to switch how migrations handle observing CRUD operations in multi-document transactions. After the change in A new method shardObserveTransactionCommit placed on the OpObserverShardingImpl class will do the following:
The previously stated iteration process will do the following with a vector of ReplOperations.
NOTE: We previously stated that we needed an iterator class to analyze the list of replOperations. However, the iteration process is actually very simple – for now, I think an anonymous function in the same file as the OpObserverShardingImpl class will be sufficient. As a result of the previously stated work, we will now be duplicating ops sent to the shard observer – we will be observing every individual CRUD operation, as well as those same operations if they are also inside a multi statement transaction. We don't want the migration to see the same write twice (since these updates are idempotent, it wouldn't be incorrect, just unnecessary). We should only send writes to the cloner if we are not in a multi statement transaction, and a simple check of an already-given boolean will do that. |
| Comments |
| Comment by Githook User [ 21/Feb/19 ] |
|
Author: {'name': 'Blake Oler', 'email': 'blake.oler@mongodb.com', 'username': 'BlakeIsBlake'}Message: |
| Comment by Pavithra Vetriselvan [ 06/Feb/19 ] |
|
blake.oler Yes, feel free to CC me on this review! |
| Comment by Blake Oler [ 06/Feb/19 ] |
|
As part of this ticket, we'd like to change prepared transactions from:
to
So that we can guarantee that we will be in a WriteUnitOfWork during observing the transaction commit. This will match the contract for op observers that observing always happens before storage commits. pavithra.vetriselvan gave the okay to change this ordering. pavithra.vetriselvan@mongodb.com shall I add you as a reviewer to my change for this ticket? |
| Comment by Randolph Tan [ 15/Jan/19 ] |
|
lgtm |
| Comment by Blake Oler [ 15/Jan/19 ] |
|
kaloian.manassiev or renctan, could I get an LGTM on this new approach? |