[SERVER-63878] Make resharding aggregation pipeline emit applyOps documents for committed transactions before doing findAndModify image lookup Created: 22/Feb/22  Updated: 29/Oct/23  Resolved: 07/Mar/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0-rc3

Type: Task Priority: Major - P3
Reporter: Cheahuychou Mao Assignee: Cheahuychou Mao
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-63879 Make findAndModify image lookup aggre... Closed
is depended on by SERVER-63444 [Retryability] Make retryable findAnd... Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2022-03-07
Participants:

 Description   

Having the DocumentSourceReshardingIterateTransaction stage after DocumentSourceFindAndModifyImageLookup stage makes the resharding pipeline unable to forge a pre/post image oplog entry for a non terminal applyOps oplog entry. We should reorder the stages as follows:

  1. [DocumentSourceReshardingIterateTransaction] For committed transactions, emit the applyOps prevOpTime chain in ascending opTime order.
  2. [DocumentSourceFindAndModifyImageLookup] For a needsRetryImage CRUD or applyOps oplog entry, emit an op=n and modify the subsequent the oplog entry to say pre/postImageOpTime
  3.  [DocumentReshardingAddId] Fill in an _id so already-seen oplog entries can be ignored.

Right now the _id adding is done as part of the DocumentSourceReshardingIterateTransaction stage since it is only stage that tracks the transaction commit timestamp. Therefore, for this ordering to work, we need to store the commit timestamp in each applyOps document emitted in the DocumentSourceReshardingIterateTransaction stage and in its forged pre/post image document in the DocumentSourceFindAndModifyImageLookup stage (if any) similar to how was done here.

 



 Comments   
Comment by Githook User [ 07/Mar/22 ]

Author:

{'name': 'Cheahuychou Mao', 'email': 'mao.cheahuychou@gmail.com', 'username': 'cheahuychou'}

Message: SERVER-63878 Make resharding aggregation pipeline emit applyOps documents for committed transactions before doing findAndModify image lookup
Branch: master
https://github.com/mongodb/mongo/commit/9447e18925d66860386ba8f354278679708f9492

Generated at Thu Feb 08 05:58:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.