[SERVER-55566] Expand invariant checks in ARM to accommodate resharding pipelines Created: 26/Mar/21  Updated: 06/Dec/22

Status: Open
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Bernard Gorman Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: PM-234, needs-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-55330 Rewrite ReshardingOplogFetcher to use... Backlog
Assigned Teams:
Query Execution
Backport Requested:
v5.0
Participants:

 Description   

Currently, the oplog-tracking aggregation used by resharding executes on a dedicated thread. This has some drawbacks, so we would like to change it to use the Pipeline class and API instead. However, this requires a DocumentSourceMergeCursors stage to be prepended to the pipeline, and at present there are some invariants which only expect to see a postBatchResumeToken for a change stream.

Eventually we want to allow the ARM to handle either kind of postBatchResumeToken. However, this will require further engineering effort on the mongoS side and an expansion of the work carried out under SERVER-53534 on the mongoD side. In the short-term, we should relax these constraints to allow resharding postBatchResumeTokens.



 Comments   
Comment by Ana Meza [ 14/Dec/21 ]

Pushing the Fix version to 6.0 Desired, this to allow time for a discussion with Max and agree on the implementation approach

 

 

Comment by Kyle Suarez [ 13/Jul/21 ]

After a discussion with max.hirschhorn it is important to sharding that these changes be backportable to 5.0. I'm requesting a backport as well.

Generated at Thu Feb 08 05:36:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.