[SERVER-38311] Investigate changing host type requirement for $out Created: 29/Nov/18  Updated: 29/Oct/23  Resolved: 13/Feb/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.1.9

Type: Task Priority: Major - P3
Reporter: Charlie Swanson Assignee: Charlie Swanson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-12472 Docs for SERVER-38311: Investigate ch... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 2018-12-17, Query 2018-12-31, Query 2019-01-14, Query 2019-01-28, Query 2019-02-11, Query 2019-02-25
Participants:
Linked BF Score: 50

 Description   

During the design we agreed to the following change in merging policy:

We will change the pipeline splitting logic such that $out is not forced to execute in the merger part of the pipeline, and does not cause the pipeline to split. If $out is preceded by stages that can all execute in the shards part, then each shard will be responsible for dispatching writes to the correct shard.



 Comments   
Comment by Githook User [ 13/Feb/19 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-38311 Change out merging strategy

Allows an $out stage to run in parallel on all shards if the target
collection is sharded and so is the input collection to the aggregate.
Branch: master
https://github.com/mongodb/mongo/commit/87194fbe0c24525bc1f2d674012fe6978eca77d2

Comment by Githook User [ 13/Feb/19 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-38311 Conform backup cursor stages to new mergingLogic interface
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/c6497d94d328ee39788a85b882323ebe92a57918

Comment by Charlie Swanson [ 17/Jan/19 ]

Re-opening due to failures in the sharding_csrs_continuous_config_stepdown: https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_display_sharding_csrs_continuous_config_stepdown_fe187bdf22a6c67d5ff6f035d51a308870255e10_19_01_16_20_33_09

Comment by Githook User [ 17/Jan/19 ]

Author:

{'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson'}

Message: Revert " SERVER-38311 Adjust $out merging strategy"

This reverts commit 0cb2195939494660334db8e9d0a52509caeb621c.
Branch: master
https://github.com/mongodb/mongo/commit/97821dfa13cccb5acad6faef0f536e3d5d77d678

Comment by Githook User [ 17/Jan/19 ]

Author:

{'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson'}

Message: Revert "SERVER-38311 Conform to new DocumentSource interface"

This reverts commit 02d19b792c3a48a45b4a2b1ce29998c6a6b04f9e.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/991976694c0ce0fc4bc511a68b812ae7e4e7fd8f

Comment by Githook User [ 16/Jan/19 ]

Author:

{'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson'}

Message: SERVER-38311 Adjust $out merging strategy
Branch: master
https://github.com/mongodb/mongo/commit/0cb2195939494660334db8e9d0a52509caeb621c

Comment by Githook User [ 16/Jan/19 ]

Author:

{'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson'}

Message: SERVER-38311 Conform to new DocumentSource interface

This was shifted around a bit to help change the merging strategy
required to run $out in a sharded cluster
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/02d19b792c3a48a45b4a2b1ce29998c6a6b04f9e

Comment by Charlie Swanson [ 06/Dec/18 ]

This one got a little hairier than originally anticipated. Temporarily pausing progress on this one to get some of the quicker and higher priority things out of the way first.

Generated at Thu Feb 08 04:48:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.