[SERVER-35419] Make diskRequirement in StageConstraints in DocumentSource reflect child pipeline requirements. Created: 05/Jun/18 Updated: 29/Oct/23 Resolved: 14/Sep/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.4 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Sam Mercier | Assignee: | Kyle Suarez |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | read-only-views | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.0
|
||||||||||||||||||||||||||||||||
| Sprint: | Query 2018-09-24 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Description |
|
Currently, if a stage (e.g. `$lookup`) has a "sub" pipeline that requires it to write persistent data, the stage's diskRequirement doesn't reflect that. Instead the parent stage should have the max diskRequirement of it's pipeline's stages under the total ordering
e.g. of the bug https://github.com/mongodb/mongo/blob/b93fe0e61bf7e8bc96da2edeb66afa1b915b0b77/src/mongo/db/pipeline/document_source_lookup.h#L100-L112 |
| Comments |
| Comment by Githook User [ 14/Sep/18 ] |
|
Author: {'name': 'Kyle Suarez', 'email': 'kyle.suarez@mongodb.com', 'username': 'ksuarz'}Message: By default, $lookup and $facet do not write persistent data and are |
| Comment by Kyle Suarez [ 24/Jul/18 ] |
|
I'm throwing this back into Needs Triage because the bug here allows views to be saved to the catalog when they contain a $lookup with a $out pipeline, and this affects the Upgrade/Downgrade design for the Improved $out project. |
| Comment by David Storch [ 15/Jun/18 ] |
|
kyle.suarez suggests that due to this bug, the view catalog will permit a $lookup into a sub-pipeline with $out. |
| Comment by Ian Boros [ 14/Jun/18 ] |
|
david.storch I don't think it will impact which node is chosen as a merger: We only know of this bug in two places: $facet and $lookup. The bug only affects stages which have a subpipeline with a stage that has DiskUseRequirement of kWritesPersistentData. The only stage with this DiskUseRequirement is $out, so there are only a few possibilities to consider. $facet explicitly bans $out from running in a subpipeline. $lookup has the HostTypeRequirement of kPrimaryShard, meaning that it will always be run on on a shard anyway (see this). So while this is a "bug" I can't see how it can have any consequences right now, unless stages other than $lookup and $facet can have subpipelines |
| Comment by Sam Mercier [ 13/Jun/18 ] |
|
david.storch I ran into this issue because it impacts the solution to |
| Comment by David Storch [ 13/Jun/18 ] |
|
samuel.mercier kyle.suarez, have you investigated the impact of this bug from a user perspective? Does it just impact whether a mongos node is nominated as a merger? |