[SERVER-26598] conditionally push $match before $addFields in aggregation pipeline Created: 12/Oct/16  Updated: 10/May/18  Resolved: 19/Jan/17

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 3.5.2

Type: Improvement Priority: Major - P3
Reporter: Asya Kamsky Assignee: Kyle Suarez
Resolution: Done Votes: 0
Labels: bkp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Documented
is documented by DOCS-9798 Document that $match can be condition... Closed
Related
related to SERVER-19153 Conditionally push $match before $pro... Closed
is related to SERVER-25510 Differentiate between Inclusion and C... Backlog
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4
Sprint: Query 2017-01-23
Participants:
Case:

 Description   

$addFields is a new stage for 3.4 for aggregation and the same optimization that was made for some $project in SERVER-19153 should be made for $addFields.

(i.e. if $match is not on fields that are created/involved in $addFields then it can go before $addFields).



 Comments   
Comment by Kyle Suarez [ 19/Jan/17 ]

I'm marking documentation changes as "Needed", in case we want to list this under our Aggregation Pipeline Optimizations page.

Comment by Githook User [ 19/Jan/17 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'kyle.suarez@mongodb.com'}

Message: SERVER-26598 only report computed paths as modified in $addFields

This allows a $match to be pushed before $addFields if there are no conflicts between the paths
involved in each stage.
Branch: master
https://github.com/mongodb/mongo/commit/a96a5f24604be4a9d23ec935faeba591c91f137d

Comment by Asya Kamsky [ 13/Oct/16 ]

Moved request to reorder $sort into related SERVER-26442

Comment by Charlie Swanson [ 13/Oct/16 ]

This was thought to be challenging and/or confusing in the initial implementation of SERVER-19153, but there might be a way to do this that isn't terrible. In my mind, the ideal way to do this would involve SERVER-25510, splitting the functionality of 'including' and 'computing' into two separate classes. But that's a lot more work. If we still want this and don't want to invest in that refactor, we could add another method to InclusionNode that would be used only for $addFields. The existing InclusionNode::addPreservedPaths() only examines the pure inclusions, and ignores the computed fields. Thus, it can only be used by the $project stage to determine which fields are unmodified. We could add a new method, maybe something like InclusionNode::getComputedPaths() which would report all paths explicitly modified by the inclusion projection. This could be called by $addFields, allowing it to take part in the $match swapping optimization.

The downside to this is it leaves InclusionNode even more muddled, with addPreservedPaths() which should only be called by $project (though is not incorrect to call from elsewhere), and getComputedPaths() which should only be called by $addFields, but is actually incorrect (or at least dangerously misleading) to call in the event that the InclusionNode tree represents both inclusions and computed paths (e.g. within a $project), since it will ignore all included paths.

Generated at Thu Feb 08 04:12:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.