[SERVER-25120] aggregation requests generated field name from query Created: 18/Jul/16 Updated: 04/Apr/23 Resolved: 27/Mar/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | 3.3.9 |
| Fix Version/s: | 7.0.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Asya Kamsky | Assignee: | Alyssa Clark |
| Resolution: | Done | Votes: | 0 |
| Labels: | asya, performance | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Query Optimization
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Query 2017-03-27, Query 2017-04-17, QO 2023-03-20 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 151 | ||||||||||||||||
| Description |
|
Aggregation pipeline sends down to query fields to return which are generated later in the pipeline (as result of "as" in $lookup). Reproduce:
Note "tree.country" which is coming from the lookup and not original document. It appears that if project or group needs a field after lookup, there is no filtering out of fields that start with string passed as "as:" to $lookup and it ends up being pushed down into the query. |
| Comments |
| Comment by Githook User [ 27/Mar/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Alyssa Wagenmaker', 'email': 'alyssa.wagenmaker@mongodb.com', 'username': 'awagenmaker'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 04/Jan/17 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Turns out there is a better workaround for this for $lookup, and that's to use the existing localField that's already being fetched as the value for "as" (this only works because localField will have the same value as foreignField, so you don't lose any information).
Credit for this workaround to this comment. This workaround may not work the same way in graphLookup (depending on whether you need original/starting connectFromField later in the pipeline). | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 11/Nov/16 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
There is a work-around to prevent this from happening and that is to add an explicit $project stage before the lookup:
Explicit $project is honored over dependency analysis. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 28/Aug/16 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
We do correctly filter out new fields added in $project stage (that are $literal or $const) though.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Charlie Swanson [ 18/Jul/16 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This looks like a legitimate bug. Our dependency tracking system has no special logic to handle cases where new fields get added to documents, so it's not surprising that it thinks injected fields need to come from the input collection. I'm moving this to "Needs Triage" so we can look at it as a team. |