[SERVER-31282] $project of nested projection regression 3.2 -> 3.4 Created: 27/Sep/17 Updated: 04/Oct/17 Resolved: 04/Oct/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | 3.4.9 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Eduardo Gurgel Pinho | Assignee: | Charlie Swanson |
| Resolution: | Duplicate | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux and OS X |
||
| Issue Links: |
|
|||||||||||||||
| Operating System: | ALL | |||||||||||||||
| Steps To Reproduce: | Given this aggregation:
If you run the above aggregation against 3.2.* this is the result:
If you run the above aggregation against 3.4.* this is the result:
|
|||||||||||||||
| Sprint: | Query 2017-10-23 | |||||||||||||||
| Participants: | ||||||||||||||||
| Description |
|
The result of a projection using nested keys is different from 3.2 to 3.4. We had to use `$ifnull` in some cases to avoid this difference of results. I couldn't find any breaking change listed on 3.4 mentioning this change and I couldn't find any issue that was describing exactly this issue. We found this after changing our CI to use 3.4 I'm sorry if I missed some documentation explaining this breaking change. Thanks for your attention! |
| Comments |
| Comment by Eduardo Gurgel Pinho [ 04/Oct/17 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thank you very much for the extremely detailed explanation! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Charlie Swanson [ 04/Oct/17 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi edgurgel, Thanks for your patience. This was indeed an intentional change, and we do believe the new semantics are more consistent. In more detail: The First Compatibility Change
This change makes the $project stage more consistent with how we treat objects elsewhere in the aggregation language. Consider the following example, which works on both 3.2.0 and 3.4.0:
This is also consistent with how the (new in 3.4) $replaceRoot stage handle object literals:
The Second Compatibility Change
I actually mis-remembered here. This wasn't a change in the $project behavior, it's a change in $group's behavior. It's very surprising, but due to an oddity in the old implementation of $project, the following happens:
This was previously enforced in projection, and elsewhere when object literals were used:
The Third Compatibility Change
If I remember correctly, this change was made because:
The Last Compatibility Change:
After introducing exclusion projections, this is ambiguous as to whether it means "include everything within 'a' and 'b.c'" or "exclude everything within 'a' and 'b.c'". Similarly,
is ambiguous, is it 'include everything' or 'exclude everything'. It used to be 'include everything', which is surprising. We already documented the last compatibility change, but I've filed I apologize for the trouble this caused you! Best, | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eduardo Gurgel Pinho [ 27/Sep/17 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thank you for the very detailed explanation! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Charlie Swanson [ 27/Sep/17 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi edgurgel, This was a change introduced during the re-write of the $project stage during 3.4. In order to allow exclusion projections we essentially re-implemented that stage. I have a note written down detailing this change, but it looks like we forgot to put it in the release notes. In any case, it'll take some amount of work to decide what is 'correct' here, and make sure it's consistent with other places in the server. I'm putting this ticket into the 'Needs Triage' queue for the query team to look at and prioritize. The note I have about the re-write reads:
I think the next steps here are:
|