[SERVER-23229] Projection incorrectly returns null value instead of missing field to user if projection is covered Created: 18/Mar/16 Updated: 26/Jan/24 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Benjamin Murphy | Assignee: | Backlog - Query Optimization |
| Resolution: | Unresolved | Votes: | 3 |
| Labels: | open_todo_in_code, query-44-grooming, storch | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Query Optimization
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | QO 2022-07-11 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 135 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
If a document is missing a certain field, then covered projections on that field will incorrectly report the field as having value null. Reproduce as follows:
Fixing this issue in such a way that is not backwards-breaking for existing indexes will be difficult. This issue is caused by the fact that the index key generation process generates identical keys for missing values and null values. This also causes unintuitive behavior for features that use covered projections, such as $group (see below for details). Original description:
|
| Comments |
| Comment by Steve Tarzia [ 11/Jul/22 ] |
|
I confirmed that the commit on Jul 6, 2022 for |
| Comment by Steve Tarzia [ 06/Jul/22 ] |
|
Although the test case provided in the ticket description no longer fails, I am going to re-open this because we don't really understand why https://jira.mongodb.org/browse/SERVER-67416 would have fixed the root cause. I'd like to try to reproduce this with a different JS test. |
| Comment by Githook User [ 06/Jul/22 ] |
|
Author: {'name': 'Steve Tarzia', 'email': 'steve.tarzia@mongodb.com', 'username': 'starzia'}Message: |
| Comment by Githook User [ 06/Jul/22 ] |
|
Author: {'name': 'Steve Tarzia', 'email': 'steve.tarzia@mongodb.com', 'username': 'starzia'}Message: |
| Comment by Steve Tarzia [ 01/Jul/22 ] |
|
This seems to be fixed by my patch for |
| Comment by Kajetan Abt [ 11/Mar/20 ] |
|
Yes we suffer from the same issue. Adding an index while using a projection will result in an extra "nullish" value in the result set. Queries should not change their results depending on index availability. This broke correct code when we added an index for performance improvements. |
| Comment by David Storch [ 13/Aug/19 ] |
|
Flagging for re-triage, since this is currently complicating our query fuzz testing efforts. |