[SERVER-42568] Covered query for $replaceRoot involving missing fields returns different results than collection scan Created: 31/Jul/19  Updated: 26/Jan/24  Resolved: 02/Aug/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sandra Gould (Inactive) Assignee: Backlog - Query Team (Inactive)
Resolution: Duplicate Votes: 0
Labels: afz, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-37101 Add optimization mode aggregation (pi... Closed
Duplicate
duplicates SERVER-23229 Projection incorrectly returns null v... Backlog
Related
Assigned Teams:
Query
Operating System: ALL
Steps To Reproduce:

python buildscripts/resmoke.py --storageEngine wiredTiger --suites aggregation repro_match_replace_root.js

repro_match_replace_root.js

assert.commandWorked(db.my_coll.insert({}));
 
const res1 = db.my_coll
	.aggregate([
		{
			$match: {
				a: { 
					$not: { 
						$gte: 5
					}
				}
			}
		},
		{
			$replaceRoot: {
				newRoot:{
					v: '$a'
				}
			}
		},
]).toArray();
 
assert.commandWorked(db.my_coll.createIndex({a:1}));
 
const res2 = db.my_coll
	.aggregate([
		{
			$match: {
				a: { 
					$not: { 
						$gte: 5
					}
				}
			}
		},
		{
			$replaceRoot: {
				newRoot:{
					v: '$a'
				}
			}
		},
]).toArray();
 
assert.eq(res1, res2);

Participants:
Linked BF Score: 9

 Description   

When no index is present and $match is called before a $replaceRoot that uses the same index, it will do a collection scan and $replaceRoot will return an empty document. However, when the index is present but the field doesn't exist in the document, the query system ends up performing a covered query. The index has a null entry for the document because the field doesn't exist in the document. This causes $replaceRoot to return a non-empty document with the missing field associated with a literal null value.

[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 2019-07-31T18:09:25.363-0400 E  QUERY    [js] uncaught exception: Error: [[ { } ]] != [[ { "v" : null } ]] are not equal :
[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 doassert@src/mongo/shell/assert.js:20:14
[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 assert.eq@src/mongo/shell/assert.js:180:9
[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 @repro_match_replace_root.js:45:1
[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 2019-07-31T18:09:25.363-0400 F  -        [main] failed to load: repro_match_replace_root.js
[js_test:repro_match_replace_root] 2019-07-31T18:09:25.364-0400 2019-07-31T18:09:25.363-0400 E  -        [main] exiting with code -3



 Comments   
Comment by David Storch [ 02/Aug/19 ]

Closing as a duplicate of SERVER-23229.

Comment by Vlad Rachev (Inactive) [ 01/Aug/19 ]

This isn't only a problem with $replaceRoot. Here is another query that where we incorrectly cover it using the implicitly built `null` index rather than returning a missing document:

const agg = [
    {$match: {
        "a": {
            $not: {
                $gte: 1
            }
        }
    }},
    {
        $project: {
            "_id": 0,
            "a": 1
        }
    }
];
 
db.foo.insert({})
db.foo.aggregate(agg)
> { } // without an index to cover the query we do a collscan and return an empty document
 
db.foo.createIndex({a: 1})
db.foo.aggregate(agg)
> { "a" : null } // uses the implicitly created null index to cover the query 

Generated at Thu Feb 08 05:00:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.