[SERVER-37392] In $graphLookup the query planner in mongo/master treats undefined and null as equivalent, while in 4.0 it does not Created: 28/Sep/18  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Vlad Rachev (Inactive) Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: afz, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Query Optimization
Operating System: ALL
Steps To Reproduce:

(function() {
    "use strict";
 
    const coll = db.fuzzer_coll;
    coll.drop();
    assert.writeOK(coll.insert({_id: 293,obj: {}}));
    assert.writeOK(coll.insert({_id: 322,obj: {str: undefined}})); 
    assert.commandWorked(coll.createIndex( {"obj.str": 1}));
 
    const agg = {
        $graphLookup: {
            from: 'fuzzer_coll', startWith: null, connectFromField: 'missing',
            connectToField: 'obj.str', as: 'arrayAs'
        }
    }
 
    const res1 = coll.aggregate([agg]);
    print(tojson(res1.toArray()));
}) ();

Participants:

 Description   

Aggregation output:

4.0: 
[
	{
		"_id" : 293,
		"obj" : {
 
		},
		"arrayAs" : [
			{
				"_id" : 293,
				"obj" : {
 
				}
			}
		]
	},
	{
		"_id" : 322,
		"obj" : {
			"str" : undefined
		},
		"arrayAs" : [
			{
				"_id" : 293,
				"obj" : {
 
				}
			}
		]
	}
]
 
master:
[
	{
		"_id" : 293,
		"obj" : {
 
		},
		"arrayAs" : [
			{
				"_id" : 293,
				"obj" : {
 
				}
			},
			{
				"_id" : 322,
				"obj" : {
					"str" : undefined
				}
			}
		]
	},
	{
		"_id" : 322,
		"obj" : {
			"str" : undefined
		},
		"arrayAs" : [
			{
				"_id" : 293,
				"obj" : {
 
				}
			},
			{
				"_id" : 322,
				"obj" : {
					"str" : undefined
				}
			}
		]
	}
]

The query plan shows different index bounds in 4.0 and master:
4.0:

[js_test:ReproAgg] 2018-09-28T17:50:43.152-0500 d20021| ---------bounds = field #0['obj.str']: [null, null]

master:

[js_test:ReproAgg] 2018-09-28T17:50:43.147-0500 d20020| ---------bounds = field #0['obj.str']: [undefined, undefined], [null, null]

 



 Comments   
Comment by Jacob Evans [ 14/Nov/18 ]

Before https://jira.mongodb.org/browse/SERVER-27646 index bounds produced for null did not include undefined. Because of this, in 4.0, queries filtering with null on documents containing undefined produce inconsistent results depending on whether an index is used. This manifests for all manner of queries.

Comment by Ian Whalen (Inactive) [ 05/Nov/18 ]

jacob to investigate this sprint and come back with a plan (not necessarily a patch yet).

Comment by Charlie Swanson [ 26/Oct/18 ]

The query team would like to figure out what exactly changed here. We think the new behavior might be more correct, but it would require a little more digging to confirm. Scheduling this to be investigated in a future query sprint.

Generated at Thu Feb 08 04:45:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.