[SERVER-58193] Add test coverage for $in matching against +/-NaN Created: 01/Jul/21  Updated: 29/Oct/23  Resolved: 24/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.0-rc6
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Mohammad Dashti (Inactive) Assignee: Mihai Andrei
Resolution: Fixed Votes: 0
Labels: sbe-diff, sbe-rollout
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-58268 [SBE] $in match expression fails to m... Closed
Backwards Compatibility: Fully Compatible
Sprint: QE 2021-08-23, QE 2021-09-06
Participants:
Linked BF Score: 28

 Description   

Here are the steps to reproduce the issue:

 
> db.fuzzer_coll.drop();
true
 
> const documentList = [
...     {_id: 0, "str": "Soap", "num": NumberInt(26549), "array": [{_id: 3, "str": NaN, "num": NumberLong("43418") }], }, // 0
...     {_id: 729, "str": "pricing structure", "num": NumberLong("360"), "obj": {_id: 733, "num": NaN, "obj": {}, }, }, // 129
... ];
 
> db.fuzzer_coll.insert(documentList);
BulkWriteResult({
	"writeErrors" : [ ],
	"writeConcernErrors" : [ ],
	"nInserted" : 2,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
})
 
> // using the classic engine
> db.adminCommand({setParameter: 1, internalQueryEnableSlotBasedExecutionEngine: false})
{ "was" : false, "ok" : 1 }
> db.fuzzer_coll.aggregate([{$sort: {_id: 1}}, {$graphLookup: {from: "fuzzer_coll", startWith: ["$obj.obj.str", NumberDecimal("-NaN")], connectFromField: "obj.obj.obj.obj", connectToField: "obj.num", as: "array", maxDepth: 1, depthField: "obj.obj.obj.num"}}])
{ "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 3, "str" : NaN, "num" : NumberLong(43418) } ], "obj" : { "obj" : { "obj" : { "num" : NumberLong(0) } } } }, { "_id" : 729, "str" : "pricing structure", "num" : NumberLong(360), "obj" : { "_id" : 733, "num" : NaN, "obj" : { "obj" : { "num" : NumberLong(0) } } } } ] }
{ "_id" : 729, "str" : "pricing structure", "num" : NumberLong(360), "obj" : { "_id" : 733, "num" : NaN, "obj" : {  } }, "array" : [ { "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 3, "str" : NaN, "num" : NumberLong(43418) } ], "obj" : { "obj" : { "obj" : { "num" : NumberLong(0) } } } }, { "_id" : 729, "str" : "pricing structure", "num" : NumberLong(360), "obj" : { "_id" : 733, "num" : NaN, "obj" : { "obj" : { "num" : NumberLong(0) } } } } ] }
 
> // using SBE
> db.adminCommand({setParameter: 1, internalQueryEnableSlotBasedExecutionEngine: true})
{ "was" : false, "ok" : 1 }
> db.fuzzer_coll.aggregate([{$sort: {_id: 1}}, {$limit: 20}, {$graphLookup: {from: "fuzzer_coll", startWith: ["$obj.obj.str", NumberDecimal("-NaN")], connectFromField: "obj.obj.obj.obj", connectToField: "obj.num", as: "array", maxDepth: 1, depthField: "obj.obj.obj.num"}}])
{ "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 3, "str" : NaN, "num" : NumberLong(43418) } ], "obj" : { "obj" : { "obj" : { "num" : NumberLong(0) } } } } ] }
{ "_id" : 729, "str" : "pricing structure", "num" : NumberLong(360), "obj" : { "_id" : 733, "num" : NaN, "obj" : {  } }, "array" : [ { "_id" : 0, "str" : "Soap", "num" : 26549, "array" : [ { "_id" : 3, "str" : NaN, "num" : NumberLong(43418) } ], "obj" : { "obj" : { "obj" : { "num" : NumberLong(0) } } } } ] }

As you see, the output of SBE has fewer elements compared to the classic engine, which hints into the direction that the special values (e.g., NaN and non-existing fields) accessed from SBE by $graphLookup are treated differently compared to the classic engine.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 24/Aug/21 ]

Author:

{'name': 'Mihai Andrei', 'email': 'mihai.andrei@10gen.com', 'username': 'mtandrei'}

Message: SERVER-58193 Add test coverage for $in matching against +/-NaN
Branch: master
https://github.com/mongodb/mongo/commit/6e12fe76eb373483338c4e94293fe94e269f903c

Comment by Steve La (Inactive) [ 19/Aug/21 ]

mihai.andrei. nice work!  do we have any automated deterministic test ( not fuzzer ) that covers this case?  If not, can we convert your repro into such a test? 

 

Comment by Mohammad Dashti (Inactive) [ 18/Aug/21 ]

mihai.andrei this is a typo. The command should be the same for both engines. As a final check, can you see if with and without $limit: 20, the issue does not appear?

Comment by Steve La (Inactive) [ 17/Aug/21 ]

thx mohammad.dashti i pasted the below comment in the other similar ticket.  https://jira.mongodb.org/browse/SERVER-58155

mihai.andrei.  i see .   this goes towards my suspicion that this is not specific to lookup or graphLookup ( i commented on another similar ticket for that  SERVER-58193).    it would be idea if we can just come up with a scenario that exposes these difference for a simple match,   that will help in simplifying the discussion and hopefully the solution as well 

mihai.andrei mohammad.dashti could one of you look to try both out for simple match?   this will allow use to potentially dedup these 2 problems to 1 that would help us to better understand what work we have left

kyle.suarez FYI

 

Comment by Mohammad Dashti (Inactive) [ 17/Aug/21 ]

steve.la Even though $graphLookup itself is not supported by SBE, the rest of the query execution pipeline uses SBE. In the above example, the part that's handled by SBE is different from its equivalent in the classic engine. It's possible to recreate this issue without $graphLookup, but used the above example, which stems from BF-21717.

Generated at Thu Feb 08 05:43:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.