[SERVER-84061] Investigate NaN behavior in SBE Created: 11/Dec/23  Updated: 09/Jan/24  Resolved: 09/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Parker Felix Assignee: Parker Felix
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Operating System: ALL
Sprint: QE 2024-01-08, QE 2024-01-22
Participants:

 Description   

When comparing two doubles, valueCompare will call compareDoubles in compare_numbers.h so two NaN's will compare equal to each other. https://github.com/mongodb/mongo/blob/e67290f658e1a8529935f3c842fe9c25ccf420bf/src/mongo/base/compare_numbers.h#L58

When using genericCompare<std::equal_to<>>, no special case exists for NaN when both values are doubles, so it will fall back to using operator== for doubles. This means that when using genericCompare, NaN != NaN.
https://github.com/mongodb/mongo/blob/e67290f658e1a8529935f3c842fe9c25ccf420bf/src/mongo/db/exec/sbe/vm/vm.h#L99



 Comments   
Comment by Parker Felix [ 09/Jan/24 ]

When doing an equality comparison to a constant, SBE will use a special Instruction::isNaN instead of the regular Instruction::eq.

For example aggregate([\{$match: {a: {$eq: 1.0}}}]) }}will use {{Instruction::eq

On the other hand aggregate([\{$match: {a: {$eq: isNaN}}}]) or aggregate([\{$match: {a: {$eq: 0 / 0}}}]) will use Instruction::isNaN.

The logic for this distinction can be found in generateComparisonExpr

 

When the value being compared to isn't known at the time of plan generation, Instruction::cmp3w will be used instead. This instruction calls ByteCode::compare3way which calls valueCompare, which in turn calls compareDoubles which captures the MQL semantics that NaN should compare equal to NaN.

For example {{aggregate([{"$project": {"abEq":

{"$eq":["$a", "$b"]}

}}])}} will use Instruction::cmp3w and the result of the three way comparison will then be processed by Instruction::eq that checks if the result of Instruction::cmp3w was 0.

Generated at Thu Feb 08 06:53:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.