[SERVER-81378] Change classic behavior: equality to null should not match undefined Created: 22/Sep/23  Updated: 02/Feb/24

Status: In Code Review
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Hana Pearlman Assignee: Hana Pearlman
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-82374 CanonicalQuery::canonicalize() doesn'... Closed
is depended on by SERVER-67550 [CQF] Equality to null does not match... Open
Assigned Teams:
Query Optimization
Sprint: QO 2024-02-05, QO 2024-02-19
Participants:

 Description   

The undefined type has been deprecated for a long time. In v8.0, we will change classic and stage builders behavior so that comparison to null does not match undefined.

This ticket will include the necessary updates to the classic comparison expressions, classic index bounds building, and stage builders code. We should consider doing a performance investigation to see if there are improvements for comparison to null queries, which should be simpler after this change. Also, we will likely need to make updates to the query fuzzers. We should consider if it's worthwhile to fuzz with data containing undefined values.

In preparation for this, we will provide downstream notifications, including recommendations for ways to achieve the original behavior, when the work is complete. 



 Comments   
Comment by Hana Pearlman [ 18/Jan/24 ]

Had a conversation with some stakeholders about this ticket and people seemed lukewarm about the feature flag. It ended up seeming way easier to do the ticket without one, so I skipped it. Updated the ticket description and made a note about this in the PR. For anyone getting this notification, let me know if you feel strongly.

Comment by David Storch [ 12/Oct/23 ]

including recommendations for ways to achieve the original behavior, when the work is complete

In addition, I think we should recommend a way to migrate undefined values in existing data sets to null values. Here's one example:

db.c.update({a: {$type: "undefined"}}, {$set: {a: null}}, {multi: true})

This will work for updating all of the undefined values in top-level field a to null for collection c. Things would get more complicated if there are undefined values in dynamically named fields, undefined values nested inside arrays, or undefined values across many collections/databases. We do not yet intend to offer a complete solution which customers can use to automatically find all undefined values and rewrite them as null, but the kind of basic example above definitely needs to work in the case of customers with old datasets who wish to rid themselves of the undefined data type.

Generated at Thu Feb 08 06:46:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.