[SERVER-58680] $ifNull behaves differently in latest Created: 19/Jul/21  Updated: 29/Oct/23  Resolved: 22/Sep/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Robert Stam Assignee: Ethan Zhang (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by CSHARP-3726 Fix GroupJoin_syntax_with_select_many... Closed
Problem/Incident
is caused by SERVER-55295 Short circuit $ifNull expression if t... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Behavior in 5.0.0 and earlier:

MongoDB Enterprise > db.version()
5.0.0
MongoDB Enterprise > db.test.find()
{ "_id" : 1 }
{ "_id" : 2, "X" : null }
{ "_id" : 3, "X" : 3 }
MongoDB Enterprise > db.test.aggregate([{ $project : { Z : { $ifNull : ["$X", null] } } }])
{ "_id" : 1, "Z" : null }
{ "_id" : 2, "Z" : null }
{ "_id" : 3, "Z" : 3 }
MongoDB Enterprise >

Unexpected behavior in latest:

MongoDB Enterprise > db.version()
5.0.0-alpha0-1698-gf4169a7
MongoDB Enterprise > db.test.find()
{ "_id" : 1 }
{ "_id" : 2, "X" : null }
{ "_id" : 3, "X" : 3 }
MongoDB Enterprise > db.test.aggregate([{ $project : { Z : { $ifNull : ["$X", null] } } }])
{ "_id" : 1 } // NOTE: the Z field is missing
{ "_id" : 2, "Z" : null }
{ "_id" : 3, "Z" : 3 }
MongoDB Enterprise >

 

Sprint: QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04
Participants:
Linked BF Score: 153

 Description   

$ifNull behaves differently in latest when the replacement value is null. In earlier versions it returned null. In latest it omits the field entirely.

While it may sound silly to use a replacement value of null, the LINQ provider in the C# driver uses this construct to replace a missing field with an explicit null-valued field.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Ethan Zhang (Inactive) [ 23/Sep/21 ]

Hi dmitry.lukyanov, this is what I got from the latest master:

> db.ifnull.find()
{ "_id" : 1 }
{ "_id" : 2, "X" : null }
{ "_id" : 3, "X" : 3 }
{ "_id" : 4, "X" : undefined }
> db.ifnull.aggregate([{ $project : { Z : { $ifNull : ["$X", null] } } }])
{ "_id" : 1, "Z" : null }
{ "_id" : 2, "Z" : null }
{ "_id" : 3, "Z" : 3 }
{ "_id" : 4, "Z" : null }

Can you let me know what case do you see still failing on the latest master?

Comment by Dmitry Lukyanov (Inactive) [ 23/Sep/21 ]

It looks like it's fixed on 5.0 and still failing on 5.1

Comment by Githook User [ 22/Sep/21 ]

Author:

{'name': 'Ethan Zhang', 'email': 'ethan.zhang@mongodb.com', 'username': 'yzhang1991'}

Message: SERVER-58680 Let $ifNull not remove the last null constant
Branch: master
https://github.com/mongodb/mongo/commit/16680b2846dfd1c5c4d58de93e5255b3705ad096

Comment by Ethan Zhang (Inactive) [ 04/Sep/21 ]

Marking this 5.1 required because this is an unexpected behavioral change in mognod.

Comment by Ethan Zhang (Inactive) [ 04/Sep/21 ]

On 5.0, the winning plan is:

"winningPlan" : {
    "stage" : "PROJECTION_DEFAULT",
    "transformBy" : {
        "_id" : true,
        "Z" : {
            "$ifNull" : [
                "$X",
                {
                    "$const" : null
                }
            ]
        }
    },
    "inputStage" : {
        "stage" : "COLLSCAN",
        "direction" : "forward"
    }
}

On latest master, the winning plan is:

"winningPlan" : {
    "stage" : "PROJECTION_DEFAULT",
    "transformBy" : {
            "_id" : true,
            "Z" : "$X"
    },
    "inputStage" : {
            "stage" : "COLLSCAN",
            "direction" : "forward"
    }
},

For some reason, the $ifNull expression is dropped on the latest master. Still investigating.

Comment by James Kovacs [ 20/Jul/21 ]

Note that this behaviour only manifests if the replacement value is null. If the replacement value is non-null, latest works the same as previous versions.

(mongod-5.0.0-alpha0-1698-gf4169a7) test> db.test.find()
{
  "_id": 1
}
{
  "_id": 2,
  "X": null
}
{
  "_id": 3,
  "X": 3
}
(mongod-5.0.0-alpha0-1698-gf4169a7) test> db.test.aggregate([{ $project : { Z : { $ifNull : ["$X", null] } } }])
{ "_id": 1 }
{ "_id": 2, "Z": null }
{ "_id": 3, "Z": 3 }
(mongod-5.0.0-alpha0-1698-gf4169a7) test> db.test.aggregate([{ $project : { Z : { $ifNull : ["$X", 42] } } }])
{ "_id": 1, "Z": 42 }
{ "_id": 2, "Z": 42 }
{ "_id": 3, "Z": 3 }

Generated at Thu Feb 08 05:45:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.