[SERVER-57300] {$exists:false} on array elements returns incorrect results on MongoDB 5.0 Created: 28/May/21  Updated: 29/Oct/23  Resolved: 17/Jun/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.0-rc0
Fix Version/s: 5.0.0-rc3

Type: Bug Priority: Major - P3
Reporter: James Kovacs Assignee: Andrii Dobroshynski (Inactive)
Resolution: Fixed Votes: 0
Labels: post-rc0, query
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by CSHARP-3704 Temporarily disable tests failing due... Closed
Problem/Incident
causes CSHARP-3687 Fixing legacy LINQ tests on latest Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Steps To Reproduce:

Insert the following documents into a collection:

db.arrays.insert({a:1})        // integer
db.arrays.insert({a:[1]})     // array with 1 element
db.arrays.insert({a:[1,2]})  // array with 2 elements

With MongoDB 4.4.5:
db.arrays.find({"a.0": {$exists: false)}} returns 1 document
db.arrays.find({"a.1": {$exists: false)}} returns 2 documents
db.arrays.find({"a.2": {$exists: false)}} returns 3 documents

With MongoDB 5.0.0-alpha0-584-g325ff69:
db.arrays.find({"a.0": {$exists: false)}} returns 3 documents
db.arrays.find({"a.1": {$exists: false)}} returns 3 documents
db.arrays.find({"a.2": {$exists: false)}} returns 3 documents

Sprint: Query Execution 2021-06-14, Query Execution 2021-06-28
Participants:

 Description   

MongoDB 5.0 returns incorrect results for queries of the form:

{ "a.0": { $exists: false } }

In MongoDB 4.4 and earlier, this would return documents where the field didn't contain an array. In MongoDB 5.0, all documents are returned even if the field contains an array.

Why does this matter? The .NET/C# driver uses this technique in its legacy LINQ implementation to determine whether a field contains a single value or an array - in particular for discriminators. Let's say we have a class D that derives from C and B. When we insert a document of type D, we will write the discriminator as {{ _t: [B,C,D] }}. If we want to find all B, but not any derived types such as C and D, we will generate the following query:

{ "_t.0" : { "$exists" : false }, _t : "B" }

When querying MongoDB 4.4 and earlier, we only get documents with _t: "B" back. When querying MongoDB 5.0, we get documents of type B, C, and D back.

While we can potentially fix this problem in the C# driver by using another technique, it will be a backwards breaking change for existing C# apps that may prevent them from upgrading to MongoDB 5.0.



 Comments   
Comment by Githook User [ 12/Aug/21 ]

Author:

{'name': 'Dmitry Lukyanov', 'email': 'dmitry.lukyanov@mongodb.com', 'username': 'DmitryLukyanov'}

Message: CSHARP-3704: Temporarily disable tests failing due to SERVER-57300. (#601)
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/179d34ff8cf1f410396e4ce7753f6bc0ca311f09

Comment by Githook User [ 17/Jun/21 ]

Author:

{'name': 'Andrii Dobroshynski', 'email': 'andrii.dobroshynski@mongodb.com', 'username': 'dobroshynski'}

Message: SERVER-57300 Fix logic for detecting numeric path component to avoid executing with SBE

(cherry picked from commit a6bd767d38ca772b626afd7f90301a9701dda85f)
Branch: v5.0
https://github.com/mongodb/mongo/commit/ddca135c85508646315700bfb112d846756ec1a0

Comment by Githook User [ 17/Jun/21 ]

Author:

{'name': 'Andrii Dobroshynski', 'email': 'andrii.dobroshynski@mongodb.com', 'username': 'dobroshynski'}

Message: SERVER-57300 Fix logic for detecting numeric path component to avoid executing with SBE
Branch: master
https://github.com/mongodb/mongo/commit/a6bd767d38ca772b626afd7f90301a9701dda85f

Comment by Githook User [ 04/Jun/21 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3704: Temporarily disable tests failing due to SERVER-57300.
Branch: v2.12.x
https://github.com/mongodb/mongo-csharp-driver/commit/a2f3800a6b951ed4e7706799a638ff68e20ab8ec

Comment by Githook User [ 04/Jun/21 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3704: Temporarily disable tests failing due to SERVER-57300
Branch: v2.12.x
https://github.com/mongodb/mongo-csharp-driver/commit/2812be1036107d33fa20f8a7845e6832c7a8e0e6

Comment by Githook User [ 04/Jun/21 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3704: Temporarily disable tests failing due to SERVER-57300.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/61fa13824c0cc463b8180f76059ef96189d33a4f

Comment by Githook User [ 03/Jun/21 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3704: Temporarily disable tests failing due to SERVER-57300
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/67db5a06835d668380023f7c2265cbe39eadc940

Comment by Kyle Suarez [ 01/Jun/21 ]

This looks almost certainly like new behavior that is happening due to SBE being turned on by default - sending this to the QE triage queue.

Comment by James Kovacs [ 28/May/21 ]

Digging in a bit more, I only get incorrect results from 5.0 when using {{ "a.0": {$exists:false}}} or {{ "a.0": {$not: expr}}}. If I use their positive counterparts ({{ "a.0": {$exists:true}}} or {{ "a.0": expr}}), I get the expected results.

Comment by James Kovacs [ 28/May/21 ]

Found another example, probably the same root cause:

db.people.insert({a:["Tom","Dick","Harry"]})
db.people.find({"a.0": {$not: /^T/}})

4.4 returns no results (as expected). 5.0 returns the inserted document.

Generated at Thu Feb 08 05:41:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.