[SERVER-66379] $or to $in conversion flawed Created: 11/May/22  Updated: 29/Oct/23  Resolved: 31/May/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.10, 6.0.0-rc9, 6.1.0-rc0

Type: Bug Priority: Critical - P2
Reporter: Daniel Hegener Assignee: Timour Katchaounov
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-34012 Planner's logic for taking union of i... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0, v5.0
Sprint: QO 2022-05-30, QO 2022-06-13
Participants:
Linked BF Score: 17

 Description   

We have come across a problem that was introduced with this change: SERVER-34012. The exact line causing this is https://github.com/mongodb/mongo/blame/master/src/mongo/db/matcher/expression_tree.cpp#L250.

This problem can only be seen on MongoDB v5, earlier versions work properly.

Steps to reproduce:

Create "some" collection and then run:

db.getCollection("some").aggregate({$match: { "$or" : [ { "x" : "a" }, { "x" : "b" }, { "x" : /c/ }] } })

Expected output: No failure.
Output:

uncaught exception: Error: command failed: {
        "ok" : 0,
        "errmsg" : "Failed to optimize pipeline :: caused by :: Failed to optimize expression :: caused by :: Incorrect number of non-equivalent expressions",
        "code" : 3401204,
        "codeName" : "Location3401204"
} with original command request: {
        "aggregate" : "FileNodeVersions",
        "pipeline" : [
                {
                        "$match" : {
                                "$or" : [
                                        {
                                                "x" : "a"
                                        },
                                        {
                                                "x" : "b"
                                        },
                                        {
                                                "x" : /c/
                                        }
                                ]
                        }
                }
        ],
        "cursor" : {
 
        },
        "lsid" : {
                "id" : UUID("af1c7696-912f-4ad8-a53b-8eb9db5c0162")
        }



 Comments   
Comment by Githook User [ 02/Jun/22 ]

Author:

{'name': 'Timour Katchaounov', 'email': 'timour.katchaounov@mongodb.com', 'username': 'timourk'}

Message: SERVER-66379 $or to $in conversion flawed

(cherry picked from commit 951197b3834d509d7903efa4a66c0e9f79235262)
(cherry picked from commit b499c915d1c21f063548c96d5d547dcfcda55754)
Branch: v5.0
https://github.com/mongodb/mongo/commit/ea79d55381c353e84cd673c4b21deb0b0ea4826e

Comment by Githook User [ 02/Jun/22 ]

Author:

{'name': 'Timour Katchaounov', 'email': 'timour.katchaounov@mongodb.com', 'username': 'timourk'}

Message: SERVER-66379 $or to $in conversion flawed

(cherry picked from commit 951197b3834d509d7903efa4a66c0e9f79235262)
Branch: v6.0
https://github.com/mongodb/mongo/commit/b499c915d1c21f063548c96d5d547dcfcda55754

Comment by Githook User [ 31/May/22 ]

Author:

{'name': 'Timour Katchaounov', 'email': 'timour.katchaounov@mongodb.com', 'username': 'timourk'}

Message: SERVER-66379 $or to $in conversion flawed
Branch: master
https://github.com/mongodb/mongo/commit/951197b3834d509d7903efa4a66c0e9f79235262

Comment by Timour Katchaounov [ 26/May/22 ]

It turns out the collation is crucial. This is a reproducible test case:

db.createCollection("some");
db.getCollection("some").find({ "$or" : [ { "x" : "a" }, { "x" : "b" }, { "x" : /c/ }] }).sort({ name: 1 }).collation({ locale: 'en_US'})

Comment by Timour Katchaounov [ 26/May/22 ]

Update: I read the original thread on Stackoverflow, and tested the complete example in the thread. Ineed, the original much bigger query results in the reported assertion. So there is a bug, but the test case here seems to have been oversimplified. I will simplify it a bit before continuing work on the bug.

Comment by Timour Katchaounov [ 26/May/22 ]

Unfortunately I cannot reproduce the problem on neither master, nor v5.0. This is what I tried:

test> db.createCollection("some");
{ ok: 1 }
test> db.getCollection("some").aggregate({$match: { "$or" : [ { "x" : "a" }, { "x" : "b" }, { "x" : /c/ }] } })
 
test> db.some.insertOne({x:"a"})
{
  acknowledged: true,
  insertedId: ObjectId("628f440bda50c4aecee984f3")
}
test> db.getCollection("some").aggregate({$match: { "$or" : [ { "x" : "a" }, { "x" : "b" }, { "x" : /c/ }] } })
[ { _id: ObjectId("628f440bda50c4aecee984f3"), x: 'a' } ]

daniel.hegener@gmx.net could you please provide exact instructions how to reproduce - exact version (or commit), how it was built, complete script.

Comment by Daniel Hegener [ 11/May/22 ]

https://stackoverflow.com/questions/69098410/mongodb-failed-with-error-failed-to-optimize-expression 

Generated at Thu Feb 08 06:05:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.