[SERVER-45123] Queries should not fail with type error when $type provides the correct filter Created: 13/Dec/19  Updated: 27/Oct/23  Resolved: 18/Dec/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Boris Petrov Assignee: Jacob Evans
Resolution: Works as Designed Votes: 0
Labels: qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-37530 Provide a way to cause a well-defined... Backlog
related to DOCS-13309 MongoDB does not guarantee short circ... Closed
related to SERVER-45231 Provide a way to cause a well-defined... Closed
Operating System: ALL
Steps To Reproduce:

db.objects.insert({ x: 3 })
 
db.objects.find(
    { $and: [
        { x: { $type: 4 } },
        { $or: [
            { $expr: { $in: ['$y', '$x'] } },
            { $expr: { $in: ['$y', '$x'] } }
        ] }
    ] },
)

Sprint: Query 2019-12-30
Participants:

 Description   
Original Summary

$and does not short-circuit

Original Description

I think this is a bug. Check the "Steps To Reproduce". The interesting thing is that if one of the two expressions in the `$or` part is removed, then the query executes fine. Otherwise it leads to `$in requires an array as a second argument, found: objectId`.



 Comments   
Comment by Jacob Evans [ 18/Dec/19 ]

Hi Boris,

Thanks for helping us find an error in our docs! Like most other query languages, MongoDB does not actually support short-circuiting. In general you cannot rely on predicates being executed in a certain order and you cannot rely on predicates being skipped. In this case all predicates are put in a canonical order within the filter. The two arms of the $and are considered separate predicates. This is done to index the query into the plan cache so that other queries with the same predicates will find the cache member. Further optimization may reorder the predicates further, for example to enable the use of an index.

I've created a documentation ticket to correct the claim that we perform short-circuit evaluation: DOCS-13309

In addition I've created a ticket to track a feature request to define an evaluation order for specific predicates since you've demonstrated that this could be useful: SERVER-45231. Please feel free to follow up there.

Thanks again for pointing this out.

Comment by Carl Champain (Inactive) [ 16/Dec/19 ]

alien,

Thank you for the clarification!
I'm passing this ticket along to the Query team for further investigation.

Comment by Boris Petrov [ 16/Dec/19 ]

Hi,

Thanks for the answer.

Well, the actual issue is exactly as I've stated in the title - that I expect short-circuiting when using `$and`. I'm generating queries dynamically (so I cannot add an index because I don't know the queried property) and I want to filter by using `$in` but only when the right-side is an array (as otherwise Mongo blows up with the said error). In the documentation of `$and` it is written that it short-circuits and I would expect my query to be executed in order and the query optimizer to not rearrange it. If that's not the case, the documentation has to be fixed.

In any case I already worked-around this, now it's a matter of just fixing the bug in Mongo.

Thank you,

Boris

Comment by Carl Champain (Inactive) [ 16/Dec/19 ]

Hi alien,

Thanks for the report.

Can you please clarify what the actual issue is for you? In other words, how does it impact you?

After recreating your example, I noticed, in the winningPlan explain output, that the execution of the query selectors starts with $or and then examines $type. That's why you encounter that error message: $type is applied after $or, so the value of $x in $in is 3 and 3 is a double.

A workaround could be to add the following index:

db.objects.createIndex({x:1})

This will guarantee the order of the query selectors and remove the error message. The query will first do an index scan on x and will apply $type to filter out the result, then the query will find the documents (already filtered) that matches y.

Kind regards,
Carl

Generated at Thu Feb 08 05:07:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.