[SERVER-79488] [CQF] Different behavior with empty-array equality (discovered by fuzzer) Created: 28/Jul/23  Updated: 17/Jan/24

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: David Percy Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-79205 [CQF] $not $eq array, on a missing fi... Open
related to SERVER-83456 Skip Filter->Sargable rewrite for col... Closed
Assigned Teams:
Query Optimization
Operating System: ALL
Sprint: QO 2023-12-11, QO 2023-12-25, QO 2024-01-08
Participants:

 Description   

The fuzzer found another bug. Here's a repro:

const pipeline = [
    // Variations on this predicate seem to not matter much.
    {$match: {"num": 3}},
 
    {
        $match: {
            $or: [
                {"array1": {$eq: []}},
 
                // Although this predicate is always false, when I comment it out
                // then the test does not reproduce the bug.
                {"array2": {$eq: [false]}},
            ]
        }
    },
];
 
const docs = [
    {
        _id: 1,
        "num": 3,
 
        // This has to be empty array:
        // Changing the query to {$eq: [5]} and doc to 'array1: [5]'
        // does not repro.
        "array1": [],
 
        // This field's value and presence doesn't matter.
        // "array2": [123]
    },
];
 
const coll = db.cqf_fuzzer_bug;
 
coll.drop();
assert.commandWorked(coll.insert(docs));
 
// The first stage matches.
assert.eq(1, coll.aggregate([pipeline[0]]).itcount());
// The second stage matches.
assert.eq(1, coll.aggregate([pipeline[1]]).itcount());
 
// Their composition should also match.
assert.eq(1, coll.aggregate(pipeline).itcount());

This passes on the default configuration, but the last assertion fails on Bonsai. Each $match individually matches the document, but when put together in one query it doesn't match. So it must be a rewrite combining the predicates incorrectly.

These {$eq: <array>} predicates are interesting because they check the whole array (in addition to checking in element): we generate a disjunction where one branch doesn't use traverse.



 Comments   
Comment by Hana Pearlman [ 20/Nov/23 ]

I'm tagging this ticket as related to SERVER-83456. I think this is a bug related to Filter -> Sargable -> Filter conversions which will seem to "go away" if we no longer do those conversions (of course, we need to fix the bug anyway for M4 or for whenever we allow SargableNode conversion)

Comment by Hana Pearlman [ 13/Sep/23 ]

cc daniel.segel@mongodb.com Could SERVER-81008 be the issue here? The repro involved a disjunction and it seems possible, because array comparisons have special translations, that we might need "_retainPredicate" for one or both of the predicates... 

Comment by David Percy [ 28/Jul/23 ]

Could this be the same root cause as SERVER-79205? One involves empty arrays; one involves a missing field. Both are incorrectly returning false.

Generated at Thu Feb 08 06:41:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.