[SERVER-60197] Support $group pushdown to SBE when $match stage with $or exists in a pipeline Created: 24/Sep/21  Updated: 29/Oct/23  Resolved: 25/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Improvement Priority: Major - P3
Reporter: Yoon Soo Kim Assignee: Yoon Soo Kim
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-60191 A $match and $group pipeline fails wh... Closed
Backwards Compatibility: Fully Compatible
Sprint: QE 2022-01-10, QE 2022-02-07, QE 2022-01-24
Participants:

 Description   

Repro:

execute mongod
$ build/debug/install/bin/mongod --setParameter featureFlagSBEGroupPushdown=true --setParameter internalQueryEnableSlotBasedExecutionEngine=true
 
in MongoDB shell
var coll = db.getCollection("test");
coll.insert({a: 1});
coll.aggregate([{$match: {$or: [{"a": 1}, {"b": 10}]}}, {$group: {_id: "$a"}}]);

In SERVER-60191, we disabled $group pushdown to SBE when $match stage with $or exists in a pipeline to avoid a tripwire assertion: 5842500. This assertion happened because subplanning does not expect that the base query has pushed down $group stage(s) and it does when $group stage exists in pipeline. But two code paths are orthogonal to each other.

Need to figure out how to enable $group pushdown to SBE even when subplanning is involved.



 Comments   
Comment by Githook User [ 25/Jan/22 ]

Author:

{'name': 'Yoonsoo Kim', 'email': 'yoonsoo.kim@mongodb.com', 'username': 'yun-soo'}

Message: SERVER-60197 Fix test cases for $group pushdown when subplanning
Branch: master
https://github.com/mongodb/mongo/commit/526a557b06c4335abd63c15c92a5a78a65587a2c

Comment by Githook User [ 25/Jan/22 ]

Author:

{'name': 'Yoonsoo Kim', 'email': 'yoonsoo.kim@mongodb.com', 'username': 'yun-soo'}

Message: SERVER-60197 Support group pushdown when subplanning is involved
Branch: master
https://github.com/mongodb/mongo/commit/1c1d1cf7befb0059692bccc1e7803738386a743e

Comment by Yoon Soo Kim [ 19/Jan/22 ]

A strange plan is generated though we don't get tripwire assertion.

MongoDB Enterprise > coll.explain().aggregate([matchWithOr, groupPossiblyPushedDown]).queryPlanner.winningPlan.slotBasedPlan.stages;
[2] mkbson s15 s4 [item, quantity] keep [] true false
[1] filter {s14}
[1] nlj [s4, s5] [s4, s5]
    left
        [1] scan s4 s5 none none none none [] @"d47e303b-dc80-4477-a7a3-1caf38fc11ca" true false
    right
        [1] limit 1
        [1] union [s14] [
            [s12] [1] project [s12 = true]
            [1] filter {fillEmpty (s8, false)}
            [1] traverse s8 s7 s6 {s8 || s7} {s8}
            from
                [1] project [s6 = getField (s4, "item")]
                [1] limit 1
                [1] coscan
            in
                [1] project [s7 = fillEmpty (s6 == "a", false)]
                [1] limit 1
                [1] coscan
            ,
            [s13] [1] project [s13 = fillEmpty (s11, false)]
            [1] traverse s11 s10 s9 {s11 || s10} {s11}
            from
                [1] project [s9 = getField (s4, "price")]
                [1] limit 1
                [1] coscan
            in
                [1] project [s10 = fillEmpty (s9 == 10, false)]
                [1] limit 1
                [1] coscan
 
       ]

It does not look like that $group is actually pushed down but only $match.

Comment by Yoon Soo Kim [ 19/Jan/22 ]

Experimented the test case, ignoring sub-planning and got incorrect results.

MongoDB Enterprise > coll.aggregate([matchWithOr, groupPossiblyPushedDown])
{ "item" : "a", "quantity" : 2 }
{ "item" : "a", "quantity" : 5 }
{ "item" : "b", "quantity" : 10 }
MongoDB Enterprise > db.adminCommand({setParameter: 1, internalQueryForceClassicEngine: true});
{ "was" : false, "ok" : 1 }
MongoDB Enterprise > coll.aggregate([matchWithOr, groupPossiblyPushedDown])
{ "_id" : "a", "quantity" : 7 }
{ "_id" : "b", "quantity" : 10 }

It does not seem to be a trivial issue.

Comment by Eric Cox (Inactive) [ 05/Jan/22 ]

yoonsoo.kim Should this work now given what we observed with $unionWith?

Generated at Thu Feb 08 05:49:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.