[SERVER-59308] Match After $lookup-$unwind not applied correctly to $lookup results Created: 11/Aug/21  Updated: 29/Oct/23  Resolved: 16/Sep/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 5.0.4, 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Patrick Meredith Assignee: Katherine Wu (Inactive)
Resolution: Fixed Votes: 0
Labels: query-director-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: QO 2021-09-20
Participants:

 Description   

This applies to 5.0.2-rc0

I have narrowed down that this is not an issue with matching on nested document fields in isolation (they work fine), and also, if the nested results from the unwound lookup result are projected into the root document and the match applied to the root document, it works correctly. Additionally, matches on the fields from the current document of the $lookup (rather than the $lookup result) work correctly. This is the specific situation, minimized, as far as I can tell. Here is a repo query with data (showing results in 4.4 and 5.0.2-rc0)

Data:
> db.strings.find()
{ "_id" : ObjectId("611017e028ec416850edf3dc"), "a" : "" }
{ "_id" : ObjectId("611017e328ec416850edf3dd"), "a" : "foo" }
 
4.4-latest:
> db.strings.aggregate([{"$lookup": {"from": "strings","let": {},"pipeline": [],"as": "o"}}, {$unwind: "$o"}, {$match: {"o.a": {$ne: ""}}}])
{ "_id" : ObjectId("61101a3728ec416850edf3df"), "a" : "", "o" : { "_id" : ObjectId("61101a3d28ec416850edf3e0"), "a" : "foo" } }
{ "_id" : ObjectId("61101a3d28ec416850edf3e0"), "a" : "foo", "o" : { "_id" : ObjectId("61101a3d28ec416850edf3e0"), "a" : "foo" } }
 
5.0.2-rc0:
> db.strings.aggregate([{"$lookup": {"from": "strings","let": {},"pipeline": [],"as": "o"}}, {$unwind: "$o"}, {$match: {"o.a": {$ne: ""}}}])
{ "_id" : ObjectId("611017e028ec416850edf3dc"), "a" : "", "o" : { "_id" : ObjectId("611017e028ec416850edf3dc"), "a" : "" } }
{ "_id" : ObjectId("611017e028ec416850edf3dc"), "a" : "", "o" : { "_id" : ObjectId("611017e328ec416850edf3dd"), "a" : "foo" } }
{ "_id" : ObjectId("611017e328ec416850edf3dd"), "a" : "foo", "o" : { "_id" : ObjectId("611017e028ec416850edf3dc"), "a" : "" } }
{ "_id" : ObjectId("611017e328ec416850edf3dd"), "a" : "foo", "o" : { "_id" : ObjectId("611017e328ec416850edf3dd"), "a" : "foo" } }



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 24/Sep/21 ]

Author:

{'name': 'Katherine Wu', 'email': 'katherine.wu@mongodb.com', 'username': 'kaywux'}

Message: SERVER-59308 Ensure empty $lookup subpipeline optimizes correctly with subsequent $match

(cherry picked from commit d5156d91a608a3b7cf30fbdb63a2d31783389a47)
Branch: v5.0
https://github.com/mongodb/mongo/commit/602f7cf6baa8c43e8f3791a1a5ed09f5f42d104c

Comment by Githook User [ 15/Sep/21 ]

Author:

{'name': 'Katherine Wu', 'email': 'katherine.wu@mongodb.com', 'username': 'kaywux'}

Message: SERVER-59308 Ensure empty $lookup subpipeline optimizes correctly with subsequent $match
Branch: master
https://github.com/mongodb/mongo/commit/d5156d91a608a3b7cf30fbdb63a2d31783389a47

Comment by Patrick Meredith [ 12/Aug/21 ]

This problem also exists in 5.0.0 ga.

Comment by Patrick Meredith [ 11/Aug/21 ]

My best guess is that we have an issue in the optimizer rewrites with dropping the $match, but that is not guaranteed to be the issue

Generated at Thu Feb 08 05:46:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.