[SERVER-81502] $lookup with $documents and localField/foreignField fails to re-parse Created: 27/Sep/23  Updated: 29/Oct/23  Resolved: 11/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Charlie Swanson Assignee: Alyssa Clark
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-85096 TRACKING: M3 Correctness Tickets Closed
Related
is related to SERVER-76330 Use representative query shape in que... Closed
Assigned Teams:
Query Integration
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

The pipeline in this test case fails if we try to run $queryStats afterwards. It gives this error:

            pymongo.errors.OperationFailure: PlanExecutor error during aggregation :: caused by :: Was not able to re-parse queryStats key when reading queryStats.Status Location40602: $queue is only valid as the first stage in a pipeline Hash: 9564851472401003349 Query Shape: { cmdNs: { db: "documents", coll: "lookup_coll" }, command: "aggregate", pipeline: [ { $lookup: { from: "$cmd.aggregate", as: "names", localField: "id_name", foreignField: "xx", let: {}, pipeline: [ { $queue: [ { ?: "?" } ] }, { $project: { _id: true, ed6595d4-e41d-438b-95ca-33d56d3b5ae5: { $const: [ { ?: "?" } ] } } }, { $unwind: { path: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } }, { $replaceRoot: { newRoot: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } } ] } }, { $match: { names: { $not: { $eq: [] } } } }, { $project: { _id: false } } ] }, full error: {'ok': 0.0, 'errmsg': 'PlanExecutor error during aggregation :: caused by :: Was not able to re-parse queryStats key when reading queryStats.Status Location40602: $queue is only valid as the first stage in a pipeline Hash: 9564851472401003349 Query Shape: { cmdNs: { db: "documents", coll: "lookup_coll" }, command: "aggregate", pipeline: [ { $lookup: { from: "$cmd.aggregate", as: "names", localField: "id_name", foreignField: "xx", let: {}, pipeline: [ { $queue: [ { ?: "?" } ] }, { $project: { _id: true, ed6595d4-e41d-438b-95ca-33d56d3b5ae5: { $const: [ { ?: "?" } ] } } }, { $unwind: { path: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } }, { $replaceRoot: { newRoot: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } } ] } }, { $match: { names: { $not: { $eq: [] } } } }, { $project: { _id: false } } ] }', 'code': 7349401, 'codeName': 'Location7349401'}

The object in there is this:

{ cmdNs: 
    { db: "documents", coll: "lookup_coll" },
     command: "aggregate",
      pipeline: [
         { $lookup: {
             from: "$cmd.aggregate", 
             as: "names", 
             localField: "id_name", 
             foreignField: "xx", 
             let: {}, 
             pipeline: [ 
                { $queue: [ { "?": "?" } ] },
                 { $project: { _id: true, "ed6595d4-e41d-438b-95ca-33d56d3b5ae5": { $const: [ { "?": "?" } ] } } },
                  { $unwind: { path: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } }, 
                  { $replaceRoot: { newRoot: "$ed6595d4-e41d-438b-95ca-33d56d3b5ae5" } } 
                ] } },
          { $match: { names: { $not: { $eq: [] } } } },
           { $project: { _id: false } } ] };



 Comments   
Comment by Githook User [ 11/Oct/23 ]

Author:

{'name': 'Alyssa Wagenmaker', 'email': 'alyssa.wagenmaker@mongodb.com', 'username': 'awagenmaker'}

Message: SERVER-81502 Support queue in lookup sub-pipeline
When we parse the sub-pipeline for representative pipeline serialization, documents becomes a queue (followed by a few other stages). This change allows lookup to re-parse when its sub-pipeline starts with queue.
Branch: master
https://github.com/mongodb/mongo/commit/ba6c02859a2e7d2caad252a4a00da7db261c91db

Generated at Thu Feb 08 06:46:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.