[SERVER-32219] Pipeline optimizer exposure Created: 08/Dec/17  Updated: 06/Dec/22  Resolved: 12/Jan/18

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Andrew Harris Assignee: Backlog - Query Team (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-12527 Expose ability to directly execute a ... Open
Assigned Teams:
Query
Participants:

 Description   

It would useful if you could expose the Aggregation pipeline optimizer so that a pipeline could be fed in and the updated pipeline as optimized by the optimizer could be output. This would not only enable greater awareness of possible optimizations but also save time at execution time as the optimization wouldn't necessarily need to be done.



 Comments   
Comment by Asya Kamsky [ 28/Dec/17 ]

aharris yes, you are correct, the output "as is" is not usually in a format that can be fed directly into a new aggregate command. However, it's not really a function of the server to produce that. This might be something that a tool like Compass could do though.

Your original request mentioned saving time at execution time as the optimization wouldn't have to be redone but the over pipeline optimization time is negligible compared to actual execution of the pipeline.

Comment by Andrew Harris [ 20/Dec/17 ]

Yes and no. Yes, in that it tells you in "explain speak" what the various stages have been translated to, which I knew about already, but no, in that the essence of my suggestion was a tool that actually translated those into AP syntax stages that in itself could be re-used directly by an application. In fact, it appears that the "explain" output makes optimisations that are not possible through the AP stage syntax available. An example of which is outlined below.

When submitting the following pseudo stages that would return a maximum of a single joined document...

$lookup

{ from: <some collection> pipeline:[ <$match> <$project> ] as: <field name> }

$unwind:

{<the 'as' fieldname above>}

- this is of course to convert the single element array returned by $lookup into something more easily manageable.

...this is optimised according to the explain plan by merging the $unwind into the $lookup pipeline stages. This doesn't appear to work if done in your own stages as the target field doesn't yet exist.

Or am I doing something wrong?

Comment by Kelsey Schubert [ 19/Dec/17 ]

Hi aharris,

Did Charlie's suggestion fit your needs?

Thanks,
Kelsey

Comment by Andrew Harris [ 11/Dec/17 ]

I'll take a look at it. Thank you.

Comment by Charlie Swanson [ 08/Dec/17 ]

aharris I think you can get this information by running the aggregation and passing the explain option as true: https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/

Generated at Thu Feb 08 04:29:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.