[SERVER-32219] Pipeline optimizer exposure Created: 08/Dec/17 Updated: 06/Dec/22 Resolved: 12/Jan/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Andrew Harris | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query
|
||||||||
| Participants: | |||||||||
| Description |
|
It would useful if you could expose the Aggregation pipeline optimizer so that a pipeline could be fed in and the updated pipeline as optimized by the optimizer could be output. This would not only enable greater awareness of possible optimizations but also save time at execution time as the optimization wouldn't necessarily need to be done. |
| Comments |
| Comment by Asya Kamsky [ 28/Dec/17 ] |
|
aharris yes, you are correct, the output "as is" is not usually in a format that can be fed directly into a new aggregate command. However, it's not really a function of the server to produce that. This might be something that a tool like Compass could do though. Your original request mentioned saving time at execution time as the optimization wouldn't have to be redone but the over pipeline optimization time is negligible compared to actual execution of the pipeline. |
| Comment by Andrew Harris [ 20/Dec/17 ] |
|
Yes and no. Yes, in that it tells you in "explain speak" what the various stages have been translated to, which I knew about already, but no, in that the essence of my suggestion was a tool that actually translated those into AP syntax stages that in itself could be re-used directly by an application. In fact, it appears that the "explain" output makes optimisations that are not possible through the AP stage syntax available. An example of which is outlined below. When submitting the following pseudo stages that would return a maximum of a single joined document... $lookup { from: <some collection> pipeline:[ <$match> <$project> ] as: <field name> }$unwind: {<the 'as' fieldname above>}- this is of course to convert the single element array returned by $lookup into something more easily manageable. ...this is optimised according to the explain plan by merging the $unwind into the $lookup pipeline stages. This doesn't appear to work if done in your own stages as the target field doesn't yet exist. Or am I doing something wrong? |
| Comment by Kelsey Schubert [ 19/Dec/17 ] |
|
Hi aharris, Did Charlie's suggestion fit your needs? Thanks, |
| Comment by Andrew Harris [ 11/Dec/17 ] |
|
I'll take a look at it. Thank you. |
| Comment by Charlie Swanson [ 08/Dec/17 ] |
|
aharris I think you can get this information by running the aggregation and passing the explain option as true: https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/ |