[SERVER-24860] Optimize away entire pipeline if it can be answered using a query Created: 30/Jun/16  Updated: 29/Mar/22  Resolved: 03/Apr/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Querying
Affects Version/s: None
Fix Version/s: 4.1.10

Type: Improvement Priority: Major - P3
Reporter: Charlie Swanson Assignee: Anton Korshunov
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-45069 Add flag in the log when aggregation ... Closed
Documented
is documented by DOCS-12597 Docs for SERVER-24860: Optimize away ... Closed
Duplicate
is duplicated by SERVER-36983 Views are unnecessary slow, even for ... Closed
Problem/Incident
causes PHPLIB-431 Relax aggregate explain test for 4.2 ... Closed
causes RUBY-1788 Aggregate explain broken on 4.2 server Closed
Related
related to SERVER-7568 Aggregation framework favors non-bloc... Closed
is related to SERVER-40253 Pipelines with text search queries sh... Closed
is related to SERVER-40254 Collapse project stage in a pipeline ... Closed
is related to SERVER-40323 Plan analyzer helper functions cannot... Closed
is related to SERVER-36723 $limit should push down into the Plan... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 2017-04-17, Query 2017-05-08, Query 2017-05-29, Query 2019-03-11, Query 2019-03-25, Query 2019-04-08
Participants:
Case:
Linked BF Score: 0

 Description   

An aggregation pipeline can be used to represent a query, like the following:

db.example.aggregate([{$project: {excluded: 0}}, {$match: {predicate: true}}]);

In these cases, it would be more efficient to skip the aggregation layer, and execute the entire computation using a query.

It is expected that pipelines like the above pipeline will be much more common once non-materialized views are introduced in SERVER-142.



 Comments   
Comment by Githook User [ 03/Apr/19 ]

Author:

{'name': 'Anton Korshunov', 'username': 'antkorsh', 'email': 'anton.korshunov@mongodb.com'}

Message: SERVER-24860 Optimize away entire pipeline if it can be answered using a query
Branch: master
https://github.com/mongodb/mongo/commit/d47bbc343af0b5bbde7b810f63e6b3404ea9e4d6

Comment by Asya Kamsky [ 22/Mar/19 ]

Yes, that's fine.

Comment by Anton Korshunov [ 21/Mar/19 ]

asya just want to double check that you're ok with the proposed name for this boolean flag. Dave and Charlie are fine with the name.

"optimizedPipeline" : true

Comment by Asya Kamsky [ 13/Mar/19 ]

anton.korshunov I think the new format is fine, I agree that additional flag with Boolean indicator what happened is nice to have. Especially since sometimes support gets explain output without the command that produced it.

Comment by Craig Homa [ 28/Feb/19 ]

Removed from the all cursors globally managed epic as this was always more of a 'nice to have' and not a goal in the original scope.

Comment by Kyle Suarez [ 05/Sep/18 ]

See SERVER-36983 for an informal profiling analysis of the overhead of the BSON-Document transformations (and thus the potential performance win we might get by bypassing the aggregation system altogether).

Comment by Mathias Stearn [ 30/Jun/16 ]

This could also handle cases including $skip and $limit, although we'd have to reverse the limit-before-skip optimization before pushing down to the query system because it logically is skip-then-limit (although it has a similar optimization when combined with sorts).

Generated at Thu Feb 08 04:07:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.