[SERVER-22631] Improve efficiency of matching during an aggregation pipeline. Created: 16/Feb/16  Updated: 03/Jan/17  Resolved: 13/Apr/16

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 3.3.5

Type: Improvement Priority: Major - P3
Reporter: Charlie Swanson Assignee: Benjamin Murphy
Resolution: Done Votes: 0
Labels: optimization
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-22404 Extend MatchExpression to support mat... Closed
Related
related to SERVER-27213 Two $match stages combine incorrectly... Closed
related to SERVER-25911 Tune performance of $match stage Closed
is related to SERVER-25304 Allow a MatchExpression to match a Do... Backlog
is related to SERVER-22402 Add dependency tracking to the $match... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 13 (04/22/16)
Participants:

 Description   

It's currently very expensive to perform a $match within the aggregation pipeline. This is mostly because the aggregation pipeline works with Documents, which are incompatible with the current MatchExpression::matchesBSON interface, so we have to construct an entirely new BSONObj to match on. There are two things we could do to make this more efficient:

  1. Instead of converting the entire Document to BSON, we could only convert the fields needed by the match. For example, if the match was {a: {$gte: 4}}, we would only need the field a from the document.
  2. We could extend the MatchExpression interface to support matching Documents. A Document is logically equivalent to a BSONObj, so it would just be a matter of adapting the matching to use a different API with the object.


 Comments   
Comment by Githook User [ 10/Nov/16 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'cswanson310@gmail.com'}

Message: PERF-784 Add perf tests for SERVER-22631
Branch: master
https://github.com/mongodb/mongo-perf/commit/80570d3f5e3cc12ee8a469e64d025600782448bb

Comment by Githook User [ 10/Nov/16 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'cswanson310@gmail.com'}

Message: PERF-784 Add perf tests for SERVER-22631
Branch: develop
https://github.com/mongodb/mongo-perf/commit/80570d3f5e3cc12ee8a469e64d025600782448bb

Comment by Benjamin Murphy [ 13/Apr/16 ]

Commit 6df76e05dbd86827af7ae9b9676833348537f870 took approach #1. The second approach is still possible, and would likely result in more performance improvements, but would require significantly longer to implement.

Comment by Githook User [ 13/Apr/16 ]

Author:

{u'username': u'benjaminmurphy', u'name': u'Benjamin Murphy', u'email': u'benjamin_murphy@me.com'}

Message: SERVER-22631 Improve matching efficiency in aggregation.
Branch: master
https://github.com/mongodb/mongo/commit/6df76e05dbd86827af7ae9b9676833348537f870

Generated at Thu Feb 08 04:00:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.