[SERVER-40317] $facet execution has no limit on how much memory it can consume Created: 22/Mar/19 Updated: 29/Oct/23 Resolved: 14/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.20, 4.7.0, 4.2.10, 4.4.2, 4.0.21 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Justin Seyster | Assignee: | David Storch |
| Resolution: | Fixed | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||
| Backport Requested: |
v4.4, v4.2, v4.0, v3.6
|
||||||||||||||||||||||||||||
| Sprint: | Query 2020-08-24 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||
| Linked BF Score: | 21 | ||||||||||||||||||||||||||||
| Description |
|
A $facet always outputs one document but it completely executes all of its pipelines no matter how large they are. To avoid unreasonable memory consumption, the $facet should return an error as soon as it has >100MB of results, rather than waiting until it's computed all of its data. Note that the $facet's output document is allowed to be up to 100MB large if it is an intermediate result. However, all documents produced by an aggregation pipeline's result set must be 16MB or less due to the BSON size limit. So the 100MB limit applies to the output document produced by $facet, but the pipeline will still fail unless the size of this document is subsequently reduced to 16MB or less. Also, note that the value of allowDiskUse does not affect the behavior implemented under this ticket, since the $facet stage's current implementation cannot spill to disk. |
| Comments |
| Comment by Githook User [ 11/Sep/20 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: Co-authored-by: Justin Seyster <justin.seyster@mongodb.com> |
| Comment by Githook User [ 25/Aug/20 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: Co-authored-by: Justin Seyster <justin.seyster@mongodb.com> |
| Comment by Githook User [ 25/Aug/20 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: Co-authored-by: Justin Seyster <justin.seyster@mongodb.com> |
| Comment by Githook User [ 24/Aug/20 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: Co-authored-by: Justin Seyster <justin.seyster@mongodb.com> |
| Comment by Githook User [ 14/Aug/20 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: Co-authored-by: Justin Seyster <justin.seyster@mongodb.com> |
| Comment by Jacob Evans [ 06/Jun/19 ] |
|
We decided that this was lower value than rethinking our $facet passthrough which this would necessitate. The passthrough would hit the proposed limit many times and require engineering effort to manually disable with blacklisting. |