[SERVER-49027] Extend $group explain execution stats to report memory consumption per accumulator Created: 23/Jun/20 Updated: 29/Oct/23 Resolved: 08/Dec/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework, Diagnostics, Querying |
| Affects Version/s: | None |
| Fix Version/s: | 4.9.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | David Storch | Assignee: | Ruoxin Xu |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | qexec-team | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Query 2020-11-30, Query 2020-12-14 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
We've seen several cases in the field of $group operations which use $addToSet or $push to create very large arrays. This can lead to the memory footprint being large. In related ticket In order to help users understand which operators in the query plan are consuming memory, we should extend "executionStats" and "allPlansExecution" verbosity explain output for aggregate commands to include the maximum memory footprint attained at query runtime on a per-accumulator basis. |
| Comments |
| Comment by Githook User [ 07/Dec/20 ] | ||||||||||||||||||||||||||||
|
Author: {'name': 'Ruoxin Xu', 'email': 'ruoxin.xu@mongodb.com', 'username': 'RuoxinXu'}Message: | ||||||||||||||||||||||||||||
| Comment by Katya Kamenieva [ 24/Nov/20 ] | ||||||||||||||||||||||||||||
|
I think this is fine | ||||||||||||||||||||||||||||
| Comment by Ruoxin Xu [ 24/Nov/20 ] | ||||||||||||||||||||||||||||
|
The current patch would add a new field called "$maxAccumulatorMemoryUsageBytes" in $group stage's explain output, something like,
kateryna.kamenieva Any idea in terms of the representation of the accumulator memory consumption stats in explain output? |