[COMPASS-6217] Investigate changes in SERVER-53657: $group's executionStats should report the spilled data file size Created: 20/Oct/22  Updated: 28/Nov/22  Resolved: 28/Nov/22

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: No version

Type: Investigation Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Anna Henningsen
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-53657 $group's executionStats should report... Closed
Documentation Changes: Not Needed

 Description   
Original Downstream Change Summary

This commit introduces two fields to the execution stats explain output of $group in the classic engine:

  • spillFileSizeBytes, which reports the size of the spill file written to disk
  • numBytesSpilledEstimate, which reports an estimate of the size of $group data evicted from memory as part of spilling

Note that these are not the same quantity, typically, spillFileSizeBytes will be less than numBytesSpilledEstimate because the buffer will be compressed before being written to disk.

Description of Linked Ticket

$group's executionStats should expose a stats "spillFileSizeBytes". This stat should report the maximum files size bytes that was written to disk in bytes. This stat will help in better planning/recommendation on disk usage and maybe I/O capacity planning.



 Comments   
Comment by Anna Henningsen [ 28/Nov/22 ]

No devtools impact.

Generated at Wed Feb 07 22:42:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.