[SERVER-73717] When using in-memory storage engine spill to disk doesn't always work Created: 07/Feb/23 Updated: 11/Apr/23 Resolved: 11/Apr/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Adi Agrawal | Assignee: | Backlog - Query Execution |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Query Execution
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
The Hash Agg stage in the SBE engine spills using a record store as opposed to files as in the classic engine. There is a statistic query execution tracks "spilledDataStorageSize" introduced in https://jira.mongodb.org/browse/SERVER-73311 which reports the bytes spilled to the record store. When testing this behavior on an in memory required build variant, the value of spilledDataStorage size is 0, even though spilling is allowed. ([patch build: ). This poses the question: Should we even allow spilling to record stores when running on an in memory build? If no, we can disable spilling when in memory stores. If yes, we can spill to a file instead of record store, or ignore the memory limit. |
| Comments |
| Comment by David Storch [ 07/Feb/23 ] |
|
I see the following options:
Option (3) probably makes the most sense, but I'm not sure how easy it is to implement. |