[SERVER-26507] BSONObj::memUsageForSorter() should use buffer size instead of object size Created: 06/Oct/16  Updated: 06/Dec/22

Status: Open
Project: Core Server
Component/s: Index Maintenance, Internal Code
Affects Version/s: None
Fix Version/s: features we're not sure of

Type: Improvement Priority: Major - P3
Reporter: Tess Avitabile (Inactive) Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Participants:

 Description   

The sorter's memory requirements depend on the buffer size of the index keys, but BSONObj::memUsageForSorter() uses the object size of the index keys. This requires the developer to ensure that the object size match the buffer size for index keys, or else the sorter's memory usage estimates will be off, and it can run out of memory. This leads to bugs like SERVER-25075 and SERVER-26488. It is still useful to optimize the buffer size for index keys for efficiency, but it would be better if forgetting to do so did not cause OOM failures.

We would have to consider the performance implications of storing the buffer size of a BSONObj.



 Comments   
Comment by Bruce Lucas (Inactive) [ 06/Oct/16 ]

it would be better if forgetting to do so did not cause OOM failures

Agreed, but it would also be good if forgetting to do so left some visible trace other than (presumably) reduced performance due to using a smaller total sort size due to the wasted space. I don't however have atm a good idea how that might be done.

Generated at Thu Feb 08 04:12:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.