[SERVER-44964] [4.2] Calculate index build memory usage per key based on BSONObj buffer capacity, not size Created: 05/Dec/19 Updated: 29/Oct/23 Resolved: 25/Dec/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 3.6.0, 4.0.0, 4.2.0 |
| Fix Version/s: | 4.2.3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Eric Milkie |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Sprint: | Execution Team 2019-12-30 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 0 | ||||||||
| Description |
|
Index builds limit their memory usage with the parameter maxIndexBuildMemoryUsageMegabytes, which defaults to 500MB. The memory usage calculation depends on the BSONObj accurately reporting its memory usuage in memUsageForSorter(). BSONObj reports its memory usage as the size of the object, but not the capacity of the buffer holding that object. This is problematic because index builds can use significantly more memory than they they should. We should change the imlementation of memUsageForSorter() to report the size of the underlying SharedBuffer, in a very similar way to what we did for in
|
| Comments |
| Comment by Githook User [ 23/Dec/19 ] |
|
Author: {'name': 'Eric Milkie', 'email': 'milkie@mongodb.com', 'username': 'milkie'}Message: |