[SERVER-60137] Improve Buffer Allocation for Common BSON-related Sizes Created: 22/Sep/21 Updated: 29/Oct/23 Resolved: 05/Oct/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Query Execution |
| Affects Version/s: | None |
| Fix Version/s: | 5.1.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Mohammad Dashti (Inactive) | Assignee: | Mohammad Dashti (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Sprint: | QE 2021-10-04, QE 2021-10-18 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
Currently, when we are requesting a piece of memory that might contain a BSON object, it usually requires BSONObjMaxUserSize bytes (i.e., 16MB) plus a residual amount of memory for its wrapper data structure, which is BSONObjMaxInternalSize bytes (i.e., 16MB+16KB). Then, we ask the buffer builder to allocate this much memory, it tries to allocate an amount of memory that's a power of two. In this case, it allocates 32MB of memory, which is usually a waste of resources. In this ticket, based on the discussions with bernard.gorman, david.storch and redbeard0531, we want to change the buffer builder logic in the grow_reallocate to allocate BSONObjMaxInternalSize bytes if the requested re-allocated amount is anything between BSONObjMaxUserSize and BSONObjMaxInternalSize bytes. |
| Comments |
| Comment by Vivian Ge (Inactive) [ 06/Oct/21 ] |
|
Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you! |
| Comment by Mohammad Dashti (Inactive) [ 05/Oct/21 ] |
|
redbeard0531 Yeah, it'll be done in |
| Comment by Mathias Stearn [ 05/Oct/21 ] |
|
This change didn't adjust for the 8 bytes used for the SharedBuffer::Holder. Is that coming in a later commit? |
| Comment by Githook User [ 05/Oct/21 ] |
|
Author: {'name': 'Mohammad Dashti', 'email': 'mdashti@gmail.com', 'username': 'mdashti'}Message: |
| Comment by Mathias Stearn [ 22/Sep/21 ] |
|
BSONObjMaxInternalSize isn't enough. That is only the maximum size of a single bson object, however this allocation size is for the message, which has an additional envelope structure. I think 16M+64K aught to be enough to not worry about going over, while still having a very low relative overhead (well under 1%). |