[SERVER-47001] Reduce number of small buffer allocations for index builds Created: 19/Mar/20  Updated: 29/Oct/23  Resolved: 14/Apr/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Improvement Priority: Major - P3
Reporter: Louis Williams Assignee: Henrik Edin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
causes SERVER-61094 Generating KeyStrings from MultiIndex... Closed
Related
related to SERVER-47349 Use flat_set instead of set for KeySt... Closed
related to SERVER-47350 Reuse temporary data structures durin... Closed
related to SERVER-47416 Eliminate copying of data structures ... Closed
related to SERVER-47417 Add google benchmark for btree key ge... Closed
related to SERVER-47476 Make it possible to store a few eleme... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-04-06, Execution Team 2020-04-20
Participants:
Linked BF Score: 0

 Description   

Index builds allocate a single, small buffer for each key generated. For index builds with small keys, much of the index build CPU time is spent just in memory allocation.

It is generally faster to allocate fewer, larger memory regions than numerous, smaller ones. We can speed up index builds by allocating a few large buffers for key generation and sorting the pointers to keys within those buffers.

This has the added benefit of having better cache locality for sorting since all keys will be nearby in memory.



 Comments   
Comment by Githook User [ 10/Apr/20 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-47001 Added a pooled KeyString builder.

It uses a memory pool in the execution context to be able to share a
larger memory buffer for multiple KeyStrings.

Added SharedBufferFragment and SharedBufferFragmentBuilder to achieve this.
Branch: master
https://github.com/mongodb/mongo/commit/da923cf72003a34a45ce7775dd66ccd944da7d11

Generated at Thu Feb 08 05:13:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.