[SERVER-44095] S2 key generation vector should use a smaller type Created: 18/Oct/19 Updated: 29/Oct/23 Resolved: 10/Mar/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.0-rc0, 4.7.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Louis Williams |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Backport Requested: |
v4.4
|
||||||||
| Sprint: | Execution Team 2020-03-09, Execution Team 2020-03-23 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 49 | ||||||||
| Description |
|
S2, "2dsphere", indexes generate keys usingĀ a vector of KeyString::HeapBuilder. Unfortunately, this type is very large (608 bytes), as compared to the previously used BSONObj (16 bytes). This type is large because of the StackBufBuilder used for the TypeBits member, which when not allocated on the stack, adds an additional 512 bytes to the size of this type. Most of this space is unused. This causes problems for 2dsphere indexes, which when building a compound key where the first field is an array, need to allocate as many KeyString::HeapBuilders as there are elements in the array. If the array is very large, say 1500000 elements, then the minimum memory requirement to generate the keys is close to 900MB. We should find a way to use a smaller type in this vector, KeyString::Value (24 bytes) for example, and resort to copying it as necessary. An alternative would be to make TypeBits support the BufBuilder instead of the StackBufBuilder, but that would only reduce the HeapBuilder type to around 96 bytes. |
| Comments |
| Comment by Githook User [ 11/Mar/20 ] |
|
Author: {'name': 'Louis Williams', 'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com'}Message: This lowers the memory overhead of index key generation by reducing the (cherry picked from commit 4994a4809269c30a28cdbb686c8c876fad7b163a) |
| Comment by Githook User [ 09/Mar/20 ] |
|
Author: {'name': 'Louis Williams', 'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com'}Message: This lowers the memory overhead of index key generation by reducing the |
| Comment by Connie Chen [ 17/Jan/20 ] |
|
Check on geo-indexes, request from Atlas |