[SERVER-36709] Make radix store nodes adaptive to improve memory efficiency Created: 16/Aug/18  Updated: 29/Oct/23  Resolved: 07/Aug/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: Audrey Fang (Inactive) Assignee: Yuhong Zhang
Resolution: Fixed Votes: 0
Labels: intern_ephemeral_for_test, nyc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-38939 Be more memory efficient for leaf nod... Closed
Related
is related to SERVER-49008 Create a new jstest to track Biggie's... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-07-27, Execution Team 2020-08-10, Execution Team 2020-08-24
Participants:

 Description   

Currently, each node in the tree, including leaf pointers, has 256 shared pointers. Each shared pointer is 16 bytes, for a total of 4kB for the pointers. As the first allocation size > 4kB is 6 kB, and each document results in 2 nodes (one for the index, one for record store), each document has 12kB overhead (18 kB for replica sets due to the oplog).
This causes each Node to be > 4kB resulting in excessive memory usage during testing and OOMs.

https://db.in.tum.de/~leis/papers/ART.pdf



 Comments   
Comment by Githook User [ 11/Aug/20 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-36709 Enable previously disabled tests due to excessive memory usage
Branch: master
https://github.com/mongodb/mongo/commit/0dc74f5dc61a5a093ece56887083f8a934a45140

Comment by Githook User [ 07/Aug/20 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'danielzhangyh@gmail.com', 'username': 'YuhongZhang98'}

Message: SERVER-36709 Make radix store nodes adaptive to improve memory efficiency
Branch: master
https://github.com/mongodb/mongo/commit/d0fb0f3a888f2ce8923ab03fc5d88266af7d444e

Comment by Githook User [ 07/Aug/20 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: Revert "SERVER-36709 Make Biggie store nodes adaptive to improve memory efficiency"

This reverts commit bf95b1fabd4ee9b2154763f86eaa8f6c7ba3370c.
Branch: master
https://github.com/mongodb/mongo/commit/5ce374693fa02d05894f7119b39e11dd352b0d34

Comment by Kaloian Manassiev [ 07/Aug/20 ]

The changes for SERVER-49553 and SERVER-36709 break the Windows compile, so I am reverting them.

Comment by Githook User [ 07/Aug/20 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'danielzhangyh@gmail.com', 'username': 'YuhongZhang98'}

Message: SERVER-36709 Make Biggie store nodes adaptive to improve memory efficiency

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709debug

server-36709

server-36709

server-36709

server-36709

server-36709

only 256

server-36709 48&256

server-36709 16-256 without shrink

server-36709 16-256

server-36709 4-256

server-36709 0-256

server-36709 cleanup

server-36709
Branch: master
https://github.com/mongodb/mongo/commit/bf95b1fabd4ee9b2154763f86eaa8f6c7ba3370c

Comment by Githook User [ 07/Aug/20 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'danielzhangyh@gmail.com', 'username': 'YuhongZhang98'}

Message: SERVER-36709 Make Biggie store nodes adaptive to improve memory efficiency

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709

server-36709debug

server-36709

server-36709

server-36709

server-36709

server-36709

only 256

server-36709 48&256

server-36709 16-256 without shrink

server-36709 16-256

server-36709 4-256

server-36709 0-256

server-36709 cleanup

server-36709
Branch: master
https://github.com/mongodb/mongo/commit/bf95b1fabd4ee9b2154763f86eaa8f6c7ba3370c

Generated at Thu Feb 08 04:43:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.