[SERVER-73943] Pin code pages in memory in memory constrained systems Created: 13/Feb/23 Updated: 18/Nov/23 Resolved: 04/Apr/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.0.0-rc0, 6.0.6, 5.0.19, 4.4.23 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Andy Schwerin | Assignee: | Jason Chan |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | bkp | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Service Arch
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Backport Requested: |
v6.0, v5.0, v4.4
|
||||||||||||||||||||||||
| Sprint: | Service Arch 2023-02-20, Service Arch 2023-03-06, Service Arch 2023-03-20, Service Arch 2023-04-03, Service Arch 2023-04-17 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||
| Description |
|
When mongod is under intense memory pressure on systems that do not have swap space, the operating system pages out the only pages it can when mongod makes a request for more privately mapped memory: program text segment (code) pages. It pages them out because they are available for retrieval from stable storage. Unfortunately, this means that under heavy load mongod spends an immense amount of time paging code in from disk, which can make the system become unresponsive when it would be better for it to crash. We should offer the option to pin code segments in memory via the mlock(2) syscall. The stack tracing machinery already knows how to find the base address of every code segment loaded into the process, and could be extended with modest effort to also report the size of those segments. This would in turn allow us to invoke `mlock` to pin those pages in memory. |
| Comments |
| Comment by Githook User [ 05/Jun/23 ] |
|
Author: {'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}Message: (cherry picked from commit 85878215714ad3851e6067fce5c080dc8fe00bda) This also includes a subset backport of errno_util from 4b4a334 (cherry picked from commit dda4904d91448ade529b18f2c35815fcd0e48a6c) |
| Comment by Githook User [ 10/May/23 ] |
|
Author: {'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}Message: (cherry picked from commit 85878215714ad3851e6067fce5c080dc8fe00bda) This also includes a subset backport of errno_util from 4b4a334 |
| Comment by Githook User [ 24/Apr/23 ] |
|
Author: {'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}Message: |
| Comment by Githook User [ 20/Apr/23 ] |
|
Author: {'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}Message: |
| Comment by Githook User [ 04/Apr/23 ] |
|
Author: {'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}Message: |
| Comment by Louis Williams [ 13/Feb/23 ] |
|
FYI we already do this for some special memory allocations today. |