[SERVER-73943] Pin code pages in memory in memory constrained systems Created: 13/Feb/23  Updated: 18/Nov/23  Resolved: 04/Apr/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0, 6.0.6, 5.0.19, 4.4.23

Type: Improvement Priority: Major - P3
Reporter: Andy Schwerin Assignee: Jason Chan
Resolution: Fixed Votes: 0
Labels: bkp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-75632 Enable lockCodeSegmentsInMemory in co... Backlog
is related to SERVER-77226 Fix permissions-checking when pinning... Closed
Assigned Teams:
Service Arch
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.0, v5.0, v4.4
Sprint: Service Arch 2023-02-20, Service Arch 2023-03-06, Service Arch 2023-03-20, Service Arch 2023-04-03, Service Arch 2023-04-17
Participants:
Case:

 Description   

When mongod is under intense memory pressure on systems that do not have swap space, the operating system pages out the only pages it can when mongod makes a request for more privately mapped memory: program text segment (code) pages. It pages them out because they are available for retrieval from stable storage. Unfortunately, this means that under heavy load mongod spends an immense amount of time paging code in from disk, which can make the system become unresponsive when it would be better for it to crash.

We should offer the option to pin code segments in memory via the mlock(2) syscall. The stack tracing machinery already knows how to find the base address of every code segment loaded into the process, and could be extended with modest effort to also report the size of those segments. This would in turn allow us to invoke `mlock` to pin those pages in memory.



 Comments   
Comment by Githook User [ 05/Jun/23 ]

Author:

{'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}

Message: SERVER-73943 Pin program code segments in memory on startup

(cherry picked from commit 85878215714ad3851e6067fce5c080dc8fe00bda)

This also includes a subset backport of errno_util from 4b4a334

(cherry picked from commit dda4904d91448ade529b18f2c35815fcd0e48a6c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/927ee113113a833f099348d336896cfb0656bd81

Comment by Githook User [ 10/May/23 ]

Author:

{'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}

Message: SERVER-73943 Pin program code segments in memory on startup

(cherry picked from commit 85878215714ad3851e6067fce5c080dc8fe00bda)

This also includes a subset backport of errno_util from 4b4a334
Branch: v5.0
https://github.com/mongodb/mongo/commit/dda4904d91448ade529b18f2c35815fcd0e48a6c

Comment by Githook User [ 24/Apr/23 ]

Author:

{'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}

Message: SERVER-73943 Exclude pin_code_segments_on_startup.js from rhel-82-arm64
Branch: v6.0
https://github.com/mongodb/mongo/commit/54214eaa12304afe9385c93b8fd8a2eeec35b171

Comment by Githook User [ 20/Apr/23 ]

Author:

{'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}

Message: SERVER-73943 Pin program code segments in memory on startup
Branch: v6.0
https://github.com/mongodb/mongo/commit/85878215714ad3851e6067fce5c080dc8fe00bda

Comment by Githook User [ 04/Apr/23 ]

Author:

{'name': 'Jason Chan', 'email': 'jason.chan@mongodb.com', 'username': 'jasonjhchan'}

Message: SERVER-73943 Pin program code segments in memory on startup
Branch: master
https://github.com/mongodb/mongo/commit/8f27a596df0b2be9a44a8fff6b9bf1985937e25b

Comment by Louis Williams [ 13/Feb/23 ]

FYI we already do this for some special memory allocations today.

Generated at Thu Feb 08 06:26:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.