[SERVER-28818] Enable configuring RAM usage for MongoDB instance Created: 17/Apr/17  Updated: 27/Oct/23  Resolved: 13/Jun/19

Status: Closed
Project: Core Server
Component/s: Admin, Internal Code
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Tomak Ksenia Assignee: Backlog - Service Architecture
Resolution: Gone away Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-33742 MongoDb Ram Usage in a Kubernetes clu... Closed
Assigned Teams:
Service Arch
Participants:
Case:

 Description   

Quote from docs:
"The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory."

We need an instrument to reduce memory (RAM) usage of whole MongoDB subsystem because we should start several services on server. At this moment we can limit only WiredTiger internal cache (storage.wiredTiger.engineConfig.cacheSizeGB config setting).
For windows users the only one way to limit entire instance memory usage - to create virtual machine for MongoDB.
Why don't you add configuration setting to limit entire RAM usage?
Do not offer OS tools.



 Comments   
Comment by Ratika Gandhi [ 13/Jun/19 ]

As of WT-2670, file system cache behavior on windows is non greedy by default. 
As such, we think that the new feature reported in this ticket is no longer necessary. Please open a new ticket if you're asking for something different than this. 

Comment by Bruce Lucas (Inactive) [ 09/Mar/18 ]

Hi Tomak,

In general it's not possible to place specific limits on the amount of memory used by the operating system for its file system cache on behalf of mongod.

However, since you are running on Windows, I suspect you may have been encountering an issue that we fixed in WT-2670: prior to this fix, WiredTiger specified POSIX_FADV_RANDOM for the files that it opened. On Windows this has the unfortunate side effect of making o/s file cache usage for that file "greedy": the o/s won't release cached file data when needed for other purposes. Since WT-2670 we no longer specify POSIX_FADV_RANDOM by default, and this results in much better cache behavior: the o/s will cache the files, but will release that memory when needed.

Can you test your workload on a version that includes WT-2670 and let us know if that addresses your concern? Those versions would be 3.2.12 or later, 3.4.2 or later, and 3.6.0 or later.

Thanks,
Bruce

Comment by Boas Enkler [ 08/Mar/18 ]

@Ramon Fernandez an other point which makes this from my point of view a keyfeature is that we need it for running in kubernetes.
Currently as mongodb isn't aware of the ram a kubernetes node has the nodes run often in out out memory scnearios and there fore the pods are getting "uncontrolled" and suddenly terminated.

As a result in some scenarios (e.g. batch imports or other heavy data related operations) mongodb isn't a good solution in k8s scenarios.

or is there any way to mimic the cgroup behavior in kubernetes pods?

Comment by Tomak Ksenia [ 21/Apr/17 ]

Thanks for your reply. We will track this issue.
Unfortunately, we have windows servers on production. It means we haven't OS tools.

Comment by Ramon Fernandez Marina [ 20/Apr/17 ]

Thanks for your report tomak_k.s.. The functionality you request is part of our long-term roadmap, so I'm going to use this ticket to keep track of it. Unfortunately this feature won't happen for a while, and until then the only way to limit the amount of memory mongod uses is via operating system tools (e.g.: cgroups on linux).

Regards,
Ramón.

Generated at Thu Feb 08 04:19:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.