[SERVER-54569] Hombrew TAP for OSX has a NumbersofFile set too high Created: 14/Feb/21  Updated: 03/Mar/21  Resolved: 01/Mar/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Joe Drumgoole (Inactive) Assignee: Ryan Egesdahl (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File mongo-launch.sh    
Operating System: ALL
Sprint: Dev Platform 2021-03-08
Participants:

 Description   

Issue raised on twitter: 

 

https://twitter.com/youmeek/status/1360676436093390850

 
@MongoDB
 please fix a bug with Mac OS homebrew tap: In LaunchAgent plist file, there is NumbersOfFile is 64000 (but It doesn’t work, because it is too high value and MacOS limits it back to 256).



 Comments   
Comment by Ryan Egesdahl (Inactive) [ 01/Mar/21 ]

After some discussion, we don't think this is something we can reasonably "fix" on OSX. Our conclusion is based on fact that OSX is not a supported production OS for a reason: it's usually not set up for hosting a production MongoDB, and this is almost certainly true when Homebrew is installing it. I've explained the mechanism of the problem and some workarounds in my previous comment.

One other workaround I mentioned I would attempt is a wrapper script that sets limits before executing the service. While I haven't been able to test it myself (or ask someone to test it for me), I am reasonably sure it would work with corresponding changes to the .plist file. This is what the script looks like (and I've also attached it to the ticket):

#!/bin/sh
 
set -eu
 
if [ "$#" -lt 1 ] || ! [ -x "$1" ]; then
    echo "Usage: $0 <executable>" >&2
    exit 1
fi
 
ulimit -Sn 64000
ulimit -Hn 64000
 
exec "$@"

Again, while you can do it if you possess the knowledge and skills to do so, we cannot support production MongoDB on OSX. We don't even recommend doing performance testing. The message you're seeing is just a warning, and for development purposes, MongoDB will run with no noticeable performance issues. If the error message bothers you, or if you are just looking for a start on learning how to set up OSX to run production MongoDB, I've given some workarounds here that should help you. You'll really need to know what you're doing, though, since you're making changes OSX desktops weren't meant for. Please be careful.

Comment by Ryan Egesdahl (Inactive) [ 25/Feb/21 ]

brusheg@gmail.com The launchctl command and response in the shell are telling me that what you're seeing is probably this:

https://github.com/Homebrew/homebrew-core/issues/48277

Unfortunately, there might not be a lot we can change to fix that issue (but I'll try), since any maxfiles we set above 256 is just going to get limited because launchd is sets the minimum of the maximum allowable softlimit (256) and whatever is in the .plist. This is the default OSX configuration unless you have it set up as a server, and the recommended minimum open files for a production MongoDB server really is 64000 for best performance. If you're just doing some low-volume testing and don't expect to have a lot of traffic, you can ignore the message (it's a warning).

If you're wanting to do performance testing and don't have a server available, you will need to make the appropriate changes in /etc/sysctl.conf and set up launchd to allow a higher soft limit before you start. You can do that one of two ways:

  • For session before you manually launch services by running launchctl limit maxfiles 64000 unlimited (that might need to be run with sudo - I don't have an OSX machine to test with)
  • Permanently by running echo 'limit maxfiles 64000 unlimited' | sudo tee -a /etc/launchd.conf (and rebooting, of course)

There is one possibility I am unable to test at the moment: if launchd starts a script that sets up the appropriate ulimit before starting services, we might be able to set a higher limit. I'll write a quick launcher script and ask colleague to test it for me so we can see if it's a viable solution. No guarantees, though - I don't see any hints in the documentation that launchd will even let us do this.

Comment by Ал Фи [ 25/Feb/21 ]

@ryan.egesdahl

About set back to 256: mongo (cli) says that on connect:

$ mongo
...
2021-02-23T16:33:51.391+03:00: Soft rlimits too low
2021-02-23T16:33:51.391+03:00:         currentValue: 256
2021-02-23T16:33:51.391+03:00:         recommendedMinimum: 64000

and my maxfiles: 

$ launchctl limit maxfiles
maxfiles    256            unlimited

Comment by Ryan Egesdahl (Inactive) [ 25/Feb/21 ]

brusheg@gmail.com Could you please clarify how you're seeing that the max files is being set back to 256? I have a feeling this is probably a soft limit being set by by launchctl. You can verify that by running launchctl limit maxfiles and checking the output. Depending on what is actually causing this, I may have a workaround for you.

Comment by Ал Фи [ 14/Feb/21 ]

My  kern.maxfiles:

$ sysctl -a | grep "kern.max"
kern.maxvnodes: 263168
kern.maxproc: 4176
kern.maxfiles: 49152
kern.maxfilesperproc: 24576
kern.maxprocperuid: 2784
kern.maxnbuf: 16384

Comment by Ал Фи [ 14/Feb/21 ]

OS: MacOS Catalina 10.15.7

Homebrew 3.0.1

Homebrew/homebrew-core (git revision 38fb41; last commit 2021-02-14)

Homebrew/homebrew-cask (git revision ca795; last commit 2021-02-14)

Official MongoDB Tap: https://github.com/mongodb/homebrew-brew

https://github.com/mongodb/homebrew-brew/blob/7f65ca10e34ff16bdc45c1cb45993f1b810e8d05/Formula/mongodb-community.rb#L66

https://github.com/mongodb/homebrew-brew/blob/7f65ca10e34ff16bdc45c1cb45993f1b810e8d05/Formula/mongodb-community.rb#L71

 

Generated at Thu Feb 08 05:33:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.