[SERVER-38232] Mongod on Ubuntu 16/18 does not create a pid file Created: 22/Nov/18 Updated: 29/Oct/23 Resolved: 10/Feb/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | 3.4.15, 3.4.18 |
| Fix Version/s: | 3.4 Required, 4.4.19, 5.0.15, 7.0.0-rc0, 6.0.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | James Stocker | Assignee: | Ryan Egesdahl (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Assigned Teams: |
Server Development Platform
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Requested: |
v6.0, v5.0, v4.4
|
||||||||||||||||||||
| Steps To Reproduce: | Install a server with Ubuntu 16 or 18 and install mongodb-org-server 3.4.x from the repo.mongodb.org repos, start it up and it won't create a pid file where it is supposed to according to the systemd config file. |
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||
| Story Points: | 2 | ||||||||||||||||||||
| Description |
|
I was alerted to missing log files on some of our production mongod servers yesterday, the cause was that log rotate was trying to use the pid file to send a signal to mongo to tell it the log files had rotated. But the pid file wasn't where we expected to find it. Looking in the systemd service file it should be in /var/run/mongodb/mongod.pid but nothing appeared there, even after I created the folder which was missing. Then I added this to my mongod config file:
Then mongod failed to start up, as it didn't have permission to create. I touched the file and gave mongo permissions and it started, which should fix our issues. I would say that this should not be fixed by admins, but should be correctly managed by mongod's systemd service file. I would have expected a warning or error to appear in the logs in this situation, but nothing appears:
My server here is running 3.4.15 but I tested with 3.14.18 from repo.mongodb.org on my laptop and saw the same issue |
| Comments |
| Comment by Githook User [ 10/Feb/23 ] | |||||
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: (cherry picked from commit 1e5c67d6b9c80eda789fece2e1973a7f7c6cb7b0) | |||||
| Comment by Githook User [ 10/Feb/23 ] | |||||
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: (cherry picked from commit 1e5c67d6b9c80eda789fece2e1973a7f7c6cb7b0) | |||||
| Comment by Githook User [ 10/Feb/23 ] | |||||
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: (cherry picked from commit 1e5c67d6b9c80eda789fece2e1973a7f7c6cb7b0) | |||||
| Comment by Githook User [ 10/Feb/23 ] | |||||
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: | |||||
| Comment by Ryan Egesdahl (Inactive) [ 08/Feb/23 ] | |||||
|
We're going to backport this change along with a partial backport of Note: We are only going to backport to v6.0, v5.0, and v4.4. We have decided against backporting to v4.2 because it's going EOL in a couple of months and we think it's better to not change expectations too much unless it's for a critically important reason. | |||||
| Comment by Ryan Egesdahl (Inactive) [ 03/Feb/23 ] | |||||
|
We have moved completely over to systemd to manage the service, which makes a PID file superfluous and potentially inaccurate if not configured correctly. It's better to simply ask systemd for PID and service status information:
If you absolutely need a PID file for some reason, you can still add the configuration option in mongod.conf to write one (make sure you don't also have fork!) and use systemctl edit mongod.service to add a RuntimeDirectory=mongodb line to make sure the /run directory is created for it. This is not likely to be a common use case, but we can certainly add RuntimeDirectory=mongodb to the default unit file to support it without causing any problems. | |||||
| Comment by James Stocker [ 29/Nov/18 ] | |||||
|
In 14.04, you didn't need to specify this value, yet the pid file was created, which is not what the documentation suggests should happen. In 16.04 and 18.04, the pid file is not created automatically without this being set, like the docs suggest. However, even if you set this value, the pid file cannot be created in /var/run because mongodb doesn't have permissions to create that file, you need to create a tmpfile config file yourself otherwise mongodb will fail to start as it cannot create it's pid file | |||||
| Comment by Eric Sedor [ 28/Nov/18 ] | |||||
|
Thanks a lot James; we've been able to confirm that a pidFilePath value that matches the service config is missing from the debian mongod.conf file in 3.4.18. We think that is the problem, and have marked this issue for scheduling. You can keep checking this ticket for additional updates. | |||||
| Comment by James Stocker [ 28/Nov/18 ] | |||||
|
Yes those logs are before I defined it in the config file.
But in the systemd.service file installed by the deb, it is defined:
This is a different location than the old default of /var/run/mongodb.pid Regardless of it being in the systemd.service file the PID is not created. When we used this on 14.04 we didn't need to define the pid file location in the mongo config, it was created automatically. | |||||
| Comment by Eric Sedor [ 28/Nov/18 ] | |||||
|
Hi James, We're not seeing the pidFilePath option showing up in the parsed options log lines:
Can you please confirm that /etc/mongod.conf contains the option, or let us know if these log lines are from before the pidFilePath option is added? Thank you! | |||||
| Comment by James Stocker [ 27/Nov/18 ] | |||||
|
I realised later than /var/run is on a tmpfs, so I had to add this:
This file tells the os to create the pid file and assign ownership to mongodb |