[SERVER-30494] Running mongod server with empty mongod.lock file Created: 03/Aug/17  Updated: 27/Oct/23  Resolved: 21/Aug/17

Status: Closed
Project: Core Server
Component/s: Concurrency, Storage, WiredTiger
Affects Version/s: 3.2.10
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Marek Skalický Assignee: Mark Agarunov
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux based OS


Attachments: File mongod.log    
Operating System: ALL
Participants:

 Description   

I have running mongod 3.2 with --dbpath on NFS.

When I run mongod -f /etc/mongod.conf then mongod is running and accepting connections, but file `/var/lib/mongodb/data/mongod.lock` is empty.
So mongod -f /etc/mongod.conf --shutdown is not working for me (getting error There doesn't seem to be a server running with dbpath: /var/lib/mongodb/data)

How it is possible? I am trying to figure out how to reproduce it, but for now what could be reason of this odd behavior?

Content of /etc/mongod.conf

##
## For list of options visit:
## https://docs.mongodb.org/manual/reference/configuration-options/
##
 
# systemLog Options - How to do logging
systemLog:
  # Runs the mongod in a quiet mode that attempts to limit the amount of output
  quiet: true
 
 
# net Options - Network interfaces settings
net:
  # Specify port number (27017 by default)
  port: 27017
 
 
# storage Options - How and Where to store data
storage:
  # Directory for datafiles (defaults to /data/db/)
  dbPath: /var/lib/mongodb/data
 
 
# replication Options - Configures replication
replication:
  # Specifies a maximum size in megabytes for the replication operation log (i.e. the oplog,
  # 5% of disk space by default)
  oplogSizeMB: 64
 
storage.wiredTiger.engineConfig.cacheSizeGB: 1

File with server output is attached.



 Comments   
Comment by Mark Agarunov [ 21/Aug/17 ]

Hello mskalick,

Thank you for the additional information. I agree that this doesn't appear to be a bug in MongoDB itself, so I have closed the ticket. If new information comes to light about this issue, please let us know and we will reopen the ticket.

Thanks,
Mark

Comment by Marek Skalický [ 21/Aug/17 ]

Hi,
sorry for my long response time.

We get further with this issue. It seems that this is a Linux Kernel issue.

It appears when mandatory locking is set up on NFS server - then writes from a process are conflicting with a fcntl lock set by the same process.

This series of patches fixes this problem - https://www.spinics.net/lists/linux-nfs/msg60358.html . I think it is merged into kernel 4.10. We reproduced this with RHEL7.

Not sure if this should be mentioned somewhere in your documentation. This bug can be closed from my point of view as bug is not in MongoDB itself.

Comment by Mark Agarunov [ 15/Aug/17 ]

Hello mskalick,

Thank you for the report. Unfortunately there isn't anything in the log file to point to a cause for the empty lock file and I have not been able to reproduce the behavior. It is possible that the lock file was cleared if a new mongod process was started while another was stopping, but that is purely speculation. Is this behavior still present and an issue for you?

Thanks,
Mark

Generated at Thu Feb 08 04:24:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.