[SERVER-13143] Ignore db names that begin with a dot, when directory-per-db is active Created: 11/Mar/14  Updated: 06/Dec/22  Resolved: 05/Oct/18

Status: Closed
Project: Core Server
Component/s: MMAPv1, Storage
Affects Version/s: 2.4.9
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: nifan Assignee: Backlog - Storage Execution Team
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Assigned Teams:
Storage Execution
Operating System: ALL
Steps To Reproduce:

Install mongodb on xfs filesystem and run xfs_fsr.

Participants:

 Description   

When using mongodb directly on XFS filesystem it automatically shuts down when the XFS defragmentation tool 'xfs_fsr' runs.

This creates a .fsr directory on the mounted filesystem, which causes MongoDB to exit.

The essence of this problem is the same for all other dot directories inside the mongodb map.

One can discuss in length about if it's "proper" to do it this or the other way. But in the end we all know that directories like "lost+found", ".fsr" and really any dot directory in the root of the filesystem can exist and have nothing to do with MongoDB. (Also consider .snapshots etc)

So it makes much more sense it MongoDB would ignore it.

A user tunable parameter that allows to exclude additional file paths would be helpful as well.

This should be a easy-picking patch for people known with MongoDB and very safe to implement. Very little overhead and high usefulness factor.



 Comments   
Comment by Sara Williamson [ 05/Oct/18 ]

This went away because only MMAP is affected by this issue, and it has been removed.

Comment by David Robillard [ 17/Apr/14 ]

I agree, the lost+found directory should really be ignored by MongoD.

Comment by Andy Schwerin [ 14/Mar/14 ]

Database names cannot contain dot, anyways, so this should be a safe change.

Comment by nifan [ 14/Mar/14 ]

Yes, sorry I forgot to mention that. I am using directory-per-db.

Could a solution be to do that via a feature toggle ?
The default could be 'off' for several versions to create a smooth path forward. (Or even keep it off-per-default forever if there is a strong enough general case in which databases starting with a dot are useful)

Are database names starting with a dot not already invalid ?

$ mongo
MongoDB shell version: 2.4.9
connecting to: test
> use .test
Fri Mar 14 15:04:36.452 Error: [.test] is not a valid database name at src/mongo/shell/mongo.js:40

Comment by Eric Milkie [ 14/Mar/14 ]

I believe this problem only affects you if you are using directory-per-db. Is that correct?

Currently, mongod treats every subdirectory found in the dbpath as a database, when you start it with --directoryperdb. A solution for this ticket could be that mongod ignores database names that begin with a dot, but this would be backwards breaking for those users who already have such database names.

Comment by nifan [ 13/Mar/14 ]

Tue Mar 11 23:01:00.037 [conn366946] ERROR: Uncaught std::exception: boost::filesystem::status: Permission denied: "/var/lib/mongodb/.fsr/.fsr.ns", terminating
Tue Mar 11 23:01:00.037 dbexit:
Tue Mar 11 23:01:00.038 [conn366946] shutdown: going to close listening sockets...
Tue Mar 11 23:01:00.043 [conn366946] closing listening socket: 57
Tue Mar 11 23:01:00.043 [conn366946] closing listening socket: 58
Tue Mar 11 23:01:00.043 [conn366946] closing listening socket: 59
Tue Mar 11 23:01:00.043 [conn366946] removing socket file: /tmp/mongodb-27017.sock
Tue Mar 11 23:01:00.043 [conn366946] shutdown: going to flush diaglog...
Tue Mar 11 23:01:00.043 [conn366946] shutdown: going to close sockets...
Tue Mar 11 23:01:00.043 [conn366946] shutdown: waiting for fs preallocator...
Tue Mar 11 23:01:00.043 [conn366946] shutdown: lock for final commit...
Tue Mar 11 23:01:00.044 [conn361966] end connection 10.xxx.xxx.xxx:41831 (7 connections now open)
Tue Mar 11 23:01:00.044 [conn362141] end connection 10.xxx.xxx.xxx:42547 (7 connections now open)
Tue Mar 11 23:01:00.044 [conn368447] end connection 10.xxx.xxx.xxx:42853 (6 connections now open)
Tue Mar 11 23:01:00.044 [conn368445] end connection 10.xxx.xxx.xxx:49541 (6 connections now open)
Tue Mar 11 23:01:00.145 [conn368012] insert xxxxxxxx_meta.xxxxxxxxx_meta ninserted:1 keyUpdates:0 locks(micros) w:263016 131ms
Tue Mar 11 23:01:00.145 [initandlisten] now exiting
Tue Mar 11 23:01:00.145 dbexit: ; exiting immediately
 
 
***** SERVER RESTARTED *****
 
 
Tue Mar 11 23:16:33.616 [initandlisten] MongoDB starting : pid=7222 port=27017 dbpath=/var/lib/mongodb 64-bit host=db1.mp01.net
Tue Mar 11 23:16:33.619 [initandlisten] db version v2.4.9
Tue Mar 11 23:16:33.619 [initandlisten] git version: 52fe0d21959e32a5bdbecdc62057db386e4e029c
Tue Mar 11 23:16:33.619 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST
 2009 x86_64 BOOST_LIB_VERSION=1_49
Tue Mar 11 23:16:33.619 [initandlisten] allocator: tcmalloc
Tue Mar 11 23:16:33.619 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", directoryperdb: "true
", logappend: "true", logpath: "/var/log/mongodb/mongodb.log", replSet: "xxxxxx" }
Tue Mar 11 23:16:33.620 [initandlisten] journal dir=/var/lib/mongodb/journal
Tue Mar 11 23:16:33.620 [initandlisten] recover begin
Tue Mar 11 23:16:33.620 [initandlisten] recover lsn: 1317435139
Tue Mar 11 23:16:33.620 [initandlisten] recover /var/lib/mongodb/journal/j._707
Tue Mar 11 23:16:33.629 [initandlisten] recover skipping application of section seq:1317256418 < lsn:1317435139
Tue Mar 11 23:16:33.781 [initandlisten] recover skipping application of section seq:1317315988 < lsn:1317435139
Tue Mar 11 23:16:35.490 [initandlisten] recover /var/lib/mongodb/journal/j._708
Tue Mar 11 23:16:35.646 [initandlisten] recover skipping application of section seq:1317375583 < lsn:1317435139
Tue Mar 11 23:16:36.980 [initandlisten] recover cleaning up
Tue Mar 11 23:16:36.980 [initandlisten] removeJournalFiles
Tue Mar 11 23:16:36.994 [initandlisten] recover done
Tue Mar 11 23:16:37.008 [initandlisten] exception in initAndListen std::exception: boost::filesystem::status: Permission denied: "/var/lib/mongodb/.fsr/.fsr.ns", terminating
Tue Mar 11 23:16:37.008 dbexit:
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: going to close listening sockets...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: going to flush diaglog...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: going to close sockets...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: waiting for fs preallocator...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: lock for final commit...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: final commit...
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: closing all files...
Tue Mar 11 23:16:37.008 [initandlisten] closeAllFiles() finished
Tue Mar 11 23:16:37.008 [initandlisten] journalCleanup...
Tue Mar 11 23:16:37.008 [initandlisten] removeJournalFiles
Tue Mar 11 23:16:37.008 [initandlisten] shutdown: removing fs lock...
Tue Mar 11 23:16:37.008 dbexit: really exiting now
 
 
***** SERVER RESTARTED *****

Comment by Daniel Pasette (Inactive) [ 12/Mar/14 ]

can you post the log file of the mongod which shutdown. the existence of the .fsr file in mongod root dir should not cause it to shut down on its own.

Generated at Thu Feb 08 03:30:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.