[SERVER-8600] improve handling of non-fsyncable directories Created: 16/Feb/13  Updated: 27/Oct/15  Resolved: 27/Sep/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 2.5.3

Type: Improvement Priority: Major - P3
Reporter: Eric Milkie Assignee: Matt Dannenberg
Resolution: Done Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on DOCS-1376 Create page listing un-supported file... Closed
Related
related to SERVER-27640 Need to fsync directories on linux Closed
is related to SERVER-8521 Lazily clean up temp collections Closed
is related to SERVER-6739 Add support for panasas filesystem Closed
Participants:

 Description   

If you attempt to use MongoDB with e.g. an HGFS mount as the data directory, you may see an fassert on startup:

Sat Feb 16 07:04:41.098 [initandlisten] Assertion: 13651:Couldn't fsync directory '/mnt/hgfs/data/db/bug': errno:22 Invalid argument

This typically indicates the use of a device that does not support fsyncing on directories. We should handle this error (EINVAL) better:

  • either not worry about fsyncing the directory and give a warning
  • or still abort, but give suggestions as to how to correct the issue


 Comments   
Comment by auto [ 27/Sep/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-8600 log warning for non-fsyncable filesystems
Branch: master
https://github.com/mongodb/mongo/commit/5356c87e34275379e0a9de2ff441055d0da0bf5f

Comment by Eric Milkie [ 20/Sep/13 ]

VirtualBox shared folders may be another unsupported filesystem.

Comment by Eric Milkie [ 25/Mar/13 ]

Refined plan:
We will explicitly warn, once, and log to startupWarningsLog, a message saying that this filesystem is not supported, and to see a docs page for details.
This docs page will list the filesystems that are known not to support fsync on a directory (HGFS is one), and to notify 10gen if an unlisted filesystem generates the warning.

Comment by Andrew Emil (Inactive) [ 22/Mar/13 ]

I looked back over the locations I mentioned: As far as I can tell util/paths.h is the only place that we fsync a directory, so it seems you are right that we may only need to log a message in a single place. I will update this ticket after I do some testing with HGFS.

Comment by Eric Milkie [ 21/Mar/13 ]

I think we only need to concern ourselves with fsyncing directories. This ticket does not concern fsyncing of files. Some of the code locations in your list are only for the fsyncing of files.

We need to confirm that the fsyncing of a directory is not required for correct server operation (I believe it is).
If so, we may only need to log 1 info message if we find it doesn't work.

Comment by Andrew Emil (Inactive) [ 21/Mar/13 ]

Looked over this ticket: there are only a few places that we really call fsync (or fdatasync), so it seems very possible to add in checks in those places, particularly if we still want to abort but also give suggestions about correcting the issue. Not sure what the warning or info message should be, but it seems like the code will look like:

if(fsync(fd) != 0) {
    int e = errno;
    if(e == EINVAL) { //indicates file does not support syncronization
        log() << “SomeMessage” << endl;
    }
}

My question is what the message should be here. For example, this could be added in to path.h, where the assertion referenced in the ticket description comes from, but it does not seem useful:

log() << “Directory ‘“ << dir.string() << “‘ is not fsyncable, is it a temporary directory?” << endl;

Places where we call fsync or fdatasync:
util/paths.h +117
util/file.cpp +201
util/logfile.cpp -a few locations
db/flushtest.cpp -a few locations
db/instance.cpp +1242

Generated at Thu Feb 08 03:17:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.