[SERVER-11904] Preallocated files are sparse in FreeBSD/UFS Created: 30/Nov/13  Updated: 11/Jul/16  Resolved: 27/Dec/13

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 2.5.5

Type: Bug Priority: Major - P3
Reporter: Taha Jahangir Assignee: Davide Italiano
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

FreeBSD 9.1/9.2 amd64, mongodb 2.4.8


Operating System: FreeBSD
Steps To Reproduce:

Reproducible with clean installation of FreeBSD + mongo

Participants:

 Description   

Title is clear, allocating 2GB files takes no time / no space on disk, and thus disk fragmentation can occur.

# mkdir /tmp/x
# mongod --dbpath /tmp/x --port 27111 --replSet rs1 --oplogSize 10000
# mongo --port 27111
> rs.initiate()

Sat Nov 30 08:02:11.712 [FileAllocator] allocating new datafile x/local.1, filling with zeroes...
Sat Nov 30 08:02:11.796 [FileAllocator] done allocating datafile x/local.1, size: 2047MB,  took 0.084 secs
Sat Nov 30 08:02:11.797 [FileAllocator] allocating new datafile x/local.2, filling with zeroes...
Sat Nov 30 08:02:11.801 [FileAllocator] done allocating datafile x/local.2, size: 2047MB,  took 0.004 secs
Sat Nov 30 08:02:11.801 [FileAllocator] allocating new datafile x/local.3, filling with zeroes...
Sat Nov 30 08:02:11.805 [FileAllocator] done allocating datafile x/local.3, size: 2047MB,  took 0.003 secs
Sat Nov 30 08:02:11.805 [FileAllocator] allocating new datafile x/local.4, filling with zeroes...
Sat Nov 30 08:02:11.810 [FileAllocator] done allocating datafile x/local.4, size: 2047MB,  took 0.005 secs
Sat Nov 30 08:02:11.811 [FileAllocator] allocating new datafile x/local.5, filling with zeroes...
Sat Nov 30 08:02:11.815 [FileAllocator] done allocating datafile x/local.5, size: 2047MB,  took 0.004 secs

# ll -h
total 1032
drwxr-xr-x  2 root  wheel   512B Nov 30 08:02 _tmp/
drwxr-xr-x  2 root  wheel   512B Nov 30 08:04 journal/
-rw-------  1 root  wheel    64M Nov 30 08:02 local.0
-rw-------  1 root  wheel     2G Nov 30 08:02 local.1
-rw-------  1 root  wheel     2G Nov 30 08:02 local.2
-rw-------  1 root  wheel     2G Nov 30 08:02 local.3
-rw-------  1 root  wheel     2G Nov 30 08:02 local.4
-rw-------  1 root  wheel     2G Nov 30 08:02 local.5
-rw-------  1 root  wheel    16M Nov 30 08:02 local.ns
-rwxr-xr-x  1 root  wheel     0B Nov 30 08:04 mongod.lock*
# du -sh
1.0M    .



 Comments   
Comment by Davide Italiano [ 23/Dec/13 ]

A patch has been just committed that should fix this problem. This will be available in 2.6 (but can be tested with the current 2.5.x nightly)

Comment by Githook User [ 23/Dec/13 ]

Author:

{u'username': u'dcci', u'name': u'Davide Italiano', u'email': u'davide.italiano@10gen.com'}

Message: SERVER-11904 Preallocated files are sparse in FreeBSD/UFS.
Do not always assume FreeBSD is running on ZFS and zero-fill the file
at allocation time. This matches the behaviour of mongod on Linux for
non copy-on-write filesystem, e.g. UFS, and helps preventing disk
fragmentation.
Branch: master
https://github.com/mongodb/mongo/commit/1a30d06d98370f7804c99b18b6ae5bc7503f25dd

Comment by Taha Jahangir [ 02/Dec/13 ]

No, I'm not a FreeBSD guru, but I think the files should not be sparse unless explicitly specified in creation time.

And I don't what is actually called by mongodb to preallocate files. If we have a simple c file with few lines of code, that simulates the file preallocator process, we could debug the FreeBSD part.

And I noticed that in file_allocator.cpp, there is a useSparseFiles method that returns true in FreeBSD!, but in FileAllocator::ensureLength it seems the that return value of useSparseFiles is false (because not log about sparse file is printed)

Comment by Eliot Horowitz (Inactive) [ 02/Dec/13 ]

Do you have a link to how their sparse is implemented?
I seem unable to find it easily.

Generated at Thu Feb 08 03:27:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.