[SERVER-1181] Disk pre-allocation over estimates unreasonably - something is not kosher here Created: 02/Jun/10 Updated: 02/Jun/10 Resolved: 02/Jun/10 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 1.4.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | phpMoAdmin | Assignee: | Eliot Horowitz (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Participants: |
| Description |
|
Mongo is running with --nssize=2 --smallfiles I create a new database with 7 documents in total stored within 3 collections - total data size is 1.6kb (KILObytes) I look in /data/db and the database has already spawned a 32mb dbname.1 file - how could it think that it already filled the 16mb dbname.0 with so little data?? By this system storing 1.6kb of data takes 50mb of disk space (including ns file)! |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 02/Jun/10 ] |
|
Not sure we'll make that configurable. For smallest footprint, you should do --smallfiles --noprealloc |
| Comment by phpMoAdmin [ 02/Jun/10 ] |
|
Okay, I was not aware that their always is an empty file, so Mongo does indeed work as designed but perhaps the --smallfiles could have an option to define the starting file size? With the current setup a databases with a minimal amount of data will start at 50mb in size. If the initial file size could be reduced to 2mb and it preallocates an empty 4mb file then all the capacity problems would be resolved. |
| Comment by Eliot Horowitz (Inactive) [ 02/Jun/10 ] |
|
preallocation is supposed to always have a FULLY empty file ready. |
| Comment by phpMoAdmin [ 02/Jun/10 ] |
|
I don't know that every DB will be under 16mb as there are some that use GridFS (noprealloc is for all DBs right?) Regardless, why would it think it needs to pre-allocate another datafile when 15,998,400 bytes of 16mb are still free?? Perhaps something is miscalculating within Mongo internals? |
| Comment by Michael Dirolf [ 02/Jun/10 ] |
|
well if you know your databases will never be getting larger than 16mb then it's fine to not prealloc. otherwise, it might cause performance problems which is why we recommend that. you sort of have to decide for yourself how you want to make the tradeoff of extra disk space vs consistent performance. |
| Comment by phpMoAdmin [ 02/Jun/10 ] |
|
I saw the --noprealloc option, but your docs say not to do that on a production server and this is a production server for a cloud-based service powering a few hundred sites (each with its own database, thus cumulatively the pre-allocation is becoming a problem) |
| Comment by Michael Dirolf [ 02/Jun/10 ] |
|
that is probably preallocation. you could try turning that off if you really need to conserve space: --noprealloc |