[SERVER-1636] Allow configurable min datafile size Created: 16/Aug/10 Updated: 25/Jun/15 Resolved: 24/Jun/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Performance |
| Affects Version/s: | 1.6.0 |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Minor - P4 |
| Reporter: | David Burley | Assignee: | Unassigned |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
*NIX |
||
| Participants: |
| Description |
|
Datafiles are preallocated on *NIX platforms for optimal performance. However, in some odd use cases where it is desirable to have a lot of databases that are smaller in size interspersed with a few larger DBs, this causes the disk to grow excessively. Given some of the DBs will be split out into multiple 2GB datafiles while the majority will be in 64MB (with 128MB sitting around empty, for perhaps a few kB of data in most), this is a lot of wasted disk space. --noprealloc will impair performance of the larger DBs. Sharding is an option but a waste of hardware. It would be really nice to be able to pick a smaller starting datafile size. I would think it to be sane to lock it down to a few potential options (starting at 1MB would be great, increasing to 2, 4, 8, ... MB). That would create a lot more data files but modern filesystems are OK with that. I consider this ticket to be a "metoo" of one of the latter comments in: http://jira.mongodb.org/browse/SERVER-1181 |
| Comments |
| Comment by Ian Whalen (Inactive) [ 24/Jun/15 ] |
|
Hi David, thanks a lot for filing this feature request and apologies for the time since it was last updated. We're closing this as "Won't Fix" given the previous conversation about the proliferation of options and this being a fairly edge-case request. |
| Comment by David Burley [ 06/Jul/11 ] |
|
We have since modified our usage pattern, given there are no 'metoo' comments – I'd just close this one out since we were definitely a special edge-case. |
| Comment by David Burley [ 16/Aug/10 ] |
|
Allow --smallfiles to take a value for the min datafile size, and get rid of it making the max datafile smaller than 2GBs – at least that's my gut reaction to not adding another option and extending an existing one to cover this and other cases. Unless there is a reason why it reduces the max size – then I'll think about this some more and make another proposal. |
| Comment by Eliot Horowitz (Inactive) [ 16/Aug/10 ] |
|
Our big issue is not writing the code - but proliferation of options. So rather than working on code - the best thing to do would be propose some changes to the options that keep the total # sane while being flexible enough. |
| Comment by David Burley [ 16/Aug/10 ] |
|
I have looked at smallfiles – but its still not small enough for the long run. Also, if I am reading the code right – it looks like if you have smallfiles enabled, the max data file size is reduced – and that's not a good thing since I do want bigger DBs to be put into progressively bigger data files (I assume that's more efficient). I have poked at the code a bit to see what needs changed to add such a feature. I suspect you know better than I what would need to be touched. If you could provide some pointers to make sure I don't overlook something (I am not very familiar with the internals) I could make a first attempt at this later on this week. It looks like I'd need to change something related to: extents (e.g. like): http://github.com/mongodb/mongo/commit/a1f8c3698397b9fa395578e901250eea7b312060 And then tweak the MongoDataFile::defaultSize function And add in the command line option Am I missing anything – and if I wrote it – any change it would get uptake? I know that there is a max document size of 4MB – is there any risk of having the min data size <= 4MB? |
| Comment by Eliot Horowitz (Inactive) [ 16/Aug/10 ] |
|
Have you looked at --smallfiles? |