[SERVER-1810] ability to set minimum allocation size per collection Created: 19/Sep/10  Updated: 08/Feb/23  Resolved: 14/Sep/18

Status: Closed
Project: Core Server
Component/s: MMAPv1, Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Eliot Horowitz (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 51
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-9506 initial or new document size flag to ... Closed
is duplicated by SERVER-3563 add a setting to specify minimum docu... Closed
is duplicated by SERVER-8775 paddingFactor implementation causes 1... Closed
Related
related to SERVER-11946 colMod should expose padding factor, ... Closed
related to SERVER-12017 Add the capability to manually set a ... Closed
is related to SERVER-3752 padding factor overcompaction and rel... Closed
is related to SERVER-12224 Allow an explicit padding (or total s... Closed
is related to SERVER-9933 add the ability to set a collection's... Closed
Assigned Teams:
Storage Execution
Participants:
Case:

 Description   

With the ability to set a minimum allocation size, and power of 2 allocation, you have very good control over allocation, will greatly reduce the number of moves, and have very little to no fragmentation.



 Comments   
Comment by Asya Kamsky [ 07/Mar/15 ]

Note that specifying initial size would not have helped in this case:

at some point they had to add a field or do a very small model change, as a result mongo had to move every document on disk

since they would have had to know in advance that they will need to add specific size field(s) in the future when the collection was created.

Comment by Paul Reinheimer [ 19/May/13 ]

With https://jira.mongodb.org/browse/SERVER-8775?focusedCommentId=338871#comment-338871 this seems even more important.

I have documents that change size radically between initial insert, and final update (~20 seconds later). I'd love to give mongo guidance on how big that document is going to get.

Specifying an initial size seems better to me. With documents that start quite small the padding factor could swing radically based on small changes there.

Comment by Eliot Horowitz (Inactive) [ 03/Feb/12 ]

Still not 100% sure of the right design.

Might be better to have an initial size than padding factor.

Want to have the right option - not too many.

Comment by Antoine Girbal [ 19/Jan/12 ]

note about a recent problematic case:

  • user had many documents that were only ever inserted or updated statically
  • as a result paddingfactor was still at 1
  • at some point they had to add a field or do a very small model change, as a result mongo had to move every document on disk
  • since they have a dozen indexes to update it resulted in massive disk IO and brought db down. Also most of the slots on disk were not reusable.

User was surprised that such a small change would kill db.
It would make sense to have a way to set the minimum padding for a collection.
It could be set to 10% so that by default there is a bit of space for doc to grow.
The tradeoff is a bit more storage and some ram waste but better in general case.
If a user is sure never to change doc size, storage can be optimized by setting to 1.

Comment by Dwight Merriman [ 03/Sep/11 ]

i wonder if a knob is needed, or rather if it just needs be more intelligent.

it may be needed – a bulk import scenario comes to mind.

Comment by Liu Qishuai [ 17/Jun/11 ]

it's better to have three values for default/min/max. the default was 1.0/1.0/2.0

Comment by ganesan pandurangan [ 16/Jun/11 ]

will this factor override the default behavior ?

Lets say I configure padding factor for a collection as 1.5 and in reality the new documents are larger than expected, will mongo still be able to adapt and set the padding factor as say 2.0 and discard what user has set ?

Comment by Kenny Gorman [ 11/Apr/11 ]

+1 on this being configured in a simple way per collection.

Comment by Eliot Horowitz (Inactive) [ 21/Sep/10 ]

Correct - per collection.

Comment by Thilo Planz [ 19/Sep/10 ]

This would be per collection, right?

And ideally it could be also changed for collections that already have data (then affecting only newly added/relocated records).

Generated at Thu Feb 08 02:58:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.