[SERVER-2246] Pay (more) attention to sharding maxSize Created: 17/Dec/10  Updated: 11/Apr/23  Resolved: 11/Apr/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: David Mytton Assignee: Tommaso Tocci
Resolution: Done Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-11441 parameter maxSize is either useless o... Closed
is duplicated by SERVER-21411 allow limits on the amount of data st... Closed
is duplicated by SERVER-2473 Avoid "no space left" on a shard Closed
Related
related to SERVER-66297 Get rid of `maxSize` for shards Closed
Assigned Teams:
Sharding EMEA
Sprint: Sharding EMEA 2023-05-01
Participants:

 Description   

MongoDB should pay more attention to the sharding maxSize config option so that when total storage hits that value, no further data is added to that shard. Currently it's more of a soft limit as per http://groups.google.com/group/mongodb-user/browse_thread/thread/c35fe5922af94211/04cecf334defa9c0



 Comments   
Comment by Tommaso Tocci [ 11/Apr/23 ]

maxSize configuration option has been removed in SERVER-66297

Comment by Asya Kamsky [ 27/Jul/14 ]

MaxSize already controls the balancer not moving any chunks to the shard that's reached its maxSize.

I'm trying to ascertain whether this ticket could be addressed by enhancing the balancer to move chunks from a shard which is over its maxSize by some amount (otherwise you'd have chunks going back and forth).

This ticket requests that

when total storage hits that value, no further data is added to that shard

I'd like to get some thoughts as to whether solving this via balancer moving chunks from the shard but not changing any insert/update/etc behavior would be sufficient as it may be simpler to implement and be less drastic of a change.

I do want to point out that tagging shards and chunk ranges could be used as a temporary workaround to influence the balancer behavior, but of course that would be temporary as the sizes would have to be monitored and tag ranges kept up to date with current sizes.

Comment by Kay Agahd [ 27/Jul/14 ]

> the only way to not insert a new document into shard that reached maxSize would be for the insert to fail.

No. Why not just move the chunk to another shard and tell the balancer that the shard should not receive new chunks while it's overloaded?

Comment by Asya Kamsky [ 27/Jul/14 ]

Since documents must go to shards that hold appropriate chunk range, the only way to not insert a new document into shard that reached maxSize would be for the insert to fail.

That seems like a poor user experience (plus all operations that increase storage would have to do the same).

Maybe renaming or aliasing the maxSize parameter would be a better option?

Generated at Thu Feb 08 02:59:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.