[SERVER-2271] Increase initial sharding limit to 256gb Created: 22/Dec/10  Updated: 12/Jul/16  Resolved: 20/Feb/11

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 1.8.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Richard Kreuter (Inactive) Assignee: Kristina Chodorow (Inactive)
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-2596 handle enabling sharding for any size... Closed
is related to DOCS-1265 Maximum collection size to shard is 5... Closed
Participants:

 Description   

There's an arbitrary limit on the size of a pre-existing collection that we're split into chunks. Alberto tells me this limit should be revisited, could possibly be removed altogether. Mostly just creating this issue to allow watchers to track progress.



 Comments   
Comment by Asya Kamsky [ 19/Mar/13 ]

The code shows 512GB limit for 64MB default chunk size (max split points 8192) but the docs say 256GB still, opened DOCS-1265 to fix.

Comment by Kristina Chodorow (Inactive) [ 06/Nov/12 ]

Just the data size.

Comment by Colin Howe [ 06/Nov/12 ]

Is this a limit on the data size or the combined data/index size?

Comment by Eliot Horowitz (Inactive) [ 21/Feb/11 ]

@andrew, its on this page: http://www.mongodb.org/display/DOCS/Sharding+Limits

Comment by Andrew Armstrong [ 21/Feb/11 ]

I noticed on http://www.mongodb.org/display/DOCS/Sharding+FAQ that there is no mention of the limits to being able to go from an unsharded->sharded collection.

Having read the docs a while ago I would have thought I was safe being unsharded until my data size grew too big; but if there is a limit to where I can transition to a sharded collection that may be worth noting in the docs?

Cheers

Comment by Eliot Horowitz (Inactive) [ 20/Feb/11 ]

Note, one way around the limit is increasing chunk size temporarily.

Comment by auto [ 18/Feb/11 ]

Author:

{u'login': u'kchodorow', u'name': u'Kristina Chodorow', u'email': u'kristina@10gen.com'}

Message: increase number of initial chunk splits allowed to fit 256GB of data SERVER-2271
https://github.com/mongodb/mongo/commit/a9e3511eda147d42f79203ad6e181e789c632c49

Comment by Eliot Horowitz (Inactive) [ 18/Feb/11 ]

Try making it 16x larger, and spin up a collection with 200gb, the shard and see what happens.
If all looks good, go ahead and commit

Comment by Kristina Chodorow (Inactive) [ 18/Feb/11 ]

The constant should be upped for 1.8.0. It's ridiculously low (you can't chunk a collection larger than 16GBs) and shardCollection will look through the entire index regardless, which is a huge waste of resources.

Generated at Thu Feb 08 02:59:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.