[DOCS-945] Mongo gotchas sharding size limitation Created: 03/Jan/13  Updated: 30/Oct/23  Resolved: 20/Aug/14

Status: Closed
Project: Documentation
Component/s: manual
Affects Version/s: mongodb-2.2
Fix Version/s: Server_Docs_20231030

Type: Bug Priority: Minor - P4
Reporter: Jeff lee Assignee: Sam Kleinman (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by DOCS-1133 Add max size of existing collection t... Closed
Related
is related to DOCS-761 Best Practices on Sharding Large Coll... Closed
Participants:
Days since reply: 9 years, 26 weeks ago

 Description   

Hi,

The current version of the gotchas page indicates that collections larger than 256 GB cannot be sharded but there's some information in some google group threads and older versions of the documentation that seem to contradict this.

For example, the older version of the sharding limits page and the code point to this being a limitation in the number of split points rather than a hard limit on the collection size. If that's the case, it would be helpful to update the gotchas page to reflect that along with the available workarounds if any.

Thanks.

Here are some relevant links:

https://jira.mongodb.org/browse/SERVER-2596

http://www.mongodb.org/display/DOCS/Gotchas

http://www.mongodb.org/pages/viewpage.action?pageId=33456197#ShardingLimits-Shardinganexistingcollection

https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/4axgn8JILRE

https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/dXmiXjj55KU



 Comments   
Comment by Sam Kleinman (Inactive) [ 20/Aug/14 ]

http://docs.mongodb.org/manual/reference/limits/#Sharding-Existing-Collection-Data-Size

Comment by Jeff lee [ 05/Jan/13 ]

I ran some tests on a 291 GB collection and it looks like the limitation has been removed in 2.x.

mongos> db.foodle.stats()
{
        "sharded" : false,
        "primary" : "s01",
        "ns" : "test.foodle",
        "count" : 61871066,
        "size" : 312820109696,
        "avgObjSize" : 5056,
        "storageSize" : 315621971712,
        "numExtents" : 168,
        "nindexes" : 1,
        "lastExtentSize" : 2146426864,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 0,
        "totalIndexSize" : 3189400368,
        "indexSizes" : {
                "_id_" : 3189400368
        },
        "ok" : 1
}
 
db.getSiblingDB('config').settings.find()
{ "_id" : "chunksize", "value" : 64 }
 
Fri Jan  4 23:42:38 [conn4] request split points lookup for chunk test.foodle { : MinKey } -->> { : MaxKey }
Fri Jan  4 23:43:36 [conn4] warning: Finding the split vector for test.foodle over { _id: 1.0 } keyCount: 6636 numSplits: 9322 lookedAt: 952 took 57257ms
Fri Jan  4 23:43:36 [conn4] command admin.$cmd command: { splitVector: "test.foodle", keyPattern: { _id: 1.0 }, min: { _id: MinKey }, max: { _id: MaxKey }, maxChunkSizeBytes: 67108864, maxSplitPoints: 0, maxChunkObjects: 0 } ntoreturn:1 keyUpdates:0 numYields: 60 locks(micros) r:114453134 reslen:259959 57262ms
 
mongos> db.printShardingStatus()
--- Sharding Status ---
  sharding version: { "_id" : 1, "version" : 3 }
  shards:
        {  "_id" : "s01",  "host" : "s01/localhost:10010" }
        {  "_id" : "s02",  "host" : "s02/localhost:10020" }
  databases:
        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
        {  "_id" : "test",  "partitioned" : true,  "primary" : "s01" }
                test.foodle chunks:
                                s02     4661
                                s01     4662

Generated at Thu Feb 08 07:39:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.