[SERVER-7991] default chunkSize of 64MB is not taken care Created: 20/Dec/12  Updated: 15/Feb/13  Resolved: 21/Dec/12

Status: Closed
Project: Core Server
Component/s: Diagnostics, Internal Code, Sharding
Affects Version/s: 2.2.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Santosh Kumar L Assignee: Stennie Steneker (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, Ubuntu


Operating System: ALL
Participants:

 Description   

MongoDB documentation says that default chunksize is 64 MB.
We have setup MongoDB cluster with default chunksize and loaded 350 GB of data. This setup is on linux machines
Cluster is of 6 shards with a replication factor of 3.
After the load is completed, we have checked the sharding stats with the db.<collection>.getShardDistribution.
shard-3 has 30% data with chunksize of 162GB.
Remaining shards have 10-14% data each with a chunksize varying from 64-96GB.
Can you please let me know is this a bug in MongoDB version 2.2.1?
If not, can you please let me know why this behavior of different chunksize in the same cluster???

regards,
Santosh



 Comments   
Comment by Eliot Horowitz (Inactive) [ 21/Dec/12 ]

This is caused by: SERVER-7960, so resolving as dupe.

Comment by Santosh Kumar L [ 21/Dec/12 ]

Stephen,

I cannot get the output for the command sh.status() as we have shutdown the cluster and removed the data for setting up a fresh cluster.

regards,
Santosh

Comment by Santosh Kumar L [ 21/Dec/12 ]

Posting the details of getShardDistribution() output below.

We have setup a 6 Shard MongoDB cluster with a replication factor of 3.
When starting the router process, default chunk size and oplog size was chosen by not specifying the values for these explicitly.

Shard3 has a chunk size of 161 MB while the rest have 60-90 MB per chunk.
All shards are similar type of instances on Amazon EC2 environment.
What we have noticed using db.<collection>.getShardDistribution() command is as follows:

Shard shard1 at shard1/<ips of shard1>
data : 38.8Gb docs : 43049426 chunks : 621
estimated data per chunk : 63.99Mb
estimated docs per chunk : 69322

Shard shard2 at shard2/<ips of shard2>
data : 40.24Gb docs : 44644092 chunks : 620
estimated data per chunk : 66.47Mb
estimated docs per chunk : 72006

Shard shard3 at shard3/<ips of shard3>
data : 102.65Gb docs : 113874252 chunks : 649
estimated data per chunk : 161.97Mb
estimated docs per chunk : 175461

Shard shard4 at shard4/<ips of shard4>
data : 54.51Gb docs : 60472368 chunks : 620
estimated data per chunk : 90.04Mb
estimated docs per chunk : 97536

Shard shard5 at shard5/<ips of shard5>
data : 50.48Gb docs : 56005174 chunks : 620
estimated data per chunk : 83.38Mb
estimated docs per chunk : 90330

Shard shard6 at shard6/<ips of shard6>
data : 46.32Gb docs : 51388397 chunks : 620
estimated data per chunk : 76.51Mb
estimated docs per chunk : 82884

Totals
data : 333.05Gb docs : 369433709 chunks : 3750
Shard shard1 contains 11.65% data, 11.65% docs in cluster, avg obj size on shard : 967b
Shard shard2 contains 12.08% data, 12.08% docs in cluster, avg obj size on shard : 967b
Shard shard3 contains 30.82% data, 30.82% docs in cluster, avg obj size on shard : 967b
Shard shard4 contains 16.36% data, 16.36% docs in cluster, avg obj size on shard : 967b
Shard shard5 contains 15.15% data, 15.15% docs in cluster, avg obj size on shard : 967b
Shard shard6 contains 13.91% data, 13.91% docs in cluster, avg obj size on shard : 967b

Comment by Santosh Kumar L [ 21/Dec/12 ]

Hi Stephen,

I have raised a bug for this 3 days ago. Please check the following link for the details.

https://jira.mongodb.org/browse/SERVER-7960

regards,
Santosh

Comment by Stennie Steneker (Inactive) [ 21/Dec/12 ]

Hi Santosh,

Can you please post the output of getShardDistribution() for the collection in question as well as sh.status()?

Thanks,
Stephen

Generated at Thu Feb 08 03:16:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.