[SERVER-9997] balance is too slow Created: 24/Jun/13  Updated: 16/Dec/13  Resolved: 16/Dec/13

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: chensi Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

In our production enviroment, one chunk balance takes almost 300s. If we want to balance 100G data, it will take a very long time.
An available method is: when moveChunk, don't delete data at step6, finally we delete all the data one by one in slave using standonly instance.
But i'm not sure it's right, because data are not delete after balance.By the way, we only query by a given shardkey.

Thanks.



 Comments   
Comment by Stennie Steneker (Inactive) [ 16/Dec/13 ]

Hi,

As at MongoDB 2.4, the balancer no longer waits for the current migration's delete phase to complete before starting the next chunk migration. Please refer to Chunk Migration Queuing in the manual for more information. Another approach to speeding up the initial chunk balancing for an empty sharded collection is to pre-split the chunk ranges.

Regards,
Stephen

Comment by chensi [ 27/Jun/13 ]

actually, mongo really needs so much time to delete balanced data. I want to know whether balanced data can be stay in the source shard, and manully delete the data.

Comment by Eliot Horowitz (Inactive) [ 27/Jun/13 ]

Do you have hardware stats and logs?

Comment by chensi [ 25/Jun/13 ]

By the way, there are 900,00 records in our chunk, and Write querys come in as 50 req/s. Delete data in step6 is slow which we cannot accecpt.

Comment by chensi [ 25/Jun/13 ]

hi, we don't use mms;
mongos and mongod version are 2.2.4;
1.1T ssd 48G mem;

From config.changelog, most moveChunk cost 200s. After reading code, step6: delete data cost more than 100s.

Comment by Eliot Horowitz (Inactive) [ 24/Jun/13 ]

Would need a lot more info to give any advice.

  • host in mms?
  • mongo version?
  • mongod version?
  • hardware stats
Generated at Thu Feb 08 03:22:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.