[SERVER-14323] Balancer hits config servers hard even though there are no writes Created: 20/Jun/14 Updated: 20/Jul/16 Resolved: 20/Jul/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.6.3, 3.0.12, 3.2.8, 3.3.8 |
| Fix Version/s: | 3.3.9 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Jan Jeek | Assignee: | Kaloian Manassiev |
| Resolution: | Done | Votes: | 2 |
| Labels: | balancer | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Steps To Reproduce: | In my case, I have a database in 2 shards with slightly less than 30M documents in about 130k chunks. I have 6 mongos instances and, each reading from the first config server at about 20 mbit/s which is 120mbit/s in total. I have to control the balancer manually due to this. |
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
Every mongos instance starts a balancer round every few seconds. While looking for candidate chunks to move, it always reads all of the chunks with the given ns from the config database, many times only to realize that the collection is in fact well balanced. This may cause a significant network load. |
| Comments |
| Comment by Kaloian Manassiev [ 20/Jul/16 ] |
|
With the resolution of |
| Comment by Matthieu Rigal [ 18/Dec/15 ] |
|
I can also confirm a very similar problem, with unnecessary high network load! |
| Comment by Greg Studer [ 03/Jul/14 ] |
|
Confirmed - with chunk diffing changes, the balancer chunk reload is more expensive than it needs to be. |
| Comment by Jan Jeek [ 23/Jun/14 ] |
|
This is the query that causes the network load: |
| Comment by Jan Jeek [ 23/Jun/14 ] |
|
Sorry about the version confusion. I have looked at the source code to confirm my assumptions and that was in the v2.6.3 branch. In production we actually use a 2.4 that comes with Debian. |
| Comment by Jan Jeek [ 23/Jun/14 ] |
|
Log from a temporary mongos process run with -vvvvv |
| Comment by Thomas Rueckstiess [ 20/Jun/14 ] |
|
Hi Jan, Can you please attach the log file of the mongos that is doing the balancing? Also just to confirm: Version 2.6.3 has not been released yet. Are you running 2.6.2? Thanks, |