[SERVER-7686] Chunk migration and cleanup performance Created: 16/Nov/12 Updated: 27/Oct/15 Resolved: 04/Dec/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.2.1 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Aristarkh Zagorodnikov | Assignee: | Ian Daniel |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux 64-bit |
||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
In our environment we generally use MongoDB in two modes: You surely see the difference between the fs.chunks and fs.files migration. |
| Comments |
| Comment by Ian Daniel [ 04/Dec/12 ] | ||
|
Hi Aristarkh, I am glad that your problem is solved and that _secondaryThrottle helped somewhat. There is already an issue for documenting using it as a balancer configuration item: Kind regards, | ||
| Comment by Aristarkh Zagorodnikov [ 29/Nov/12 ] | ||
|
David, it looks like our problem is solved, and at least partially by enabling the "_secondaryThrottle" parameter. I wanted to suggest making this the default, but found out that there is already | ||
| Comment by David Hows [ 29/Nov/12 ] | ||
|
Aristarkh, Is there anything more we can do for you on this issue? Or do you believe the combination of things you have done has helped? Cheers, David | ||
| Comment by Aristarkh Zagorodnikov [ 27/Nov/12 ] | ||
|
As we're still recovering from We haven't (yet) observe the problem since we did the following: The (1) and (3) should increase responsiveness and prevent blocking, so if the problem persisted, we still would have I/O peaks. As we have none and the config.changelog collection indicates that migrations on problematic collections are still done, it's either (2) or (4). I don't believe much in (2), since disk caches are 64MB only and there is much more data transferred along with truly random I/O, so it appears that _secondaryThrottle really helps. I also observed chunk migration times that took considerably longer time than originally observed, still without putting disks to their knees. | ||
| Comment by Ian Daniel [ 27/Nov/12 ] | ||
|
Hi Aristarkh, Do you have any indication yet of whether the _secondaryThrottle option is helping? Kind regards, | ||
| Comment by Aristarkh Zagorodnikov [ 19/Nov/12 ] | ||
|
Thanks for the advice, I'll enable it now, but since fs.files migrations are not done very often it may take several days to find out whether it helps.
| ||
| Comment by Eliot Horowitz (Inactive) [ 18/Nov/12 ] | ||
|
There is a new option we're experimenting with in 2.2 called: _secondaryThrottle If you do this against a mongos:
Migrations will throttle themselves based on replication, which has the impact of elongating the window reducing impact. Can you try that? |