[SERVER-14041] enhance secondaryThrottle parameter Created: 23/May/14 Updated: 27/Oct/15 Resolved: 15/Jul/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 2.7.4 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Greg Studer | Assignee: | Randolph Tan |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
For clusters with significant load on secondary nodes of replica set shards, the default secondary throttle mode (w:1) may not ensure that chunk migration doesn't interfere with all secondaries. Support other write concern "w" modes (1, 2, "majority") for this parameter to allow configurable throttling. Also the default is changed to { _secondaryThrottle: true } for chunk migration. |
| Comments |
| Comment by Randolph Tan [ 16/Jul/14 ] | |||
|
Tested by hand. Scenarios tested: 2. MoveChunk command - 3. cleanupOrphaned - same as #2. Note that for test 2 & 3, make sure that mongos sends the right parameter when sending the right parameters for moveChunk and make sure the shard does the same way for _recvChunkStart (must have at least vebose level 1). To make sure that the write concern was passed down correctly by checking the logs for "begin removal of" (requires log level 1). | |||
| Comment by Randolph Tan [ 16/Jul/14 ] | |||
|
This patch allows the user to specify a more detailed write concern mode compared to the on/off switch before. This currently only supports the "w" and "wtimeout" parameters (fsync and j are ignored). The places where this are supported are in the balancer config, moveChunk command (as well as the moveChunk shard version and _recvChunkStart) and cleanupOrphaned. Below are a list of examples on how to pass the setting. Note: that the balancer config document 'overloads' the _secondaryThrottle field, whereas for commands, the new writeConcern field should be used.
This patch also make _secondaryThrottle and secondaryThrottle interchangable for commands (again, not for the config doc). In the case when both of them are specified, secondaryThrottle overrides _secondaryThrottle. The migration process involves a lot of write operations and here's a list of all the places where mongod waits for replication during migration: From side: To side: RangeDeleter waits for replication on: If the user explicitly passes a custom write concern, all items not marked with { w: majority } will use it. In other words, the cases where the server waits for majority is not affected by the secondaryThrottle setting. Notes:
| |||
| Comment by Githook User [ 16/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: This reverts commit 37f2a1e3b724dbd9e1f8eafd4ac87c5bf613c048 (undo revert). | |||
| Comment by Githook User [ 15/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: Revert " This reverts commit fa1233fbe4a48ef0675820f381987f1df4f42f75. | |||
| Comment by Githook User [ 15/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: Revert " This reverts commit d85f99a8a150a1932d72bd393511be2b85a409c7. | |||
| Comment by Githook User [ 15/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: Fix debug build compile failure | |||
| Comment by Githook User [ 15/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: | |||
| Comment by Githook User [ 15/Jul/14 ] | |||
|
Author: {u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}Message: |