The documentation on Secondary Throttle does not accurately describe how it changes the behavior of migration.
When Secondary Throttle is not set, then the migration code will copy the documents from the FROM shard to the TO shard as quickly as it can, without waiting for replication. When all of the documents have been copied, but before the migration enters the critical section, the FROM shard will wait for replication to complete to a MAJORITY of the nodes in the destination shard.
Ref: https://github.com/mongodb/mongo/blob/r2.4.7/src/mongo/s/d_migrate.cpp#L1965-L1980
This has the effect of creating a bottleneck right at the end of the document movement, as the migration needs to wait for all of the writes to replicate.
When Secondary Throttle is set, then the FROM shard will wait for replication to complete to a majority of the nodes in the TO shard after each document is inserted in the TO shard.
Ref: https://github.com/mongodb/mongo/blob/r2.4.7/src/mongo/s/d_migrate.cpp#L905-L911
In addition to this, the FROM shard will always wait for the replication to complete to the TO shard before entering the critical section.
The net effect of this is to smooth out the load and reduce the overall load on the replica set.
The current documentation is incorrect in two ways:
- It implies that there is no wait for replication if Secondary Throttle is not set
- It gives the write concern for Secondary Throttle as {w:2}, when it should really be {w:"majority"}