[SERVER-8835] _secondaryThrottle should GLE after every N documents rather than after each Created: 03/Mar/13  Updated: 09/Oct/18  Resolved: 01/Oct/18

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.4.0-rc1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Asya Kamsky Assignee: Alyson Cabral (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-37080 Implement tuneable batch size for chu... Closed
Related
related to SERVER-27725 Use batch insert when migrating chunks Closed
Operating System: ALL
Participants:

 Description   

The option _secondaryThrottle when set to true will GLE w:2 after every single document being copied/cloned.

For cases of very small documents (aka very large number of documents in chunk range) when there are also inserts going on in the from shard into this range, this can be a lot slower (relative to unthrottled migration) than for a collection with fewer documents in a chunk.

Maybe the waitForReplication() should be every N documents where N is some fraction of maxDocs in chunk (so say, waitForReplication() every 1/100th of chunk or 1/1000th). Otherwise the delay it introduces is too unpredictable.



 Comments   
Comment by Alyson Cabral (Inactive) [ 01/Oct/18 ]

SERVER-37080 solves this pain. Closing this as a duplicate. Check that ticket for updates. 

Comment by Asya Kamsky [ 13/Jul/18 ]

Are we removing secondary throttle as an option?

Because the issue here is not slowing down or speeding up the balancer via range deleter tuning, this was specifically about secondary throttle on initial document copying being too wildly unpredictable depending on document size.

 

Comment by Gregory McKeon (Inactive) [ 12/Jul/18 ]

We won't fix _secondaryThrottle, but there's a configuration for slowing down the balancer described in SERVER-33237.

Comment by Asya Kamsky [ 19/Apr/18 ]

SERVER-22233 was closed as dup of SERVER-23340 which turns off throttling by default.

Comment by Asya Kamsky [ 19/Apr/18 ]

This ticket is to improve secondary throttle parameter's impact on performance, even though we no longer turn it on by default, it's still not useful for collections with small documents (aka chunks with a very large number of documents).

Comment by Kaloian Manassiev [ 20/Jan/16 ]

Resolving in place of SERVER-22233, which will include gaining a better understand of what the secondary throttling needs are in version 3.0 and doing the changes to improve the speed of chunk migration.

Comment by Eliot Horowitz (Inactive) [ 01/Apr/13 ]

without more clear evidence one way or the other, don't want to change

Generated at Thu Feb 08 03:18:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.