[SERVER-37080] Implement tuneable batch size for chunk migrations Created: 11/Sep/18  Updated: 29/Oct/23  Resolved: 21/Nov/18

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.4.17, 3.6.8, 4.0.2
Fix Version/s: 3.4.18, 3.6.10, 4.0.5, 4.1.6

Type: Improvement Priority: Major - P3
Reporter: James Kovacs Assignee: Kevin Pulo
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Documented
is documented by DOCS-12241 Docs for SERVER-37080: Implement tune... Closed
Duplicate
is duplicated by SERVER-8835 _secondaryThrottle should GLE after e... Closed
Related
is related to SERVER-27725 Use batch insert when migrating chunks Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6, v3.4
Sprint: Sharding 2018-09-24, Sharding 2018-10-08, Sharding 2018-10-22, Sharding 2018-11-05, Sharding 2018-11-19, Sharding 2018-12-03
Participants:
Case:

 Description   

SERVER-27725 implemented batch inserts for chunk migrations, but for some workloads inserting a ~16MB batch can impact secondary read performance. This ticket would be to implement a tuneable parameter to control how many documents were inserted using the batch insert rather than inserting the entire batch at once.



 Comments   
Comment by Githook User [ 22/Nov/18 ]

Author:

{'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com', 'username': 'devkev'}

Message: SERVER-37080 tunable migration clone insertion batches

(cherry picked from commit 9cc66d04356cf7b542df77ebe9dbe94445308240)
Branch: v3.6
https://github.com/mongodb/mongo/commit/28b8f0c479cf7a08e84c0c3c95b750cef2c04efd

Comment by Githook User [ 22/Nov/18 ]

Author:

{'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com', 'username': 'devkev'}

Message: SERVER-37080 tunable migration clone insertion batches

(cherry picked from commit 9cc66d04356cf7b542df77ebe9dbe94445308240)
Branch: v4.0
https://github.com/mongodb/mongo/commit/b2ea16518d575bf50c25f4b9fcba07591f15d70f

Comment by Githook User [ 21/Nov/18 ]

Author:

{'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com', 'username': 'devkev'}

Message: SERVER-37080 tunable migration clone insertion batches
Branch: master
https://github.com/mongodb/mongo/commit/9cc66d04356cf7b542df77ebe9dbe94445308240

Comment by Gregory McKeon (Inactive) [ 06/Nov/18 ]

kevin.pulo would you mind forward porting this soon? the ticket's in a bit of a weird state since it's in a released 3.4 version but not in master.

Comment by Githook User [ 24/Sep/18 ]

Author:

{'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com', 'username': 'devkev'}

Message: SERVER-37080 tunable migration clone insertion batches
Branch: v3.4
https://github.com/mongodb/mongo/commit/5099801229722b5d25dfaa20a675905353b9681e

Generated at Thu Feb 08 04:44:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.