[COMPASS-6210] Investigate changes in PM-3001: BACKPORT: Add concurrency to the chunk migration code Created: 17/Oct/22  Updated: 19/Oct/22  Resolved: 19/Oct/22

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: No version

Type: Investigation Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documentation Changes: Not Needed

 Description   
Original Downstream Change Summary

We are providing a way to add a server parameter (setParameter) to change the destination shard insert concurrency. backporting to 4.4, 5.0 and 6.0.

syntax changes -
{setParameter: { migrationConcurrency: N }}

Description of Linked Ticket

Summary

This project is for backporting changes to parallelize portions of the existing chunk migration algorithm. We propose backporting this through mongoDB 4.4.

Motivation

During PM-2740 spike we implemented the following changes to realize up to 5 times higher chunk migration throughput:

  • Increase the level of concurrency on the destination for the insert portion when moving a chunk.

As a result of that spike, we created PM-3014, which implements a new server parameter specifying the level of concurrency for the item above.


Generated at Wed Feb 07 22:42:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.