[DOCS-14449] Investigate changes in SERVER-56307: The chunk migration "convergence algorithm" is very primitive Created: 13/May/21  Updated: 13/Nov/23  Resolved: 26/Aug/21

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 4.2.15, 4.4.7, 4.0.26, 5.0.0-rc3, 5.1.0-rc0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Jeffrey Allen
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
backported by DOCS-14468 [BACKPORT] [v4.0] The chunk migration... Closed
backported by DOCS-14484 [BACKPORT] [v4.4] The chunk migration... Closed
backported by DOCS-14485 [BACKPORT] [v4.2] The chunk migration... Closed
backported by DOCS-14583 BACKPORT [v5.0] The chunk migration "... Closed
Documented
documents SERVER-56307 The chunk migration "convergence algo... Closed
Participants:
Days since reply: 2 years, 23 weeks, 6 days ago
Epic Link: DOCSP-15042
Story Points: 3

 Description   

Description

Downstream Change Summary

The changeset adds a new server parameter ('minCatchUpPercentageBeforeBlockingWrites', loaded at startup time) to adjust the behaviour of the live migration protocol in the catchup phase.

Higher percentage values anticipate the moment when incoming write operations over the data being migrated get suspended: this allows the migration protocol to converge earlier in situations of high load - at the cost of introducing an increased latency over concurrent upserts and deletes.

Description of Linked Ticket

The chunk migration "convergence algorithm" is the logic which the donor and participant implement in order for the donor to decide when to enter the critical section and block writes.

The current implementation relies on the recipient being capable of catching up on every modification occurred during the migration and reaching the STEADY state to allow the donor to enter the critical section.

It has been detected that under situations of heavy load such condition may be never met, as the rate of incoming modifications is bigger than what can be transferred to the recipient.

Potetial fixes are:

(1) The donor sends to the recipient some measure of how much mods are left, so it can enter the steady state based on some delta, rather than zero.

(2) The donor decides to enter the critical section not based on whether the recipient has decided to enter steady-state (which includes a wait for majority)
(Option 2 might not be appropriate, though because it adds the recipient's majority wait under the critical section)

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 26/Aug/21 ]

Author:

{'name': 'Jeff Allen', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}

Message: (DOCS-14449): maxCatchUpPercentageBeforeBlockingWrites server parameter
Branch: v4.0
https://github.com/mongodb/docs/commit/11015e548c96d1df3cc6c7bc58820014b7293060

Comment by Githook User [ 26/Aug/21 ]

Author:

{'name': 'Jeff Allen', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}

Message: (DOCS-14449): maxCatchUpPercentageBeforeBlockingWrites server parameter
Branch: v4.2
https://github.com/mongodb/docs/commit/fa522b867c5dc7dd4c6380b443626ab67c4b8ec0

Comment by Githook User [ 26/Aug/21 ]

Author:

{'name': 'Jeff Allen', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}

Message: (DOCS-14449): maxCatchUpPercentageBeforeBlockingWrites server parameter
Branch: v5.0
https://github.com/mongodb/docs/commit/660765c3ef55873deaa3a5731fe4b80955cb1893

Comment by Githook User [ 26/Aug/21 ]

Author:

{'name': 'Jeff Allen', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}

Message: (DOCS-14449): maxCatchUpPercentageBeforeBlockingWrites server parameter
Branch: v4.4
https://github.com/mongodb/docs/commit/2a02f2d334d7ba2fe2456da06a1e34167c8e33f4

Comment by Githook User [ 26/Aug/21 ]

Author:

{'name': 'Jeff Allen', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}

Message: (DOCS-14449): maxCatchUpPercentageBeforeBlockingWrites server parameter
Branch: master
https://github.com/mongodb/docs/commit/6f202efc9993c8d40a135b0c407e5c0891faf922

Generated at Thu Feb 08 08:10:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.