Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-14449

Investigate changes in SERVER-56307: The chunk migration "convergence algorithm" is very primitive

      Description

      Downstream Change Summary

      The changeset adds a new server parameter ('minCatchUpPercentageBeforeBlockingWrites', loaded at startup time) to adjust the behaviour of the live migration protocol in the catchup phase.

      Higher percentage values anticipate the moment when incoming write operations over the data being migrated get suspended: this allows the migration protocol to converge earlier in situations of high load - at the cost of introducing an increased latency over concurrent upserts and deletes.

      Description of Linked Ticket

      The chunk migration "convergence algorithm" is the logic which the donor and participant implement in order for the donor to decide when to enter the critical section and block writes.

      The current implementation relies on the recipient being capable of catching up on every modification occurred during the migration and reaching the STEADY state to allow the donor to enter the critical section.

      It has been detected that under situations of heavy load such condition may be never met, as the rate of incoming modifications is bigger than what can be transferred to the recipient.

      Potetial fixes are:

      (1) The donor sends to the recipient some measure of how much mods are left, so it can enter the steady state based on some delta, rather than zero.

      (2) The donor decides to enter the critical section not based on whether the recipient has decided to enter steady-state (which includes a wait for majority)
      (Option 2 might not be appropriate, though because it adds the recipient's majority wait under the critical section)

      Scope of changes

      Impact to Other Docs

      MVP (Work and Date)

      Resources (Scope or Design Docs, Invision, etc.)

            Assignee:
            jeffrey.allen@mongodb.com Jeffrey Allen
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved:
              2 years, 33 weeks, 6 days ago