Limiting the memory usage during the cloning phase on the recipient shard

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Critical - P2
    • 7.1.1, 7.2.0-rc0, 5.0.22, 7.0.3, 6.0.12
    • Affects Version/s: 7.0.0, 6.0.5, 7.1.0-rc0, 5.0.16
    • Component/s: None
    • None
    • Sharding NYC
    • Fully Compatible
    • v7.0, v6.0, v5.0
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      As part of PM-3014 (and its backports to 5.0/6.0 done under PM-3001) we changed the amount of memory used during the cloning phase on the recipient shard, going from a constant amount to a variable one that depends on the speed of fetching data (producer) and locally inserting it (consumer).

      Cloning a chunk of 80GB we saw spikes of memory of 52GB.

              Assignee:
              Randolph Tan
              Reporter:
              Sergi Mateo Bellido
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: