Limiting the memory usage during the cloning phase on the recipient shard

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Critical - P2
    • 7.1.1, 7.2.0-rc0, 5.0.22, 7.0.3, 6.0.12
    • Affects Version/s: 7.0.0, 6.0.5, 7.1.0-rc0, 5.0.16
    • Component/s: None
    • None
    • Sharding NYC
    • Fully Compatible
    • v7.0, v6.0, v5.0
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None

      As part of PM-3014 (and its backports to 5.0/6.0 done under PM-3001) we changed the amount of memory used during the cloning phase on the recipient shard, going from a constant amount to a variable one that depends on the speed of fetching data (producer) and locally inserting it (consumer).

      Cloning a chunk of 80GB we saw spikes of memory of 52GB.

            Assignee:
            Randolph Tan
            Reporter:
            Sergi Mateo Bellido
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: