Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32885

Overlap chunk clone application on the donor with fetching documents from the recipient

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 3.4.15, 3.6.4, 3.7.4
    • Sharding
    • None
    • Fully Compatible
    • v3.6, v3.4
    • Sharding 2018-02-12, Sharding 2018-02-26, Sharding 2018-03-12, Sharding 2018-03-26

    Description

      The current chunk clone logic (as part of moveChunk), ping-pongs serially between the donor fetching chunk documents and the recipient applying them. Since both sequences are potentially slow and might be bringing cold data from cache, there could be benefit in overlapping them.

      By putting this loop in a separate thread we can overlap that sequence and possibly double the clone speed.

      This can be implemented by spawning a ThreadPool at the beginning of the clone phase and putting a fixed-size BlockingQueue (similar to the JournalWriter) between the migrate driver thread and the worker thread.

      It is possible that such aggressive clone might not be appropriate for the MMAP V1 storage engine, so this optimization should only be done for WT.

      Attachments

        Issue Links

          Activity

            People

              blake.oler@mongodb.com Blake Oler
              kaloian.manassiev@mongodb.com Kaloian Manassiev
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: