Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-57667

Improve processing speed for resharding's collection cloning pipeline

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 5.0.3, 5.1.0-rc0
    • Query Language
    • None
    • Minor Change
    • v5.0
    • Query Optimization 2021-07-12, Query Optimization 2021-07-26, Query Optimization 2021-08-09
    • 170

    Description

      SERVER-57483 partially addressed the regression of $lookup results not being cached by special casing the config.cache.chunks collection. There is still room for significant improvement. It has been found that using ChunkManager to binary search the temporary resharding collection's chunk ranges rather than using DocumentSourceSequentialDocumentCache to scan linearly through the same chunk ranges results in a >10x speedup in the overall collection cloning runtime.

      This ticket represents the work to add a custom aggregation stage or expression or otherwise optimize resharding's collection cloning pipeline to achieve the observed >10x speedup.

      Attachments

        Issue Links

          Activity

            People

              joshua.lapacik@mongodb.com Joshua Lapacik
              max.hirschhorn@mongodb.com Max Hirschhorn
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: