Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-57667

Improve processing speed for resharding's collection cloning pipeline

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.0.3, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: Query Language
    • None
    • Minor Change
    • v5.0
    • Query Optimization 2021-07-12, Query Optimization 2021-07-26, Query Optimization 2021-08-09
    • 170

      SERVER-57483 partially addressed the regression of $lookup results not being cached by special casing the config.cache.chunks collection. There is still room for significant improvement. It has been found that using ChunkManager to binary search the temporary resharding collection's chunk ranges rather than using DocumentSourceSequentialDocumentCache to scan linearly through the same chunk ranges results in a >10x speedup in the overall collection cloning runtime.

      This ticket represents the work to add a custom aggregation stage or expression or otherwise optimize resharding's collection cloning pipeline to achieve the observed >10x speedup.

            Assignee:
            joshua.lapacik@mongodb.com Joshua Lapacik (Inactive)
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: