Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38761

Explore options for batch size when gathering shard results

    XMLWordPrintableJSON

Details

    • Query Execution
    • 0

    Description

      Some of our benchmarks appear to be sensitive to the batch size that mongos uses to get documents from documents from cluster cursors. Currently, we use large batches, but we may want to consider an approach that adapts to more workloads.

      Some possibilities:
      1) Use a small batch size when the pipeline has a $limit that puts an upper bound on the number of documents that the mongos will need (perhaps upper_bound / num_shards, which would allow for one optimally sized batch from each shard in the best case).
      2) Use small batches for the first few getMore commands and then ramp up batch size as execution continues.
      3) Investigate if always using a small batch size is potentially a better trade-off than always using a large batch size.

      Attachments

        Activity

          People

            backlog-query-execution Backlog - Query Execution
            justin.seyster@mongodb.com Justin Seyster
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: