Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38761

Explore options for batch size when gathering shard results

    • Query Execution
    • 0

      Some of our benchmarks appear to be sensitive to the batch size that mongos uses to get documents from documents from cluster cursors. Currently, we use large batches, but we may want to consider an approach that adapts to more workloads.

      Some possibilities:
      1) Use a small batch size when the pipeline has a $limit that puts an upper bound on the number of documents that the mongos will need (perhaps upper_bound / num_shards, which would allow for one optimally sized batch from each shard in the best case).
      2) Use small batches for the first few getMore commands and then ramp up batch size as execution continues.
      3) Investigate if always using a small batch size is potentially a better trade-off than always using a large batch size.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            justin.seyster@mongodb.com Justin Seyster
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: