-
Type: Task
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Aggregation Framework
-
Labels:
-
Query Execution
-
0
Some of our benchmarks appear to be sensitive to the batch size that mongos uses to get documents from documents from cluster cursors. Currently, we use large batches, but we may want to consider an approach that adapts to more workloads.
Some possibilities:
1) Use a small batch size when the pipeline has a $limit that puts an upper bound on the number of documents that the mongos will need (perhaps upper_bound / num_shards, which would allow for one optimally sized batch from each shard in the best case).
2) Use small batches for the first few getMore commands and then ramp up batch size as execution continues.
3) Investigate if always using a small batch size is potentially a better trade-off than always using a large batch size.
- related to
-
SERVER-43657 Investigate whether there are situations in which we don't need to set batchSize to 0 when opening a cursor for a pipeline
- Backlog