[SERVER-60138] Improve the allocated memory for getMore commands Created: 22/Sep/21  Updated: 29/Oct/23  Resolved: 26/Oct/21

Status: Closed
Project: Core Server
Component/s: Query Execution
Affects Version/s: None
Fix Version/s: 5.2.0

Type: Improvement Priority: Major - P3
Reporter: Mohammad Dashti (Inactive) Assignee: Mohammad Dashti (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-55535 Performance tests to exercise change ... Closed
is depended on by SERVER-36346 Large memory consumption per pending ... Closed
Related
Backwards Compatibility: Fully Compatible
Sprint: QE 2021-10-04, QE 2021-10-18, QE 2021-11-01
Participants:
Linked BF Score: 120

 Description   

We want to improve the reservation of the maximum size versus when we should reserve something smaller for the getMore command. Here's a pragmatic approach:

  1. For a normal cursor with no explicit batch size, 16MB.
  2. For a normal cursor with explicit batch size, perhaps 8MB.
  3. For a tailable cursor, which may often return 0 or a small number of results, perhaps 4MB.

We can better fine-tune the above heuristics based on the performance benchmarks in PERF-2565 and PERF-2566. Another worthwhile question that's worth answering as part of this task and its related experiments is that "What happens if we do not do pre-allocation at all"?



 Comments   
Comment by Githook User [ 26/Oct/21 ]

Author:

{'name': 'Mohammad Dashti', 'email': 'mdashti@gmail.com', 'username': 'mdashti'}

Message: SERVER-60138 Improved the allocated memory for the `getMore` commands.
Branch: master
https://github.com/mongodb/mongo/commit/f6d975343b9f919b61e6d1b4e7333fe0ecfe50c4

Comment by Githook User [ 21/Oct/21 ]

Author:

{'name': 'Mohammad Dashti', 'email': 'mdashti@gmail.com', 'username': 'mdashti'}

Message: SERVER-60138 Added the `big_collection` benchmark to the list of performance benchmarks
Branch: master
https://github.com/mongodb/mongo/commit/9a62d366c34b8bb913f5f48a9a40351c21502025

Comment by Mohammad Dashti (Inactive) [ 21/Oct/21 ]

All reviewers accepted the PR and it's ready to be merged. bernard.gorman suggested waiting on merging this PR until SERVER-55535 is closed.

Generated at Thu Feb 08 05:49:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.