[SERVER-31992] getMore on tailable awaitData cursor doesn't attempt to fill batch on mongos Created: 15/Nov/17  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework, Querying
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Trivial - P5
Reporter: Siyuan Zhou Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: change-streams-improvements, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Query Execution
Participants:

 Description   

Once the cluster cursor returns any result, RouterStageMerge doesn't issue any new getMore. However the batch may haven't been filled up. If the result set isn't empty, RouterStageMerge should wait for ARM to return an EOF when ARM gets an empty batch rather than checking _arm.ready().

The following patch fixes it. However, if the batch size is enforced for getMore's, getMore's will always wait for a full awaitDataTimeout (1 second by default) if there isn't enough data to fill a batch. The original implementation can return some results earlier without waiting. The difference is interesting when resuming change streams where returning full batches is more efficient.

diff --git a/src/mongo/s/query/router_stage_merge.cpp b/src/mongo/s/query/router_stage_merge.cpp
index e571ed4ed2..fcf6f560fa 100644
--- a/src/mongo/s/query/router_stage_merge.cpp
+++ b/src/mongo/s/query/router_stage_merge.cpp
@@ -70,7 +70,7 @@ StatusWith<ClusterQueryResult> RouterStageMerge::awaitNextWithTimeout(ExecContex
     invariant(_params->tailableMode == TailableMode::kTailableAndAwaitData);
     // If we are in kInitialFind or kGetMoreWithAtLeastOneResultInBatch context and the ARM is not
     // ready, we don't block. Fall straight through to the return statement.
-    while (!_arm.ready() && execCtx == ExecContext::kGetMoreNoResultsYet) {
+    while (!_arm.ready() && execCtx != ExecContext::kInitialFind) {
         auto nextEventStatus = getNextEvent();
         if (!nextEventStatus.isOK()) {
             return nextEventStatus.getStatus();



 Comments   
Comment by David Storch [ 26/Feb/18 ]

asya, I don't think so. It doesn't appear on our prioritized list of change streams improvements.

Comment by Asya Kamsky [ 22/Feb/18 ]

david.storch should this be part of ongoing change streams epic?

Comment by Siyuan Zhou [ 16/Nov/17 ]

charlie.swanson, here's the issue found by testing shell helper. The behavior is tested on an unshared collection. Not sure if this issue applies to sharded collections.

Generated at Thu Feb 08 04:28:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.