[SERVER-53534] Support including postBatchResumeToken in cursor response for non-change streams aggregations Created: 29/Dec/20  Updated: 29/Oct/23  Resolved: 10/Mar/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.9.0

Type: New Feature Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Mindaugas Malinauskas
Resolution: Fixed Votes: 0
Labels: query-work-resharding
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-49897 Insert no-op entries into oplog buffe... Closed
Related
is related to SERVER-49895 Expose getLatestOplogTimestamp() in a... Closed
is related to SERVER-54857 Re-examine disposal approach of the i... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query Execution 2021-02-22, Query Execution 2021-03-08, Query Execution 2021-03-22
Participants:

 Description   

The changes from 7d915a6 as part of SERVER-49895 made it so including {$_requestReshardingResumeToken: true} in the aggregation request would cause a {postBatchResumeToken: {ts: <latestOplogEntryTimestamp>}} to be included in the cursor response. However, the plumbing was only done in CollectionScan and therefore only applies when the pipeline can be entirely optimized away. PlanExecutorPipeline::_latestOplogTimestamp is only updated when PlanExecutorPipeline::_isChangeStream == true. This makes including {$_requestReshardingResumeToken: true} not useful when running more complex aggregations such as the one resharding is using to fetch donor shard's oplog entries.

Steps to reproduce

python buildscripts/resmoke.py run --suite=sharding jstests/sharding/resharding_oplog_sync_agg_resume_token.js

diff --git a/jstests/sharding/resharding_oplog_sync_agg_resume_token.js b/jstests/sharding/resharding_oplog_sync_agg_resume_token.js
index 047a01abc8..3dbeb5d4d3 100644
--- a/jstests/sharding/resharding_oplog_sync_agg_resume_token.js
+++ b/jstests/sharding/resharding_oplog_sync_agg_resume_token.js
@@ -28,7 +28,9 @@ const localDb = rst.getPrimary().getDB("local");
 jsTest.log("Run aggregation pipeline on oplog with $_requestReshardingResumeToken set");
 const resEnabled = localDb.runCommand({
     aggregate: "oplog.rs",
-    pipeline: [{$match: {ts: {$gte: Timestamp(0, 0)}}}],
+    // The $addFields prevents the pipeline from being optimized away as a simple plan executor.
+    // This is necessary to force the pipeline to be evaluated using PlanExecutorPipeline.
+    pipeline: [{$match: {ts: {$gte: Timestamp(0, 0)}}}, {$addFields: {extra: {$literal: "hi"}}}],
     $_requestReshardingResumeToken: true,
     cursor: {batchSize: 1}
 });



 Comments   
Comment by Githook User [ 10/Mar/21 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com'}

Message: SERVER-53534 Support including postBatchResumeToken in cursor response for non-change streams aggregations
Branch: master
https://github.com/mongodb/mongo/commit/159a29e1e69948106f5bc601042b1baecf6fe982

Comment by Githook User [ 10/Mar/21 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com'}

Message: SERVER-53534 Support including postBatchResumeToken in cursor response for non-change streams aggregations
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/259dcf0d9af74a043f7e7466942b21bc53bc9bde

Generated at Thu Feb 08 05:31:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.