[SERVER-62804] Support retention of valid cursor across aggregation and getMore for the $exchange operator Created: 20/Jan/22  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-68575 Investigate how to store storage stat... Closed
Duplicate
is duplicated by SERVER-62574 Change exchange aggregation queries c... Closed
Related
related to SERVER-60742 Retain valid and positioned cursors a... Open
Assigned Teams:
Storage Execution
Sprint: Execution Team 2022-09-05, Execution Team 2022-09-19
Participants:

 Description   

SERVER-60742 implemented support for agg/getMore, but disabled it in the case of $exchange. The issue is that a PlanExecutor can have nested PlanExecutors in their plan trees, and in the case of $exchange the nested PlanExecutor can be shared across multiple ClientCursors – a mutex protects against concurrent access. So we will need to associate RecoveryUnits with PlanExecutors rather than ClientCursors, and have a mechanism by which we load the appropriate RU onto the opCtx for the duration of the use of an associated PlanExecutor.



 Comments   
Comment by Dianna Hohensee (Inactive) [ 04/Aug/22 ]

I think we can probably skip the global RU registry. See SERVER-68575. This problem fix might just fall out of SERVER-60742.

Comment by Dianna Hohensee (Inactive) [ 01/Aug/22 ]

Proposed solution:

We will create a globally accessible RecoveryUnit registry that will hold RUs while idle. The registry will effectively contain a map of some new ID to RecoveryUnit, and each PlanExecutor/ClientCursor will save the ID in order to find the right RecoveryUnit when needed.

RUs cannot be associated with PlanExecutors because PlanExecutors can be nested; nor can RUs be associated with ClientCursors because ClientCursors can be shared across threads for $exchange. Since there is non 1:1 mapping of storage state to query-level cursor concept, we should place the storage state (RecoveryUnit) by itself to the side until needed.

Generated at Thu Feb 08 05:56:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.