[SERVER-59517] POC: change query yielding to retain a valid cursor position and storage data pointer Created: 23/Aug/21  Updated: 16/Sep/21  Resolved: 16/Sep/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Ian Boros
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-59518 Change query yielding behavior to WT:... Closed
Sprint: Execution Team 2021-09-06, Execution Team 2021-09-20
Participants:

 Description   

The following is how the current code path for yielding works:

Query yield releases the storage engine resources here. Locked reads will implicitly open a new storage transaction on the next storage access; lock-free reads will explicitly open a new storage transaction in the restore logic here.
abandonSnapshot() then closes the storage transaction here.
_txnClose then calls rollback_transaction here.

UPDATE:
We've identified a few specifics we wish to cover with this POC before splitting the work into final SERVER tickets
1) Change query yielding behavior to WT::commit_transaction instead of WT::rollback_transaction. This will cause the WT cursor to retain its position with a data pointer across commit_transaction, and then start_transaction will set a new storage snapshot from which getNext() on the cursor should smoothly forward.
2) Remove the save/restore logic that saves the current RecordID on query yield and then restores the cursor position by doing WT::search_near(recordID). This logic should no longer be necessary because the cursor will retain its position on WT::commit_transaction. It also obscures whether switching to WT::commit_transaction works for retaining cursor validity.
3) Remove the query layer cursor BSON copying logic across query yield. The WT::CURSOR should remain valid and available during yield, so the query layer pointers to the cursor data won't disappear.

Lastly, we want to get some initial perf results for these changes, to help direct design decisions. It would be helpful to see
1) if there are any perf changes on existing workloads just changing WT::rollback_transaction to WT::commit_transaction.
2) if there are any perf changes on existing workloads with all of the changes.
3) if there are any differences between SBE and the classic query engine regarding performance – this will inform whether we need to special handle the classic engine, say.



 Comments   
Comment by Ian Boros [ 16/Sep/21 ]

Closing as "Done" since nothing was merged into the codebase but we have the POC.

Comment by Ian Boros [ 03/Sep/21 ]

As part of this work I will run the existing perf workloads, just to see if anything changed by switching the call to rollback -> commit.

Generated at Thu Feb 08 05:47:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.