Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-59517

POC: change query yielding to retain a valid cursor position and storage data pointer

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Execution Team 2021-09-06, Execution Team 2021-09-20

      The following is how the current code path for yielding works:

      Query yield releases the storage engine resources here. Locked reads will implicitly open a new storage transaction on the next storage access; lock-free reads will explicitly open a new storage transaction in the restore logic here.
      abandonSnapshot() then closes the storage transaction here.
      _txnClose then calls rollback_transaction here.

      UPDATE:
      We've identified a few specifics we wish to cover with this POC before splitting the work into final SERVER tickets
      1) Change query yielding behavior to WT::commit_transaction instead of WT::rollback_transaction. This will cause the WT cursor to retain its position with a data pointer across commit_transaction, and then start_transaction will set a new storage snapshot from which getNext() on the cursor should smoothly forward.
      2) Remove the save/restore logic that saves the current RecordID on query yield and then restores the cursor position by doing WT::search_near(recordID). This logic should no longer be necessary because the cursor will retain its position on WT::commit_transaction. It also obscures whether switching to WT::commit_transaction works for retaining cursor validity.
      3) Remove the query layer cursor BSON copying logic across query yield. The WT::CURSOR should remain valid and available during yield, so the query layer pointers to the cursor data won't disappear.

      Lastly, we want to get some initial perf results for these changes, to help direct design decisions. It would be helpful to see
      1) if there are any perf changes on existing workloads just changing WT::rollback_transaction to WT::commit_transaction.
      2) if there are any perf changes on existing workloads with all of the changes.
      3) if there are any differences between SBE and the classic query engine regarding performance – this will inform whether we need to special handle the classic engine, say.

            Assignee:
            ian.boros@mongodb.com Ian Boros
            Reporter:
            dianna.hohensee@mongodb.com Dianna Hohensee (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: