Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-40854

Hybrid index builds should reposition cursors to the beginning of the side table after yielding

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.11
    • Component/s: None
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Sprint:
      Storage NYC 2019-05-06
    • Linked BF Score:
      64

      Description

      During the first drain phase of hybrid index builds, operations are applied in batches from the side table record store into the index. At the end of each batch (1000 operations by default), the side table cursor is saved and restored. This restore may reposition on new records, but not necessarily the newest records. This can cause batch applications after yield to apply causal operations out of order.

      For example, imagine two writer threads, A and B, writing into the side table and an index build thread. In this example, thread A inserts then deletes the key {a: 1}. Thread B inserts an unrelated key.

      writer A writer B index builder
      start_transaction    
      RID 1: insert key {a: 1}    
        start_transaction  
        RID 2: insert key {b: 1}  
        commit_transaction  
          Apply side write for RID 2 in a single batch. Save cursor. Commit.
      RID 3: delete key {a: 1}    
      commit_transaction    
          Reposition cursor on RID 3. Apply side write for RID 3 in a single batch. Save cursor. Commit
          Reposition cursor on RID 1. Apply side write for RID 1 in a single batch. Commit

      In this scenario, it is possible to insert the key {a: 1} into the index even after it has been deleted.

      The fix for this bug should be to reposition cursors at the beginning of the record store after every yield to guarantee all causally related records are applied in order.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              louis.williams Louis Williams
              Reporter:
              louis.williams Louis Williams
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: