mergeSpills silently drops or duplicates elements when a write-conflict retry resets WiredTiger cursor positions

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • ALL
    • Storage Execution 2026-04-27, Storage Execution 2026-05-11
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      During ContainerBasedSpiller::mergeSpills, the merge-then-write loop
      reads from a MergeIterator whose sources are ContainerIterators backed
      by WiredTiger cursors. When an insert triggers a WT_ROLLBACK (write
      conflict), writeConflictRetry calls rollback_transaction() on the
      WiredTiger session. That resets all cursor positions in that session —
      including the read cursors held by the ContainerIterators inside
      mergeIterator.

      On the next retry iteration, mergeIterator resumes from the cursors'
      new (reset) position rather than from where the merge left off. Depending
      on where the reset lands, elements can be silently dropped or duplicated
      in the output spill file.

      *Fix*

      Collect all merged elements into a std::vector before entering the
      write loop. Separating the read phase from the write phase ensures that
      cursor resets during write retries cannot affect elements that have
      already been consumed from the iterator.

      *Steps to reproduce*

      Enable the WTWriteConflictException failpoint with nTimes >= 1 and
      call mergeSpills with multiple spill iterators. The merged output will
      be missing or contain duplicate entries depending on which cursor
      position the rollback resets to.

      *Tests added*

      MergeSpillsRetryOnWriteConflict, MergeSpillsTwoWriteConflictsRetryCount,
      MergeSpillsRandomWriteConflicts in container_based_spiller_test.cpp
      (ContainerBasedSpillerWTTest fixture).

            Assignee:
            Jess Balint
            Reporter:
            Jess Balint
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: