ExportXMLWordPrintableJSON

    • Storage Engines - Transactions
    • 20.957
    • SE Transactions - 2026-11-06
    • 3

      Problem

      On a follower, a positioned layered cursor can remove a key using a stale cached value and skip the write.

      In one transaction:

      1. Cursor A searches a key that lives in the ingest table.
      2. A removes the key.
      3. Cursor B, in the same session, reinserts the key.
      4. A removes the key again without repositioning.

      The second remove returns WT_NOTFOUND, and after commit the key still has B's value. Expected: the second remove succeeds and the key is gone after commit.

      Reproduction

      Reproduced on develop at f1599c3069 with a Python test on a follower connection:

      a = session.open_cursor(uri)
      b = session.open_cursor(uri)
      session.begin_transaction()
      a.set_key('k')
      a.search()           # 0
      a.remove()           # 0
      b['k'] = 'reinserted'
      a.remove()           # WT_NOTFOUND, expected 0
      session.commit_transaction('commit_timestamp=...')
      # 'k' still has the value 'reinserted'
      

      Cause

      __clayered_remove_from_ingest trusts the current constituent's cached value when the layered cursor is positioned and the value is still held (WT_CURSTD_VALUE_INT). After A's first remove, that cached value is A's own tombstone. B's write does not clear A's value flag, so A treats the key as already removed and returns WT_NOTFOUND without writing.

      WT-18839 fixed the same stale state for iteration by comparing the transaction's modification count, but the remove path does not use that check.

      Scope

      • Follower removes. Verified.
      • A leader outside a step-down removes through stable and is not affected. Verified.
      • Leader writes routed to ingest during a step-down without write mirroring go through the same function. Not tested.
      • With the blind_remove cursor option from WT-18920, the same path returns 0 instead of WT_NOTFOUND, so the skipped remove becomes silent.

      Proposed fix

      Record the transaction's modification count when each layered cursor operation finishes, and let remove trust its cached value only if the count is unchanged. A write through another cursor then forces a fresh lookup, while the cursor's own writes do not.

      Reusing the existing txn_mod_count field does not work. The cursor's own first remove changes it, so a second remove on the same cursor during a walk loses its position. A separate field recorded when the operation finishes avoids that.

      Tests

      Add a Python test that runs the sequence above, commits, and checks the key is absent. Run it with and without blind_remove=true.

      Acceptance criteria

      • The second remove returns 0 and the key is absent after commit, with and without blind_remove.
      • Removing the same key twice through one cursor during a walk still keeps the cursor's position.
      • The new test fails without the fix.

            Assignee:
            Zunyi Liu
            Reporter:
            Zunyi Liu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: