Follower reads stale btree ID after leader create/drop/create of a layered table

XMLWordPrintableJSON

      Summary

      On a disaggregated follower, a leader create -> drop -> create of the same layered table leaves the follower with a corrupt table handle. On checkpoint pick-up, reads of the recreated table fail with a bare WT_ERROR.

      Hits a pure follower: one that only picks up checkpoints and never publishes its own schema operations, so pickup relies entirely on diffing shared against local metadata. This is the realistic replica topology.

      Impact

      Ordinary drop-and-recreate of a namespace poisons every pure follower:

      • Best case: block manager catches the btree-ID mismatch and returns WT_ERROR; the recreated table is unreadable on the follower until it is rebuilt from scratch.
      • Worse case: on a read path with looser validation, the follower serves the old table's data under the new table's name with no error, a silent correctness violation.

      MongoDB drops and recreates namespaces routinely, so this affects data integrity on followers in the field.

      Root cause

      Two gaps in __disagg_apply_checkpoint_meta (src/conn/conn_layered_checkpoint_pick_up.c) combine on the same table name:

      1. Drop is a no-op on the follower (FIXME-WT-17746). The dropped table survives in local metadata with its original btree ID; the pickup drop branch is empty.
      2. Recreate reuses the stale entry (FIXME-WT-14730). Pickup copies the leader's new checkpoint address onto the stale entry but keeps the old id=, so the handle opens as the old btree while following the recreated btree's checkpoint.

      Reproduction

      Attached: test_layered_schema14.py. Asserts correct behavior, so it fails today and passes once the gaps are fixed.

      Fix

      No specific fix is needed. The corruption cannot happen silently: the recreated table arrives with a different btree id, and the unconditional id comparison panics during the pickup before either handle is opened. There is no correctness bug left to fix.

      Refactoring: This check had been shadowed in diagnostic builds. This ticket corrects it.

        1. test_layered_schema14.py
          6 kB
          Jie Chen
        2. test_layered_schema15_no_epoch.py
          6 kB
          Jie Chen
        3. wt18068_single_pickup_repro.py
          5 kB
          Jie Chen

            Assignee:
            Alex Blekhman
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: