Dropping a published but uncheckpointed table discards all of its data

XMLWordPrintableJSON

    • Storage Engines - Foundations
    • 222.49
    • None
    • None

      On a disaggregated leader, dropping a table that has been published but never checkpointed discards all committed data written to the table. The drop closes the stable constituent's dhandle, and __wt_conn_dhandle_close() in src/conn/conn_dhandle.c treats a WT_BTREE_AWAITS_PUBLISH tree as in-memory, taking the discard path instead of returning an error.

      Impact

      • Data loss: A plain WT_SESSION::drop of a published, uncheckpointed table throws away its committed rows. No verify or exclusive open is required.
      • Checkpoint inconsistency: __drop_layered() enqueues the shared metadata REMOVE at WT_SCHEMA_EPOCH_UNPUBLISHED, while the table's CREATE was already published. A checkpoint whose stable schema epoch covers the CREATE but not the unpublished REMOVE includes the table even though its data was already discarded. A follower or replay that picks up that checkpoint sees a published table with no data.

      Root cause

      _wt_btree_stays_in_memory() (src/include/btree.h) returns true for both WT_BTREE_IN_MEMORY and WT_BTREE_AWAITS_PUBLISH. At conn_dhandle.c:479 the close sets discard = true for such a tree and calls _wt_evict_file(session, WT_SYNC_DISCARD). Because an AWAITS_PUBLISH table cannot be checkpointed until it is published, the in-memory pages are the only copy, so the discard is unrecoverable.

      Reproduction

      Deterministic, no verify required.

      1. Create a table under a stable schema epoch, so it carries WT_BTREE_AWAITS_PUBLISH.
      2. Publish it.
      3. Write committed rows.
      4. Drop the table, all without an intervening checkpoint.
      5. Take a checkpoint at a timestamp/epoch that covers the table's CREATE.

      The committed rows are gone, and the checkpoint references a published table with no data.

      Expected fix

      Attempting to drop a published but uncheckpointed table should return EBUSY, forcing a checkpoint (which publishes the table) before the drop proceeds. The EBUSY must be raised before the drop issues its side effects, the leader trim (__drop_issue_trim(), schema_drop.c:197) and the enqueued shared REMOVE (schema_drop.c:200), so a refused drop leaves no partial state.

      Related

      WT-18212 (verify discards data for the same class of table).

            Assignee:
            Jie Chen
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: