Validate local vs shared metadata equivalence on checkpoint pickup (strict_checkpoint_metadata)

XMLWordPrintableJSON

    • Storage Engines - Foundations
    • 706.597
    • SE Foundations - 2026-07-21, SE Foundations - 2026-08-04
    • 3

      Problem

      In disaggregated storage there is no engine-side check that the local metadata and the shared metadata describe the same set of tables. Divergence is only caught indirectly, through queue-processing invariants during checkpoint (for example the "stable data checkpointed for unpublished table" panic) and one-way reconciliation on follower step-up. Neither compares the two metadata tables and fails on an unexplained difference. As a result, tests such as schema_disagg_leader_abort (WT-17972) can only approximate this check in test code, post-recovery and single node, which is weaker than a real engine guarantee.

      Proposal

      Add a disaggregated=(strict_checkpoint_metadata=true) option to checkpoint and/or checkpoint pickup. When set, WiredTiger validates that the local metadata and the shared metadata contain equivalent tables at the picked-up checkpoint:

      • A table present in the local metadata but not in the shared metadata must have an enqueued INSERT metadata update with a schema epoch greater than the checkpoint's schema epoch.
      • A table present in the shared metadata but not in the local metadata must have an enqueued REMOVE metadata update with a schema epoch greater than the checkpoint's schema epoch.

      If either check fails, the difference is unexplained and indicates data corruption, so WiredTiger must panic.

      Benefit

      Deterministic, engine-side corruption detection at checkpoint pickup. Tests can enable the flag to get an immediate panic on divergence instead of relying on hand-rolled, post-hoc verification.

            Assignee:
            Yury Ershov
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: