Validate local vs shared metadata equivalence on checkpoint pickup (strict_checkpoint_metadata)

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines - Foundations
    • 217.628
    • SE Foundations - 2026-07-21
    • 3

      Problem

      In disaggregated storage there is no engine-side check that the local metadata and the shared metadata describe the same set of tables. Divergence is only caught indirectly, through queue-processing invariants during checkpoint (for example the "stable data checkpointed for unpublished table" panic) and one-way reconciliation on follower step-up. Neither compares the two metadata tables and fails on an unexplained difference. As a result, tests such as schema_disagg_leader_abort (WT-17972) can only approximate this check in test code, post-recovery and single node, which is weaker than a real engine guarantee.

      Proposal

      Add a disaggregated=(strict_checkpoint_metadata=true) option to checkpoint and/or checkpoint pickup. When set, WiredTiger validates that the local metadata and the shared metadata contain equivalent tables at the picked-up checkpoint:

      • A table present in the local metadata but not in the shared metadata must have an enqueued INSERT metadata update with a schema epoch greater than the checkpoint's schema epoch.
      • A table present in the shared metadata but not in the local metadata must have an enqueued REMOVE metadata update with a schema epoch greater than the checkpoint's schema epoch.

      If either check fails, the difference is unexplained and indicates data corruption, so WiredTiger must panic.

      Benefit

      Deterministic, engine-side corruption detection at checkpoint pickup. Tests can enable the flag to get an immediate panic on divergence instead of relying on hand-rolled, post-hoc verification.

            Assignee:
            Yury Ershov
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: