Leader records stable data for unpublished tables in shared metadata (follower visibility violation)

XMLWordPrintableJSON

    • Storage Engines - Foundations
    • 240.076
    • None
    • None

      Context

      On the disaggregated-storage leader, schema ops (CREATE/DROP) and stable-data UPDATEs are enqueued into the shared metadata queue, applied at the next checkpoint, and replicated to followers. Each entry carries a schema_epoch; an unpublished CREATE sits at WT_SCHEMA_EPOCH_UNPUBLISHED (WT_TS_MAX).

      The bug

      For an UPDATE recording a table's stable checkpointed state, the table's CREATE must be applied at or before the UPDATE's schema epoch. If a CREATE entry is still queued with a schema epoch ahead of the UPDATE's (including the unpublished WT_TS_MAX), the table is not visible to followers before the update lands, leaving stable data in an unpublished table. That breaks the API contract: a table must be published before the checkpoint that includes its data. On the branch this surfaced as the "stable data in unpublished table" invariant violation.

      The fix

      Before enqueuing an UPDATE, __disagg_update_gated_by_unpublished_create (src/conn/conn_layered.c) scans the queue for a CREATE of the same table at a higher schema_epoch; if found, the UPDATE is skipped and re-enqueued at a valid epoch by a later checkpoint once the table is published. A WT_ASSERT at apply time enforces the invariant.

      Committed on the branch under WT-17972: f4815cb41c, 52eb00c, 203a084.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: