Apply the disaggregated checkpoint pickup's local metadata updates in a single transaction

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Layered Tables
    • None
    • Storage Engines - Transactions
    • 58.887
    • SE Transactions - 2026-08-28
    • 3

      The checkpoint pickup applies its local metadata updates one autocommit at a time - the checkpoint entry, then one merge per layered table - with no compensation path. A failure partway leaves a partially adopted checkpoint: tables already merged resolve to the new checkpoint while the published LSN still admits only the old one, and a stable cursor bound in that window can read content its transaction snapshot cannot exclude. Since WT-18156 such failures panic (the restarted node deletes local state and adopts from scratch), which also makes transient conflicts with concurrent metadata writers - for example replicated DDL - fatal.

      Apply the local metadata updates in a single transaction instead, so that a failure rolls back to a clean state:

      • a metadata write conflict then becomes a clean, retryable failure (normalized to EBUSY for the step-up retry loop, restoring the retry-while-blocked behavior WT-18156 originally had) rather than a panic;
      • the missing ingest table creations need to be hoisted out of the merge loop first, since table creation cannot run inside a transaction; they are idempotent and harmless when partial;
      • consider a fault-injection hook (timing stress or a debug configuration) so the failure path can be exercised by tests.

      See the FIXME referencing this ticket at the fatal boundary in __disagg_pick_up_checkpoint (conn_layered_checkpoint_pick_up.c).

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Chenhao Qu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: