Extend disagg Python hook to publish schema ops and advance schema epoch

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines - Foundations
    • 217.327
    • None
    • None

      Background

      test/suite/hooks/hook_disagg.py rewrites table: creates into layered disaggregated tables so the existing Python suite runs against disagg storage. It does not call session.publish and never advances stable_disaggregated_schema_epoch, so schema epochs stay at WT_SCHEMA_EPOCH_NONE and the publish/schema-epoch code path is effectively disabled. That path is only exercised by the bespoke csuite test (WT-17972).

      Proposal

      Extend the disagg hook (behind a new parameter, e.g. --hook 'disagg=(publish=true)') so the whole suite drives the publish/schema-epoch path:

      • On session.create of a layered table: after the create, assign a monotonically increasing schema epoch and call session.publish(uri, "disaggregated=(schema_epoch=N)") inline.
      • On session.drop of a layered table: after the drop, assign the next epoch and publish the removal.
      • Advance stable_disaggregated_schema_epoch on session.checkpoint to the current epoch counter, so published schema operations become durable-eligible.

      Advancing the watermark on checkpoint (rather than only in set_timestamp) is deliberate: many tests never call set_timestamp, so coupling the watermark to set_timestamp alone would leave their published schemas permanently deferred and could trip the "stable data checkpointed for unpublished table" invariant.

      Scope and non-goals

      Only table:/layered: URIs that the hook marks layered are published; indices, column groups, LSM, and column-store tables are skipped. Reopen/recovery tests need the epoch counter to reset coherently.

      This complements, and does not replace, the csuite schema_disagg_leader_abort test: the hook gives broad API coverage of the publish/schema-epoch path across many workloads, but single-threaded and without the crash/SIGKILL and concurrency-contention coverage the csuite test provides.

      This is also a natural place to enable the strict_checkpoint_metadata validation from WT-18029 once available.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: