-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Layered Tables
-
Storage Engines - Foundations
-
22.628
-
None
-
None
We want a diagnostic assertion at layered dhandle open (__wt_schema_open_layered): when a leader cannot find the stable constituent in local metadata, that must be because the table was created inside the step-down window (step-down timestamp set) or the open is racing an in-flight step-up. Any other combination means the flag derivation that routes cursors to the ingest constituent silently loses its signal, so we want to fail fast instead of serving a table with no constituents.
The assertion was implemented on the WT-17091 branch but had to be reverted, because the invariant only holds under the disaggregated deployment contract: every open acknowledges lose_all_my_data, whose default local_files_action=delete wipes local files, so a node never adopts stale local state. Our tests break that contract. test_layered_stepup12 failed deterministically: a step-down scenario creates a table as a follower (legitimately no stable constituent), and the harness teardown reopens the home as a leader over that state, producing a legal-looking assertion violation that cannot be distinguished from corruption at open time.
Making the contract hold in tests is the bulk of the work. Of 239 Python tests using the disagg helpers, only 55 set lose_all_my_data. A blanket default was prototyped and is not viable yet:
- Injecting the flag in wttest.wiredtiger_open took the disagg suites from 1 failure / 2 errors to 81 failures / 24 errors across 19 files. The dominant cause is the harness's own teardown verification, which reopens the database and verifies through local state, exactly what the wipe deletes. Prepared-transaction, page-delta, key-provider and step-up suites also reopen with state mid-test.
- Adding the flag to TESTUTIL_ENV_CONFIG_DISAGG (test_util.h) breaks format with CONFIG.disagg at open (missing turtle file), against a clean baseline.
Proposed work:
- Make wttest teardown disagg-aware: skip the local final verify for disagg scenarios, or verify through a checkpoint_meta pickup instead of local files.
- Audit mid-test reopen patterns in the affected suites; convert local-state handoffs to shared-storage pickups where the test intent allows.
- Fix the format open/reopen flow to survive the wipe.
- Flip the default at the two choke points: wttest.wiredtiger_open (guarded so explicit settings win) and TESTUTIL_ENV_CONFIG_DISAGG. Add the flag individually to the two raw open sites: cppsuite/test_disagg_failover_perf.cpp and model/src/core/util.cpp.
- Re-land the reverted assertion in __wt_schema_open_layered and keep it as the acceptance test: the full disagg suites, format, and the schema_disagg_abort stress must pass with it enabled.
- is related to
-
WT-17091 Investigate and implement step-down for publish
-
- In Code Review
-