-
Type:
Technical Debt
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Test Python
-
None
-
Storage Engines, Storage Engines - Persistence
-
23.9
-
SE Persistence backlog
-
None
Issue Summary
suite_subprocess.run_subprocess_function() re-invokes test/suite/run.py in a child process, but it does not forward the -hook options the parent run was started with. A test that runs under -hook disagg therefore has its subprocess write a plain, local database. Depending on what the test does afterwards this is either a silent loss of coverage or an outright failure, and today it is worked around per test with skip_for_hook.
# test/suite/helpers/suite_subprocess.py procargs = [ sys.executable, runscript, '-p', '--dir', directory, *scenario_args, funcname]
Context
- Eleven tests call run_subprocess_function: test_bug018, test_schema09, test_key_provider_disagg02, test_checkpoint_crash01, and the test_layered_* family, plus a helper in test/suite/helpers/helper_disagg.py.
- Two distinct symptoms:
- Silent — under -
hook parallel_checkpointor -hook timestamp the subprocess simply runs without the hook. The test passes while exercising none of what the hook was meant to exercise. - Loud — under --hook disagg the parent's own connection is still hooked, because hook_disagg.py installs a HOOK_REPLACE on wiredtiger_open. A test that crashes in the subprocess and then reopens the resulting home directory in the parent opens a local database with disaggregated configuration, and fails.
- Silent — under -
- test_checkpoint_crash01.py (added by WT-18392) hit the loud case and carries @wttest.skip_for_hook("disagg", "the subprocess does not inherit the hook") as a result.
- Note that disaggregated storage itself has no trouble crashing a checkpoint in a subprocess — test_key_provider_disagg02.py does exactly that and passes. It escapes the problem only because it is disaggregated by construction (@disagg_test_class), so its configuration travels into the subprocess through its own conn_config rather than through the hook.
Proposed Solution
- Forward the active hooks from run_subprocess_function by appending --hook <name> for each entry the parent was started with.
- WiredTigerHookManager currently discards hook arguments: _init_ splits "somename=arg" and stores only the name in self.hook_names, which is what WiredTigerTestCase.hook_names exposes. Every hook's initialize(arg) accepts an argument, so forwarding names alone would silently change behaviour for any --hook name=arg invocation. Retain the unsplit specifications alongside the names and forward those.
- Once hooks reach the subprocess they can skip the function there. A skipped test exits zero and removes its home directory, which is indistinguishable from a pass to the caller, so the subprocess needs a way to report the skip and the caller needs to turn it into a skip of its own.
- Add a test that has the subprocess report the hooks it was started with and compares them against the parent's, covering the =arg form.
Scope note: this does not remove the skip_for_hook markers
Forwarding hooks is necessary but not sufficient to let the affected tests run under -hook disagg. The disagg hook decides which uris are layered at Session.create time and records that in a per-test-case set (testcase.layered_uris in hook_disagg.py), so a table the subprocess creates is not known to be layered in the parent. The parent's open_cursor("table:...") is then not rewritten to layered:... and fails. This is FIXMEWT-16920 in hook_disagg.py: the mapping needs to be keyed by home directory rather than by test case.
Measured on test_checkpoint_crash01 with hook forwarding in place and its skip_for_hook removed: all eight crash scenarios error in the parent at session.open_cursor(self.uri). So skip_for_hook markers that exist for this reason stay until FIXME-WT-16920 is addressed, and the acceptance signal for this ticket is the new hook-forwarding test rather than the removal of any skip.
Definition of Done
- run_subprocess_function forwards every active hook, including any =arg suffix, to the subprocess.
- A subprocess function that a hook skips results in a skipped test rather than a spurious pass.
- A test asserts that the subprocess runs under the same hook specifications as its parent, exercised under at least one hook that takes an argument.
- The existing run_subprocess_function callers show no new failures under -
hook disagg,hook parallel_checkpointand -hook timestamp.