Forward the active test hooks to the subprocess in run_subprocess_function

XMLWordPrintableJSON

    • Type: Technical Debt
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Test Python
    • None
    • Storage Engines, Storage Engines - Persistence
    • 23.9
    • SE Persistence backlog
    • None

      Issue Summary

      suite_subprocess.run_subprocess_function() re-invokes test/suite/run.py in a child process, but it does not forward the -hook options the parent run was started with. A test that runs under -hook disagg therefore has its subprocess write a plain, local database. Depending on what the test does afterwards this is either a silent loss of coverage or an outright failure, and today it is worked around per test with skip_for_hook.

      # test/suite/helpers/suite_subprocess.py
      procargs = [ sys.executable, runscript, '-p', '--dir', directory,
          *scenario_args, funcname]
      

      Context

      • Eleven tests call run_subprocess_function: test_bug018, test_schema09, test_key_provider_disagg02, test_checkpoint_crash01, and the test_layered_* family, plus a helper in test/suite/helpers/helper_disagg.py.
      • Two distinct symptoms:
        • Silent — under -hook parallel_checkpoint or -hook timestamp the subprocess simply runs without the hook. The test passes while exercising none of what the hook was meant to exercise.
        • Loud — under --hook disagg the parent's own connection is still hooked, because hook_disagg.py installs a HOOK_REPLACE on wiredtiger_open. A test that crashes in the subprocess and then reopens the resulting home directory in the parent opens a local database with disaggregated configuration, and fails.
      • test_checkpoint_crash01.py (added by WT-18392) hit the loud case and carries @wttest.skip_for_hook("disagg", "the subprocess does not inherit the hook") as a result.
      • Note that disaggregated storage itself has no trouble crashing a checkpoint in a subprocess — test_key_provider_disagg02.py does exactly that and passes. It escapes the problem only because it is disaggregated by construction (@disagg_test_class), so its configuration travels into the subprocess through its own conn_config rather than through the hook.

      Proposed Solution

      • Forward the active hooks from run_subprocess_function by appending --hook <name> for each entry the parent was started with.
      • WiredTigerHookManager currently discards hook arguments: _init_ splits "somename=arg" and stores only the name in self.hook_names, which is what WiredTigerTestCase.hook_names exposes. Every hook's initialize(arg) accepts an argument, so forwarding names alone would silently change behaviour for any --hook name=arg invocation. Retain the unsplit specifications alongside the names and forward those.
      • Once hooks reach the subprocess they can skip the function there. A skipped test exits zero and removes its home directory, which is indistinguishable from a pass to the caller, so the subprocess needs a way to report the skip and the caller needs to turn it into a skip of its own.
      • Add a test that has the subprocess report the hooks it was started with and compares them against the parent's, covering the =arg form.

      Scope note: this does not remove the skip_for_hook markers

      Forwarding hooks is necessary but not sufficient to let the affected tests run under -hook disagg. The disagg hook decides which uris are layered at Session.create time and records that in a per-test-case set (testcase.layered_uris in hook_disagg.py), so a table the subprocess creates is not known to be layered in the parent. The parent's open_cursor("table:...") is then not rewritten to layered:... and fails. This is FIXMEWT-16920 in hook_disagg.py: the mapping needs to be keyed by home directory rather than by test case.

      Measured on test_checkpoint_crash01 with hook forwarding in place and its skip_for_hook removed: all eight crash scenarios error in the parent at session.open_cursor(self.uri). So skip_for_hook markers that exist for this reason stay until FIXME-WT-16920 is addressed, and the acceptance signal for this ticket is the new hook-forwarding test rather than the removal of any skip.

      Definition of Done

      • run_subprocess_function forwards every active hook, including any =arg suffix, to the subprocess.
      • A subprocess function that a hook skips results in a skipped test rather than a spurious pass.
      • A test asserts that the subprocess runs under the same hook specifications as its parent, exercised under at least one hook that takes an argument.
      • The existing run_subprocess_function callers show no new failures under -hook disagg, hook parallel_checkpoint and -hook timestamp.

            Assignee:
            Etienne Petrel
            Reporter:
            Etienne Petrel
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: