Resmoke selector path validation is a no-op for relative suite paths

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Catalog and Routing
    • ALL
    • CAR Team 2026-08-03, CAR Team 2026-08-17
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Overview

      The selector-path validation introduced by SERVER-120474 (VALIDATE_SELECTOR_PATHS, enabled by default) is currently a global no-op: suite YAMLs can reference nonexistent test paths without any error, allowing silent loss of test coverage.

      Details

      • buildscripts/resmokelib/configure_resmoke.py (~line 1088) sets JSTESTS_DIR to an absolute path (os.path.join(RESMOKE_ROOT, "jstests")).
      • _evaluate_paths in buildscripts/resmokelib/selector.py (~line 255) only flags a nonexistent path as unrecognized if path.startswith(get_jstests_dir()).
      • Suite YAMLs use relative paths (jstests/...), so the prefix check never matches and the validation never fires.

      Verified empirically: resmoke.py run --suites=sharding_smoke_tests --dryRun tests succeeds with zero errors even though three of the suite's roots no longer exist.

      Impact

      • sharding_smoke_tests.yml silently lost 3 roots (jstests/core_sharding/chunk_migration/*.js, moved to chunk_operations/ by SERVER-130568) — that smoke coverage is currently not running.
      • ~15 stale exclude_files / archive.tests entries across suites point at moved/deleted tests (e.g. sharding_auth.yml, core_wildcard_indexes.yml, no_passthrough.yml archive entries, sharding_csrs_continuous_config_stepdown.yml) and are silently ignored.

      Suggested Fix

      Normalize the path prefix comparison (e.g. compare paths relative to the repo root, or make suite paths absolute before the check) in selector.py / configure_resmoke.py, then fix the stale suite entries that the restored validation flags.

            Assignee:
            Paolo Polato
            Reporter:
            Paolo Polato
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: