Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31470

Support having resmoke.py run resource intensive tests within a test suite individually

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.0-rc1
    • Component/s: Testing Infrastructure
    • Labels:
      None

      Description

      The changes from SERVER-23111 introduced a serial_run.yml test suite to run tests in a jstests/serial_run/ directory with --jobs=1. This is undesirable because (1) moving tests to run in the serial_run.yml test suite may cause coverage from test reuse in different configurations to be lost and (2) moving tests to the serial_run.yml test suite disrupts the ability for the directory structure to convey the functional area covered by those tests. As an alternative, it would be useful if we could set the following properties on a resmokelib.testing.Suite instance

      • fail_fast - whether to continue on test failure or not
      • include_tags - whether to run certain kinds of tests or not (e.g. unreliable, resource intensive, etc.)
      • num_jobs - the amount of concurrency to run certain kinds of tests with
      • num_repeats - the number of repetitions to run certain kinds of tests with
      • report_failure_status - whether to mark tests as "fail" or "silentfail"

      to capture the settings needed to run (or not run) unreliable, resource intensive tests. We could then have another script invoke resmoke.py for each test suite to run a version that

      • includes all unreliable AND resource intensive tests
      • includes all unreliable AND non-resource intensive tests
      • includes all non-unreliable AND resource intensive tests
      • includes all non-unreliable AND non-resource intensive tests

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              max.hirschhorn Max Hirschhorn
              Reporter:
              max.hirschhorn Max Hirschhorn
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: