Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31470

Support having resmoke.py run resource intensive tests within a test suite individually

    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.6.0-rc1
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • Labels:
    • Fully Compatible
    • TIG 2017-10-23

      The changes from SERVER-23111 introduced a serial_run.yml test suite to run tests in a jstests/serial_run/ directory with --jobs=1. This is undesirable because (1) moving tests to run in the serial_run.yml test suite may cause coverage from test reuse in different configurations to be lost and (2) moving tests to the serial_run.yml test suite disrupts the ability for the directory structure to convey the functional area covered by those tests. As an alternative, it would be useful if we could set the following properties on a resmokelib.testing.Suite instance

      • fail_fast - whether to continue on test failure or not
      • include_tags - whether to run certain kinds of tests or not (e.g. unreliable, resource intensive, etc.)
      • num_jobs - the amount of concurrency to run certain kinds of tests with
      • num_repeats - the number of repetitions to run certain kinds of tests with
      • report_failure_status - whether to mark tests as "fail" or "silentfail"

      to capture the settings needed to run (or not run) unreliable, resource intensive tests. We could then have another script invoke resmoke.py for each test suite to run a version that

      • includes all unreliable AND resource intensive tests
      • includes all unreliable AND non-resource intensive tests
      • includes all non-unreliable AND resource intensive tests
      • includes all non-unreliable AND non-resource intensive tests

            max.hirschhorn@mongodb.com Max Hirschhorn
            max.hirschhorn@mongodb.com Max Hirschhorn
            0 Vote for this issue
            3 Start watching this issue