Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31470

Support having resmoke.py run resource intensive tests within a test suite individually

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 3.6.0-rc1
    • Testing Infrastructure
    • None
    • Fully Compatible
    • TIG 2017-10-23

    Description

      The changes from SERVER-23111 introduced a serial_run.yml test suite to run tests in a jstests/serial_run/ directory with --jobs=1. This is undesirable because (1) moving tests to run in the serial_run.yml test suite may cause coverage from test reuse in different configurations to be lost and (2) moving tests to the serial_run.yml test suite disrupts the ability for the directory structure to convey the functional area covered by those tests. As an alternative, it would be useful if we could set the following properties on a resmokelib.testing.Suite instance

      • fail_fast - whether to continue on test failure or not
      • include_tags - whether to run certain kinds of tests or not (e.g. unreliable, resource intensive, etc.)
      • num_jobs - the amount of concurrency to run certain kinds of tests with
      • num_repeats - the number of repetitions to run certain kinds of tests with
      • report_failure_status - whether to mark tests as "fail" or "silentfail"

      to capture the settings needed to run (or not run) unreliable, resource intensive tests. We could then have another script invoke resmoke.py for each test suite to run a version that

      • includes all unreliable AND resource intensive tests
      • includes all unreliable AND non-resource intensive tests
      • includes all non-unreliable AND resource intensive tests
      • includes all non-unreliable AND non-resource intensive tests

      Attachments

        Issue Links

          Activity

            People

              max.hirschhorn@mongodb.com Max Hirschhorn
              max.hirschhorn@mongodb.com Max Hirschhorn
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: