[SERVER-31470] Support having resmoke.py run resource intensive tests within a test suite individually Created: 09/Oct/17  Updated: 30/Oct/23  Resolved: 18/Oct/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.6.0-rc1

Type: New Feature Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Max Hirschhorn
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
causes SERVER-33075 Evergreen tasks with tests that have ... Closed
Related
related to SERVER-31579 Tag specific sharding tests as being ... Closed
is related to SERVER-43802 Create ReplSetTest and ShardingTest c... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG 2017-10-23
Participants:

 Description   

The changes from SERVER-23111 introduced a serial_run.yml test suite to run tests in a jstests/serial_run/ directory with --jobs=1. This is undesirable because (1) moving tests to run in the serial_run.yml test suite may cause coverage from test reuse in different configurations to be lost and (2) moving tests to the serial_run.yml test suite disrupts the ability for the directory structure to convey the functional area covered by those tests. As an alternative, it would be useful if we could set the following properties on a resmokelib.testing.Suite instance

  • fail_fast - whether to continue on test failure or not
  • include_tags - whether to run certain kinds of tests or not (e.g. unreliable, resource intensive, etc.)
  • num_jobs - the amount of concurrency to run certain kinds of tests with
  • num_repeats - the number of repetitions to run certain kinds of tests with
  • report_failure_status - whether to mark tests as "fail" or "silentfail"

to capture the settings needed to run (or not run) unreliable, resource intensive tests. We could then have another script invoke resmoke.py for each test suite to run a version that

  • includes all unreliable AND resource intensive tests
  • includes all unreliable AND non-resource intensive tests
  • includes all non-unreliable AND resource intensive tests
  • includes all non-unreliable AND non-resource intensive tests


 Comments   
Comment by Githook User [ 18/Oct/17 ]

Author:

{'email': 'max.hirschhorn@mongodb.com', 'name': 'Max Hirschhorn', 'username': 'visemet'}

Message: SERVER-31470 Move "run tests" logic into evergreen_run_tests.py.
Branch: master
https://github.com/mongodb/mongo/commit/046a5a01c1bc6eeb05852bed9981cbc457802a00

Generated at Thu Feb 08 04:27:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.