[SERVER-73708] Filter out excluded tests when calculating evergreen timeouts Created: 07/Feb/23  Updated: 29/Oct/23  Resolved: 17/Apr/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Mikhail Shchatko Assignee: Mikhail Shchatko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: DAG 2023-04-17
Participants:
Story Points: 0

 Description   

We are using test_flags build variant expansion to exclude tests from running on specific variants, e.g. https://github.com/mongodb/mongo/blob/713c60157fea0cab9da39ff90a8d0a8eb6583a1b/etc/evergreen.yml#L931-L934

Those tests are perfectly excluded from running in resmoke suite. However when we are calculating the evergreen timeout, the list of tests that we get there contains those tests
https://github.com/mongodb/mongo/blob/713c60157fea0cab9da39ff90a8d0a8eb6583a1b/buildscripts/timeouts/timeout_service.py#L56

Since the test is excluded from running there will be no historic runtime data for such test. It makes evergreen timeout calculation think that we are missing historic data and avoid task timeout calculation.

AC:

  • Accomodate test exclusion during evergreen timeout calculation.
    We are instantiating resmoke proxy here. Maybe we can pass in resmoke test exclusion flags there, so the test discovery will be working the same way as it works during the resmoke suite run.
  • Verify and if possible remove default timeouts for replica_sets_jscore_passthrough task from evergreen_timeouts.yml
  • File a new ticket where we list the tests to exclude

 

Timebox: 2 days



 Comments   
Comment by Githook User [ 17/Apr/23 ]

Author:

{'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}

Message: SERVER-73708 Filter out excluded tests when calculating evergreen timeouts
Branch: master
https://github.com/mongodb/mongo/commit/8e004a1aa0450a52a7a5665f5f6d47b131128cfc

Generated at Thu Feb 08 06:25:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.