[SERVER-80220] JavaScript tests tagged with in-development feature flags are not being run by burn_in_tests Created: 17/Aug/23  Updated: 02/Nov/23  Resolved: 15/Sep/23

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Mikhail Shchatko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-81001 burn_in_tests does not correctly dete... Closed
Related
is related to SERVER-54420 Add feature flag checker to generated... Closed
is related to SERVER-72928 Expand burn-in-tags to all required &... Closed
Assigned Teams:
Decision Automation Group
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: DAG 2023-09-18
Participants:
Story Points: 2

 Description   

buildscripts/burn_in_tests.py generate-test-membership-map-file-for-ci generates the association of a particular JavaScript test file and all of the resmoke YAML suite files the test would run under. However the decision about whether a test is suitable to run depends on the state of resmokelib.config.EXCLUDE_WITH_ANY_TAGS which in particular is populated by the names of all of the in-development feature flags unless --runAllFeatureFlagTests is specified.

The buildscripts/burn_in_tests.py script does not enable the --runAllFeatureFlagTests mode of resmoke and so these tests are currently never run by burn_in_tests. This defeats a large portion of the use case for burn_in_tests because engineers are writing new tests for their projects under an in-development feature flag and means these JavaScript test files won't be stressed to determine if the tests are unreliable.



 Comments   
Comment by Githook User [ 15/Sep/23 ]

Author:

{'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}

Message: SERVER-80220 Enable burn_in_tests to pick up tests tagged with in-development feature flags
Branch: master
https://github.com/mongodb/mongo/commit/5b8d8a30ba2a22eada4abf7bf4f1e7644c89f13b

Generated at Thu Feb 08 06:42:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.