-
Type: Technical Debt
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:
-
5
-
StorEng - Defined Pipeline, Storage Engines - 2022-12-12
Context
Currently when a test/format failure occurs on the evergreen mainline build, a BFG is automatically created which describes the test failure. The test triage baron will then look at the BFG and link it to an existing test/format failure. The linked BFG is accessed through here:
Developers that try to fix a test/format failure would often look at the BFG's test/format configs and try to reproduce the failure as the first step. As part of trying to reproduce the failure, the developer would look at each BFG's tes/tformat config and decide which config is best at reproducing the format failure.
Ticket
This ticket aims to create a script to automate the process of finding the best config reproducer out of all the BFGs. The script will grab the configs from each BFG, and attempt to reproduce the test/failure. The end result should be that we find the best config to reproduce the failure.
The script should look at all the available test/format failures in Jira with test/format name and attempt to reproduce the problem. The script will run on a host machine.
The developer will need to investigate how this can be done on a basic level.
Definition of Done
A basic script that finds the best config out of all the BFGs on a test/format failure ticket.