Status: Needs Design
Major - P3
Scenario: only one test is failing in 30 suites.
Problem : user needs to perform 60 clicks in the evergreen UI (and open 30 different pages) to simply discover that the same test has failed in several suites
Solution : group failures by test name providing a view <failing test name, [list of suites it fails in]>
Given the current UI, inspecting failures can be extremely user-unfriendly. The minimal way to get to know which test cases have failed is the following:
- The patch page is showing failing suites
- A click on "+" is required next to each failing suite to see a list of failing tasks
- A click on each failing task (entering a new page) is required to see the list of tests failing within that specific task
The scenario mentioned above is over-simplified, as it often happens that different tests are failing, thous requiring the user to note down all failing tests while inspecting plenty of different pages.
This script allows users to avoid performing the process manually, but it would be nice for evergreen itself to provide the feature (also because the script is not tested and doesn't handle corner cases, e.g. network failures or unfinished patches)
- is related to
EVG-15196 Group failures by test name, for a whole patch