-
Type:
Investigation
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
Foliage catches [task × variant] that reach one of the thresholds at a certain point in time, but I suspect this doesn't give us a full picture. Often, singular flakes disappear after a re-run and never really get looked into. But they still contribute to our death by a thousand cuts.
Can we gather data from evergreen over a longer period of time and look for trends that foliage might be missing? We could look at time series to see rate of failure over time. We could look at CI minutes burned by retried task durations. We could see outliers of different groups - not just [task x variant] but for example individual test across all variants, or variants and tasks separately.. (stretch) Potentially even certain errors/keywords that appear often? (/stretch)