We currently make pass/fail decisions in the compare.py script when trying to compare to performance results. As it turned out, this is an inflexible solution considering the threshold needed for different tests varies.

The solution is to not make that decision in compare.py script, but run the regression analysis script after the ratios are calculated by compare.py.