resmoke.py shouldn't use "silentfail" test status for dynamic tests

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Major - P3
    • 3.5.9
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • None
    • Fully Compatible
    • TIG 2017-06-19
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Tests that are started dynamically by resmoke.py tend to perform data consistency checks. Failures when running these "hooks" represent potential data corruption issues and shouldn't be caused by a test being unable to run reliably.

      The combine_reports.py script has special logic to treat silent failures in dynamic tests as real failures, but the logic would be simpler if the report.json file never contained silent failures for those tests in the first place. This also has the advantage of making it so that test failures which cause the task to fail are displayed in the Evergreen sidebar using #d9534f rather than #e7908e.

            Assignee:
            Yves Duhem (Inactive)
            Reporter:
            Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: