Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29061

Add buildscripts/promote_silent_failures.py script to convert silent test failures into non-silent test failures

    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.5.9
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • Labels:
      None
    • Fully Compatible
    • TIG 2017-05-29, TIG 2017-06-19

      The buildscripts/promote_silent_failures.py script takes

      1. the path to a single report.json file as a command line argument, as well as
      2. optionally a path to an output filename via a --output-file command line argument (defaults to stdout)

      and writes a report.json file to the specified location.

      Note: This is effectively how the command line parsing for the buildscripts/combine_reports.py script works with the exception that the buildscripts/promote_silent_failures.py script should only accept a single report.json file and error if multiple are specified.

      For any test files that for at least 2 executions in the report.json file have a "silentfail" status, the buildscripts/promote_silent_failures.py script should change the outputted report to have a "fail" status instead.

      Note: The TestReport.from_dict() class method should be used to construct the TestReport instance corresponding to the specified report.json file to avoid manipulating the Python dict structure directly after parsing the JSON file.

            Assignee:
            eddie.louie Eddie Louie
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: