Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33075

Evergreen tasks with tests that have only "silentfail" status are still marked as failure

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major - P3 Major - P3
    • None
    • 3.6.0-rc1
    • Testing Infrastructure
    • Decision Automation Group
    • ALL
    • Hide

      Apply the following patch to pretend jstests/core/all.js is unreliable and have it fail.

      python buildscripts/evergreen_run_tests.py --suites=core jstests/core/all.js --tagFile=etc/test_lifecycle.yml --reportFile=report.json --patchBuild
      echo $?
      

      diff --git a/buildscripts/resmokeconfig/suites/core.yml b/buildscripts/resmokeconfig/suites/core.yml
      index ae4b60dc3e..c336731195 100644
      --- a/buildscripts/resmokeconfig/suites/core.yml
      +++ b/buildscripts/resmokeconfig/suites/core.yml
      @@ -8,7 +8,9 @@ executor:
         config:
           shell_options:
             readMode: commands
      -      eval: load("jstests/libs/override_methods/detect_spawning_own_mongod.js");
      +      eval: >-
      +        load("jstests/libs/override_methods/detect_spawning_own_mongod.js");
      +        throw new Error("Intentionally thrown");
         hooks:
         - class: ValidateCollections
           shell_options:
      diff --git a/etc/test_lifecycle.yml b/etc/test_lifecycle.yml
      index 6a5146c6a0..edb996ca3d 100644
      --- a/etc/test_lifecycle.yml
      +++ b/etc/test_lifecycle.yml
      @@ -1,4 +1,7 @@
       # This file is used to tag JS tests that run under resmoke.py.
       # The content of this file lives in the https://github.com/mongodb/mongo-test-metadata repository.
       # It is fetched automatically as part of the compile task.
      -selector: {}
      +selector:
      +  js_test:
      +    jstests/core/all.js:
      +    - unreliable
      

      Show
      Apply the following patch to pretend jstests/core/all.js is unreliable and have it fail. python buildscripts/evergreen_run_tests.py --suites=core jstests/core/all.js --tagFile=etc/test_lifecycle.yml --reportFile=report.json --patchBuild echo $? diff --git a/buildscripts/resmokeconfig/suites/core.yml b/buildscripts/resmokeconfig/suites/core.yml index ae4b60dc3e..c336731195 100644 --- a/buildscripts/resmokeconfig/suites/core.yml +++ b/buildscripts/resmokeconfig/suites/core.yml @@ -8,7 +8,9 @@ executor: config: shell_options: readMode: commands - eval: load("jstests/libs/override_methods/detect_spawning_own_mongod.js"); + eval: >- + load("jstests/libs/override_methods/detect_spawning_own_mongod.js"); + throw new Error("Intentionally thrown"); hooks: - class: ValidateCollections shell_options: diff --git a/etc/test_lifecycle.yml b/etc/test_lifecycle.yml index 6a5146c6a0..edb996ca3d 100644 --- a/etc/test_lifecycle.yml +++ b/etc/test_lifecycle.yml @@ -1,4 +1,7 @@ # This file is used to tag JS tests that run under resmoke.py. # The content of this file lives in the https://github.com/mongodb/mongo-test-metadata repository. # It is fetched automatically as part of the compile task. -selector: {} +selector: + js_test: + jstests/core/all.js: + - unreliable

    Description

      This issue was introduced by my changes from 046a5a0 as part of SERVER-31470 because resmoke.py exits with a non-zero return code if any tests failed (regardless of the evergreen_status it'll use in the report.json file).

      Attachments

        Activity

          People

            backlog-server-dag [DO NOT ASSIGN] Backlog - Decision Automation Group (DAG) (Inactive)
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: