[SERVER-29192] resmoke.py shouldn't use "silentfail" test status for dynamic tests Created: 15/May/17  Updated: 30/Oct/23  Resolved: 05/Jun/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.5.9

Type: Improvement Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Yves Duhem
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: TIG 2017-06-19
Participants:

 Description   

Tests that are started dynamically by resmoke.py tend to perform data consistency checks. Failures when running these "hooks" represent potential data corruption issues and shouldn't be caused by a test being unable to run reliably.

The combine_reports.py script has special logic to treat silent failures in dynamic tests as real failures, but the logic would be simpler if the report.json file never contained silent failures for those tests in the first place. This also has the advantage of making it so that test failures which cause the task to fail are displayed in the Evergreen sidebar using #d9534f rather than #e7908e.



 Comments   
Comment by Githook User [ 05/Jun/17 ]

Author:

{u'username': u'syev', u'name': u'Yves Duhem', u'email': u'yves.duhem@mongodb.com'}

Message: SERVER-29192 Prevent 'silentfail' status for dynamic tests
Branch: master
https://github.com/mongodb/mongo/commit/0347baa06a6040af11752047fb8503c77c71c61a

Comment by Yves Duhem [ 30/May/17 ]

eddie.louie already fixed the issue mentioned in the comment above (tests with "fail" status not being converted) in 50ea467 as part of the work on SERVER-29061.

Comment by Max Hirschhorn [ 15/May/17 ]

The changes from abeacd9 as part of SERVER-28439 to TestReport.as_dict() didn't actually convert tests with the "fail" test status to the _config.REPORT_FAILURE_STATUS test status instead, so the work done for this ticket should also end up addressing that issue. (More specifically, tests with the "fail' test status should be converted to the _config.REPORT_FAILURE_STATUS test status when the dynamic attribute is false.

Generated at Thu Feb 08 04:20:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.