[SERVER-24924] Exceptions during resmoke.py execution do not cause task failures Created: 06/Jul/16 Updated: 25/Apr/18 Resolved: 18/Apr/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 3.7.6 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kamran K. | Assignee: | Jonathan Abrahams |
| Resolution: | Done | Votes: | 0 |
| Labels: | tig-resmoke | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Sprint: | TIG 2018-04-23 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
The unittests in this job didn't run because of TypeErrors, but the task still succeeded. We need to investigate why the TypeErrors didn't propagate and cause the task to fail. |
| Comments |
| Comment by Githook User [ 18/Apr/18 ] |
|
Author: {'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}Message: |
| Comment by Githook User [ 18/Apr/18 ] |
|
Author: {'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}Message: |
| Comment by Githook User [ 17/Apr/18 ] |
|
Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'}Message: Revert " This reverts commit 2b20c142c7e15cdad1ef609bcb5f14ba288dacea. |
| Comment by Githook User [ 17/Apr/18 ] |
|
Author: {'name': 'Jonathan Abrahams', 'email': 'jonathan@mongodb.com', 'username': 'hptabster'}Message: |
| Comment by Max Hirschhorn [ 23/Mar/18 ] |
|
I think we should make it so that if the TestReport instance says that no tests were run, but the list of test files (or the list of dbtest suites) was non-empty, then resmoke.py should raise an exception that it expected to run at least N tests but didn't. I don't see a reason why after test selection occurs that there shouldn't be a guarantee we have at least that many tests in the report. |
| Comment by Robert Guo (Inactive) [ 14/Jul/16 ] |
|
Just to clarify the problem: exceptions in child threads are silently ignored. Resmoke currently catches them by having each failure be added explicitly to a test report. This could be problematic if a failure is unexpected and therefore not added to the test report. We want a fail-safe here to ensure that any uncaught exceptions are propagated back to the main thread. This can be done by subclassing the Thread class and checking for exceptions in the parent. Similar to what is done here. Any exception also needs to be added to the report, possibly under a new category of failures, so that --continueOnFailure can ignore it if necessary. |
| Comment by Kamran K. [ 14/Jul/16 ] |
|
Reopening. Our test harness can't silently break like this. |
| Comment by Robert Guo (Inactive) [ 14/Jul/16 ] |
|
After chatting with max.hirschhorn, I'm going to close this as won't fix. There're 5 lines of code that can cause this problem. Making the problem go away will require adding more than 5 lines of code, so its seems to not be worth it. Here's the last of the 5 lines that can cause the problem: https://github.com/mongodb/mongo/blob/r3.3.9/buildscripts/resmokelib/testing/job.py#L82 |