[SERVER-42228] LoggerRuntimeConfigError exceptions can lead to background dbhash thread running until Evergreen task times out Created: 13/Jul/19 Updated: 29/Oct/23 Resolved: 16/Jul/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.14, 4.0.11, 4.2.0-rc3, 4.3.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Max Hirschhorn | Assignee: | Max Hirschhorn |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | tig-resmoke | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
|||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | |||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | |||||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.2, v4.0, v3.6
|
|||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: | Apply the following patch and run a test suite which uses the CheckReplDBHashInBackground hook:
|
|||||||||||||||||||||||||||||||||||||||
| Sprint: | STM 2019-07-29 | |||||||||||||||||||||||||||||||||||||||
| Participants: | ||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 11 | |||||||||||||||||||||||||||||||||||||||
| Story Points: | 2 | |||||||||||||||||||||||||||||||||||||||
| Description |
|
If TestReport.startTest() raises an exception, then Job._run_hooks_after_tests() won't be called. For test suites which use the CheckReplDBHashInBackground hook, this leads to the background thread continuing to spawn mongo shell processes and running the run_check_repl_dbhash_background.js hook. If logkeeper is overwhelmed, then an errors.LoggerRuntimeConfigError exception can also occur when attempting to tear the fixture down. This leads the Evergreen task to time out instead of failing with code 75 because the background dbhash check will continue to run while the fixture is still running and resmoke.py's flush thread will therefore never exit. We don't want to always run the after_test() method for a hook though. For example, if running a test crashes the server, then we shouldn't attempt to run any data consistency checks because they'll just fail to connect to the downed server. |
| Comments |
| Comment by Githook User [ 16/Jul/19 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: (cherry picked from commit c8259b383f8e6f9ce9229774881a55c8dc41e222) |
| Comment by Githook User [ 16/Jul/19 ] |
|
Author: {'name': 'Max Hirschhorn', 'username': 'visemet', 'email': 'max.hirschhorn@mongodb.com'}Message: (cherry picked from commit c8259b383f8e6f9ce9229774881a55c8dc41e222) |
| Comment by Githook User [ 16/Jul/19 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: (cherry picked from commit c8259b383f8e6f9ce9229774881a55c8dc41e222) |
| Comment by Githook User [ 16/Jul/19 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: |