[SERVER-45644] Reevaluate timeouts used by burn_in_test Created: 17/Jan/20 Updated: 29/Oct/23 Resolved: 24/Jan/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.2.4, 4.3.3 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | David Bradford (Inactive) | Assignee: | Alexander Costas (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | tig-burnin | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Backport Requested: |
v4.2
|
||||
| Sprint: | DAG 2020-01-27 | ||||
| Participants: | |||||
| Story Points: | 3 | ||||
| Description |
|
In burn_in_tests, we will dynamically set timeouts based on the expected runtime of the test being run. There have, however, been some issues with the timeouts being used. We should investigate if we can improve the timeout calculations. As a Server engineer, AC:
|
| Comments |
| Comment by Githook User [ 10/Feb/20 ] |
|
Author: {'name': 'Alexander Costas', 'username': 'ancostas', 'email': 'alexander.costas@mongodb.com'}Message: |
| Comment by Githook User [ 24/Jan/20 ] |
|
Author: {'email': 'alexander.costas@mongodb.com', 'username': 'ancostas', 'name': 'Alexander Costas'}Message: |
| Comment by David Bradford (Inactive) [ 23/Jan/20 ] |
|
For the near term, we should probably just add a ~4 minute buffer to the exec timeout for test setup. It could be added to the value returned by calculate_exec_timeout |
| Comment by David Bradford (Inactive) [ 22/Jan/20 ] |
|
I think this might explain what we are seeing . When we set the timeouts in burn_in_tests, we are setting them based on the test execution time. However, the timeouts are for the entire task. We spend some amount of time downloading artifacts and doing setup. If this ends up being larger than the buffer we include in the timeout we will end up timing out. We need someway of including the setup time in the timeouts. Since the setup time can be different across different build variants this could be tricky. |
| Comment by Lydia Stepanek (Inactive) [ 21/Jan/20 ] |
|
Here is another patch in which burn_in_tests had some tests timeout: https://evergreen.mongodb.com/version/5e2749671e2d171abfe245dc |