[SERVER-53058] Better account of CleanEveryN runtime when setting dynamic timeouts Created: 24/Nov/20  Updated: 29/Oct/23  Resolved: 03/Dec/20

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.9.0, 4.4.3

Type: Bug Priority: Major - P3
Reporter: David Bradford (Inactive) Assignee: David Bradford (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
duplicates SERVER-48698 burn_in_tests can exceed time budget ... Closed
Problem/Incident
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Sprint: DAG 2020-12-14
Participants:
Linked BF Score: 50
Story Points: 3

 Description   

When setting dynamic timeouts, we look at the historic runtime of a test/task. The runtime of any hooks that are run will be included with the test they are associated with. This works pretty well, except for the "CleanEveryN' hook which is run every Nth test (usually 20). The problem with this hook is that it has a relatively large runtime and it can be associated with different tests on different runs. This sometimes results in sub-suites containing tests with no "CleanEveryN" runtimes associated with them that will suddenly start having "CleanEveryN" run. And it quickly eats up the padding we put in place for timeouts.

We need to be smarter about how we include the "CleanEveryN" runtimes when determining timeouts to avoid this problem.



 Comments   
Comment by Githook User [ 03/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53058: Better account for CleanEveryN runtime for dynamically split tasks

(cherry picked from commit 3cb926298aaa7b55d4e0f51daf7c176b65ccc32b)
Branch: v4.4
https://github.com/mongodb/mongo/commit/59b50edbde7b2c9a926d5890f767387330fbadab

Comment by Githook User [ 03/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53058: Refactor how historic test stats are collected

(cherry picked from commit 8fefa30a34e5fb6d47daea312280cd09a6d8967c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/fb67f77bd073a31d0511f9b60b7850df64b1321b

Comment by Githook User [ 02/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53058: Better account for CleanEveryN runtime for dynamically split tasks
Branch: master
https://github.com/mongodb/mongo/commit/3cb926298aaa7b55d4e0f51daf7c176b65ccc32b

Comment by Githook User [ 30/Nov/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53058: Refactor how historic test stats are collected
Branch: master
https://github.com/mongodb/mongo/commit/8fefa30a34e5fb6d47daea312280cd09a6d8967c

Generated at Thu Feb 08 05:29:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.