[SERVER-78727] Update timeout calculator to handle `max_duration_pass ` Created: 06/Jul/23  Updated: 13/Jul/23  Resolved: 10/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc7, 7.0.0-rc8, 4.4.24, 5.0.20, 6.0.9

Type: Improvement Priority: Major - P3
Reporter: Jeffrey Zambory Assignee: Jeffrey Zambory
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.0, v6.0, v5.0, v4.4, v4.2
Sprint: DAG 2023-07-10
Participants:
Linked BF Score: 156
Story Points: 1

 Description   

We currently only track and make available for use in the timeout calculator the average time that a test took to pass recently. This metric is fine to use for most task timeout calculations as tests for the most part are supposed to have a set runtime they adhere to and deviations from that are incorrect.

However, tests such as benchmarks can have a semi-random runtime instead and can vary pretty widely from run to time. This makes it difficult for the task generator to split up benchmark tasks effectively and can cause semi-random timeouts and also task groups  with very uneven wall clock runtimes in them.

To help minimize the risk of these timeouts, we instead want to begin bucketing based upon the recent max runtime of benchmark tests. This should be a better metric to use in regards to these benchmark tests and lead to less of a risk of new BFs that arise and then closed as noise.

To support this, we're exposing a new parameter called `max_duration_pass` as part of the task runtime history and need to update the Python buildscript code in the mongo repo to expect this new parameter. The task generation work will be done in a separate ticket. 



 Comments   
Comment by Jeffrey Zambory [ 13/Jul/23 ]

tommaso.tocci@mongodb.com Sorry about that! Just fixed all the backport tickets and gave them the correct status. Also updated the description to call out what this is doing. 

Comment by Githook User [ 10/Jul/23 ]

Author:

{'name': 'Jeff Zambory', 'email': 'jeff.zambory@mongodb.com', 'username': ''}

Message: SERVER-78727: Update timeout calculator to handle `max_duration_pass`

(cherry picked from commit 8cad9c092d31ed5322e81669655d72d7f0270927)
Branch: v6.0
https://github.com/mongodb/mongo/commit/4407b58d3fb53414ed7d42332a8cfb2f13579111

Comment by Githook User [ 10/Jul/23 ]

Author:

{'name': 'Jeff Zambory', 'email': 'jeff.zambory@mongodb.com', 'username': ''}

Message: SERVER-78727: Update timeout calculator to handle `max_duration_pass`

(cherry picked from commit 8cad9c092d31ed5322e81669655d72d7f0270927)
Branch: v5.0
https://github.com/mongodb/mongo/commit/12a50d3fdfe96ad3d50928e7947a9804c11d4e7f

Comment by Githook User [ 10/Jul/23 ]

Author:

{'name': 'Jeff Zambory', 'email': 'jeff.zambory@mongodb.com', 'username': ''}

Message: SERVER-78727: Update timeout calculator to handle `max_duration_pass`

(cherry picked from commit 8cad9c092d31ed5322e81669655d72d7f0270927)
Branch: v4.4
https://github.com/mongodb/mongo/commit/e960a69e34f42ab72982da1ed2b7313d9811d1d5

Comment by Githook User [ 10/Jul/23 ]

Author:

{'name': 'Jeff Zambory', 'email': 'jeff.zambory@mongodb.com', 'username': ''}

Message: SERVER-78727: Update timeout calculator to handle `max_duration_pass`
Branch: v7.0
https://github.com/mongodb/mongo/commit/02fbd323441c0c047c800f39c5489e0424ab94a8

Comment by Githook User [ 07/Jul/23 ]

Author:

{'name': 'Jeff Zambory', 'email': 'jeff.zambory@mongodb.com', 'username': ''}

Message: SERVER-78727: Update timeout calculator to handle `max_duration_pass`
Branch: master
https://github.com/mongodb/mongo/commit/8cad9c092d31ed5322e81669655d72d7f0270927

Generated at Thu Feb 08 06:39:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.