[SERVER-53196] Fail to generate tasks if a large distro is specified but not available Created: 02/Dec/20  Updated: 29/Oct/23  Resolved: 07/Dec/20

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.9.0, 4.4.3, 4.2.13

Type: Bug Priority: Major - P3
Reporter: David Bradford (Inactive) Assignee: David Bradford (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2
Sprint: DAG 2020-12-14
Participants:
Linked BF Score: 50
Story Points: 2

 Description   

Certain test suites need to be run on "large" distros. These distros have extra memory needed to support certain configurations that are being tested. When we dynamically split tasks, there are two attributes used to determine if a large distro is needed. First, the task being split needs to have an expansion "use_large_distro" set in order to indicate that sub-tasks should be configured to use a large distro. Second, the build variant needs to have an expansion called "large_distro_name" to indicate what distro should be used as the "large" distro.

If a build variant does not specify the "large_distro_name" it will not be used for any of the generated tasks regardless of whether the tasks have the "use_large_distro" expansion set. Not all build variants need this set as some do not run any generated tasks that would require it. And some build variants are against configurations that do not even have the concept of a large distro. However, it is really easy to create a new build variant that should specify a large distro to use, but not. The only indication that the large distro name is needed is that tests may unreliable hit out of memory issues.

In order to handle this problem, we should fail to generate any tasks that set "use_large_distro" that are running on a build variant that does not have the "large_distro_name" expansion. We will also create a new config file, "etc/generate_subtasks_config.yml" that will contain a list of exception build variants. If we do fail due to this situation, a detailed error message should be displayed describing the situation and the options to resolve it (add a large_distro_name or include this build variant in the exception list).

As part of this ticket, we should also update existing build variants to conform to the specified convention.



 Comments   
Comment by Githook User [ 03/Feb/21 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53196: Fail to generate tasks if a large distro is required but not specified

(cherry picked from commit f946a64ec3cadae717759479ddeaea368c5a6b1c)
(cherry picked from commit 2e49448cbf41df0c8b75a6ad067459e15f31902a)
Branch: v4.2
https://github.com/mongodb/mongo/commit/41436d006e411259e4476d27be51208b02a9d974

Comment by Githook User [ 07/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53196: Fail to generate tasks if a large distro is required but not specified

(cherry picked from commit f946a64ec3cadae717759479ddeaea368c5a6b1c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/2e49448cbf41df0c8b75a6ad067459e15f31902a

Comment by Githook User [ 05/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53196: Adding missing large distro names
Branch: master
https://github.com/mongodb/mongo/commit/9dd51095f6ab6408af01656f7fa3f54da2661f4d

Comment by Githook User [ 04/Dec/20 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53196: Fail to generate tasks if a large distro is required but not specified
Branch: master
https://github.com/mongodb/mongo/commit/f946a64ec3cadae717759479ddeaea368c5a6b1c

Generated at Thu Feb 08 05:30:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.