[SERVER-56921] Task generation should depend on the completion of compile_dist_test Created: 13/May/21  Updated: 07/Jul/21  Resolved: 07/Jul/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: David Bradford (Inactive)
Resolution: Duplicate Votes: 0
Labels: tig-diag-patch-build, tig-evgconfig
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-56922 Task generation should depend on the ... Closed
Duplicate
is duplicated by SERVER-57003 Generate all tasks for a build varian... Closed
Related
related to SERVER-57003 Generate all tasks for a build varian... Closed
Sprint: DAG 2021-06-28
Participants:
Story Points: 2

 Description   

If task generation is not constrained to depend on the completion of compile_dist_test, then the task generation tasks can end up queued in front of compile_dist_test. That's essentially a priority inversion, since the tasks generated by the task generation tasks can't actually execute until after archive_dist_test_[debug] runs, and that task depends on compile_dist_test. Worse, other tasks which depend on compile_dist_test, like compile_unittests, are also delayed if task generation preempts compile_dist_test, despite the fact that they could run in parallel with task generation, since the generated tasks don't depend on those artifacts.

Another factor to consider is that if compile_dist_test fails, then all of the task generation work was entirely wasted. While that may not be that common an occurrence in the commit queue or waterfall, it is a very common occurrence in patch builds.

The best way to address this would be to make task generation depend on the start of archive_dist_test[_debug]. That way, task generation would run in parallel with archiving the results of compile_dist_test. If compile_dist_test fails, then archive_dist_test will never start and no task generation will happen. If compile_dist_test passes, then task generation would proceed in parallel with archiving up the results if compile_dist_test, and that task has a very low failure rate, so there is little chance of wasted work. Unfortunately, we can't make task generation wait on the start of a task until we have EVG-14537.

Since we don't currently have the ability to make task generation wait on the start of archive_dist_test[_debug], we should do the next best thing, and make it wait on the completion of compile_dist_test, which is very nearly the same thing.



 Comments   
Comment by David Bradford (Inactive) [ 07/Jul/21 ]

With the changes to SERVER-57003, the situation described in this tickets goes away. We changed how task generation works, so only a single task in each build variant will do the task generation. The single "*gen" still exist and are needed to activate the generated tasks, but they have all been marked as dependent on `archive_dist_test[debug]`.

Comment by Maria van Keulen [ 11/Jun/21 ]

david.bradford Could we consider blocking this on SERVER-57003, to reduce the number of moving parts in flight here? IIRC, we were leaning towards running generators on all commits without scheduling the generated tasks in order to accommodate stepback in this new single-generator approach. If the generators must run only after compile, the UX for 'inactive' commits would be tricky and we'd end up consuming an even larger amount of host resources by default. I see we're now doing a scope + design for this, so perhaps the approach might change, but could we finalize the generator story before proceeding with the dependency change? CC brian.samek annie.black

Generated at Thu Feb 08 05:40:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.