-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Dev Platform 2022-03-21
We would like to investigate a simple and generic task distributor that can accept and run tasks generated programmatically in real-time and return outputs to the caller. The general idea is that it should function somewhat similarly to Icecream, which is able to schedule and run compiler tasks in real-time on remote hosts and return object files; however, we would like a new system which is not necessarily specific to compiler and/or linker output (such as unit test runs).
We do not want to replace Evergreen here. Evergreen works very well as a first-pass task distributor where the dependencies are knowable at task run time. What we want is a task distributor that can handle dynamic tasks, such as those generated by a build system. (In other words, Evergreen runs the build system as a first-pass task, and the build system can redistribute compile/link/test tasks after determining build task dependencies.) Another difference with Evergreen is that we want real-time and guaranteed-issue for the new system, similar to how Icecream works.