Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-63729

SPIKE: generic task distributor

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Dev Platform 2022-03-21

      We would like to investigate a simple and generic task distributor that can accept and run tasks generated programmatically in real-time and return outputs to the caller. The general idea is that it should function somewhat similarly to Icecream, which is able to schedule and run compiler tasks in real-time on remote hosts and return object files; however, we would like a new system which is not necessarily specific to compiler and/or linker output (such as unit test runs).

      We do not want to replace Evergreen here. Evergreen works very well as a first-pass task distributor where the dependencies are knowable at task run time. What we want is a task distributor that can handle dynamic tasks, such as those generated by a build system. (In other words, Evergreen runs the build system as a first-pass task, and the build system can redistribute compile/link/test tasks after determining build task dependencies.) Another difference with Evergreen is that we want real-time and guaranteed-issue for the new system, similar to how Icecream works.

            Assignee:
            ryan.egesdahl@mongodb.com Ryan Egesdahl (Inactive)
            Reporter:
            iryna.zhuravlova@mongodb.com Iryna Zhuravlova
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: