Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-63729

SPIKE: generic task distributor

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • None
    • None
    • None
    • Dev Platform 2022-03-21

    Description

      We would like to investigate a simple and generic task distributor that can accept and run tasks generated programmatically in real-time and return outputs to the caller. The general idea is that it should function somewhat similarly to Icecream, which is able to schedule and run compiler tasks in real-time on remote hosts and return object files; however, we would like a new system which is not necessarily specific to compiler and/or linker output (such as unit test runs).

      We do not want to replace Evergreen here. Evergreen works very well as a first-pass task distributor where the dependencies are knowable at task run time. What we want is a task distributor that can handle dynamic tasks, such as those generated by a build system. (In other words, Evergreen runs the build system as a first-pass task, and the build system can redistribute compile/link/test tasks after determining build task dependencies.) Another difference with Evergreen is that we want real-time and guaranteed-issue for the new system, similar to how Icecream works.

      Attachments

        Activity

          People

            ryan.egesdahl@mongodb.com Ryan Egesdahl (Inactive)
            iryna.zhuravlova@mongodb.com Iryna Zhuravlova
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: