[SERVER-33963] Use an Evergreen task group for compile-related tasks Created: 18/Mar/18  Updated: 29/Oct/23  Resolved: 27/Apr/18

Status: Closed
Project: Core Server
Component/s: Build, Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.0.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Jonathan Abrahams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Problem/Incident
Related
related to SERVER-34709 The compile_all task runs longer than... Closed
related to SERVER-40034 Set setup_group_can_fail_task to true... Closed
related to SERVER-41262 Decouple compile_all from compile_all... Closed
is related to SERVER-25941 remove library dependency graph gener... Closed
is related to SERVER-31765 Dagger tool has dramatically slowed t... Closed
is related to SERVER-27483 Avoid stripping debug symbols from C+... Closed
is related to SERVER-28644 Add Dagger Project to Evergreen Closed
is related to SERVER-25691 Break out unit test compilation and r... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG 2018-04-09, TIG 2018-04-23, TIG 2018-05-07
Participants:
Linked BF Score: 61

 Description   

We made changes in SERVER-25691 and SERVER-27483 to avoid having compiling, linking, and running all of the C++ unit tests from having a signficant impact on the makespan of a full patch build by reducing the amount of time the compile task takes. Switching to represent these units of work as part of an Evergreen task group should save EC2 machine time (not necessarily wall-clock time) because we'll be able to reuse the build artifacts from the machine's local SCons cache (which should be even faster than the shared SCons cache) and avoid duplicate work.

Per an in-person discussion with cristopher.stauffer and acm, we'll likely want to have the following Evergreen tasks execute as part of a task group with max_hosts=1:

  • compile: runs SCons with the core, tools, dbtest, integration_tests, dist, dist-debugsymbols, distsrc-${ext|tgz}, and ${msi_target|} targets
  • compile_unittests: runs SCons with the unittests target
  • unittests: runs the C++ unit tests via resmoke.py (depends on compile_unittests)
  • compile_benchmarks: runs SCons wtih the benchmarks target
  • benchmarks_orphaned: runs the C++ microbenchmarks via resmoke.py (depends on compile_benchmarks)
  • compile_all: runs SCons with the all target

Note: There were some ideas to also have compile_dist and compile_dagger tasks, but adding those should be done as follow-up work either on this ticket or another one.



 Comments   
Comment by Githook User [ 27/Apr/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}

Message: SERVER-33963 Skip unittests if compile_unittests fails
Branch: master
https://github.com/mongodb/mongo/commit/873b3feaae29803d17baf3ba07652c38e36543ac

Comment by Githook User [ 27/Apr/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}

Message: SERVER-33963 Add display_tasks to Enterprise Windows 2008R2 (inMemory)
Branch: master
https://github.com/mongodb/mongo/commit/580d395d40e935adb8453ba7420cdc28e6f601a1

Comment by Githook User [ 27/Apr/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}

Message: SERVER-33963 Add the task_group compile_all_run_unittests_TG to Enterprise Windows 2008R2 (inMemory)
Branch: master
https://github.com/mongodb/mongo/commit/69ba26c48be6a1ae334e6221765941a413e37bf5

Comment by Githook User [ 26/Apr/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'username': 'hptabster', 'name': 'Jonathan Abrahams'}

Message: SERVER-33963 Add task_groups & display_tasks for compile, compile_all, compile_unittests & unittests
Branch: master
https://github.com/mongodb/mongo/commit/a1601dd85764e1aac6128057b5681107931c64b8

Comment by Jonathan Abrahams [ 04/Apr/18 ]

Some limitations have been discovered from the original design. If there are tasks in a task_group with dependencies, i.e., depends_on, then those tasks will not be scheduled with the other tasks in that group. The workaround is as follows:

  • Create a task_group with these tasks, which have no dependencies:

    - name: compile_all_run_unittests_TG
      max_hosts: 1
      tasks:
      - compile
      - compile_unittests
      - unittests
      - compile_all
    

  • Create a display_tasks group:

    - name: unittests!
      execution_tasks:
      - compile_unittests
      - unittests
    

Comment by Githook User [ 03/Apr/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'name': 'Jonathan Abrahams', 'username': 'hptabster'}

Message: SERVER-33963 Refactor functions in etc/evergreen.yml
Branch: master
https://github.com/mongodb/mongo/commit/d1b3c0a0a4dd31e1d1b37ed21cf00a92e59f276e

Comment by Jonathan Abrahams [ 03/Apr/18 ]

Adding first part of work, refactoring evergreen.yml

Comment by Max Hirschhorn [ 03/Apr/18 ]

Per comment from Robert Guo:
Max Hirschhorn Can we not have benchmarks_orphaned run in the task group? I don't think there's anything special about this suite vs other benchmark suites.

robert.guo, jonathan.abrahams, the idea behind having benchmarks_orphaned being run as part of the same task group is that we wouldn't need to upload a tarball containing the C++ benchmark executables to S3 as part of the compile_benchmarks task only to download them in subsequent ones. Until we'd be able to run compile_benchmarks on the centos6-perf / CBI machines, I don't think it would be meaningful to run the benchmarks* tasks in the same task group.

Comment by Jonathan Abrahams [ 03/Apr/18 ]

Per comment from robert.guo:
max.hirschhorn Can we not have benchmarks_orphaned run in the task group? I don't think there's anything special about this suite vs other benchmark suites.

Generated at Thu Feb 08 04:35:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.