[SERVER-40032] Move post-scons compile tasks that do not block tests into a separate phase Created: 07/Mar/19  Updated: 06/Dec/22  Resolved: 28/Jan/21

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: [DO NOT ASSIGN] Backlog - Decision Automation Group (DAG) (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-48893 Use dynamic linking for the commit qu... Closed
Gantt End to End
has to be finished together with SERVER-40872 Spawning host with data from failed t... Closed
Problem/Incident
causes SERVER-40872 Spawning host with data from failed t... Closed
Related
related to SERVER-40929 upload_debug_symbols should be requir... Closed
Assigned Teams:
Decision Automation Group
Sprint: DAG 2019-05-06
Participants:
Story Points: 3

 Description   

After the SCons phase of the compile task completes, we spend something like 6 minutes doing post-build packaging and uploading tasks. Many of these tasks are not actual prerequisites for starting to run off-box tests. Moving them to a post-compile phase would allow those tasks to run concurrently with initiating off-box testing.

Not all parts can be elided: we still need at least the "binaries" and "archive.tgz" artifacts to get created.

However, it seems likely that the following long running tasks could be parallelized:

  • Generating rpm and deb packages
  • Uploading the debug symbols archive to s3
  • Creating other tarfiles like the shell archive.

Making these changes would both allow off-box testing to start a few minutes faster, as well as make the timing of the 'compile' task more reflective of the actual time spent compiling.



 Comments   
Comment by David Bradford (Inactive) [ 28/Jan/21 ]

It looks like this work was already done.

Comment by Andrew Morrow (Inactive) [ 28/Jan/21 ]

david.bradford - I think this work is actually already done, from when we split things into "compile_no_archive".

Comment by David Bradford (Inactive) [ 28/Jan/21 ]

acm I know SDP has been making a lot of improvements to the compile task recently. Is this ticket still something we would want to do? If so, do you think SDP be in a better position to implement it than DAG?

Comment by David Bradford (Inactive) [ 28/Oct/19 ]

I'm marking this as Backlog while we wait for PM-601 to be completed.

Comment by Andrew Morrow (Inactive) [ 19/Jul/19 ]

david.bradford - Thanks for adding the PM link.

Comment by David Bradford (Inactive) [ 19/Jul/19 ]

The evergreen work has not landed yet. This is dependent on PM-601. I've added that dependency in this ticket to better track it.

Comment by Andrew Morrow (Inactive) [ 19/Jul/19 ]

david.bradford - Has the better dependencies in evg work landed? Can we move this forward again?

Comment by David Bradford (Inactive) [ 02/May/19 ]

We need better support for dependencies in Evergreen before we can do this work.

Comment by Githook User [ 02/May/19 ]

Author:

{'email': 'david.bradford@mongodb.com', 'name': 'David Bradford', 'username': 'dbradf'}

Message: Revert "SERVER-40032: Move upload_debug_symbols out of compile task"

This reverts commit 833dc84bc7f784e70e8a10f21c432745fbc24040.
Branch: master
https://github.com/mongodb/mongo/commit/8721322fedc70435bb1e6c2fc7174ce021433e6f

Comment by Githook User [ 24/Apr/19 ]

Author:

{'email': 'david.bradford@mongodb.com', 'name': 'David Bradford', 'username': 'dbradf'}

Message: SERVER-40032: Move upload_debug_symbols out of compile task
Branch: master
https://github.com/mongodb/mongo/commit/833dc84bc7f784e70e8a10f21c432745fbc24040

Generated at Thu Feb 08 04:53:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.