[SERVER-46296] Migrate non-windows ! builders to --link-model=dynamic Created: 21/Feb/20 Updated: 29/Oct/23 Resolved: 16/Jun/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Andrew Morrow (Inactive) | Assignee: | Andrew Morrow (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Dev Platform 2020-02-24, Dev Platform 2020-06-15, Dev Platform 2020-06-29 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
The dynamic link model is much faster for incremental builds. We can make all of the non-windows waterfall ! builders use --link-model=dynamic. Any pushing builders that are currently ! builders can simply be duplicated to be * builders, so they will still exist, but they won't be part of the patch workflow. Our current list of ! and * builders is as follows:
This particular mix meets several properties. When considering only the ! builders, we have:
When the * builders are included, we get even more coverage, including
There are probably a few other variations as well that aren't called out here. By switching over to dynamic, we can reduce to the following list:
The most important thing to note here is that the old ! Enterprise RHEL 6.2 has been duplicated to * Enterprise RHEL 6.2, since we can't push the artifacts from the new ! Shared Library Enterprise RHEL 6.2 builder. A careful review of this shows that we have essentially the same coverage in the ! set:
The remaining * builders cover the same cases as before. As an additional benefit, we will be able to remove the old special case dynamic builders that ran no tests, because now the primary builders in ! will be dynamic. Overall, this change should greatly reduce the time spent waiting on compile and link tasks in both the waterfall and the patch workflows. |
| Comments |
| Comment by Githook User [ 16/Jun/20 ] |
|
Author: {'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}Message: |
| Comment by Andrew Morrow (Inactive) [ 01/Apr/20 ] |
|
Kicking this a sprint. Patch testing revealed that the startup hit from moving to dynamic is too high, and we will need to consider how to address that. |
| Comment by David Bradford (Inactive) [ 31/Mar/20 ] |
|
Overall we are trying to keep the makespan of the commit queue under 15 minutes. There is some variant due to lots of factors, most notably what files are being changed, but we would like the majority of commits to be under that target. Our first step would be migrating the commit queue to use the same compile task as one of the required builders (right now it is doing its own compile that skips some packaging to improve the task runtime). And it would be great to get compile on both a windows variant and a linux variant. After that unittests would be our next target. Having unittests run on both windows and linux would be great, but even just one would be an improvement. We would also love to pulling something like jsCore. compile_all I have considered a lower priority since it seems compiling the binaries and unittests would be the most valuable. But we can certainly consider it. The runtime is going to be the biggest issue. |
| Comment by Andrew Morrow (Inactive) [ 31/Mar/20 ] |
|
Not immediately, but getting compile_all and unittests into the commit queue is one of the major drivers for this. david.bradford can provide more details on how we will achieve that step. |
| Comment by Eric Milkie [ 31/Mar/20 ] |
|
This change will also decrease commit queue task times, correct? Does the commit queue use one of the builders above? I'm really hoping we can soon add the compile_all and/or unittests (compile) task to the commit queue. |