[SERVER-56623] Guard against large compiles without icecream enabled in the invocation Created: 05/Mar/21 Updated: 29/Oct/23 Resolved: 04/May/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.9.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Maria van Keulen | Assignee: | Daniel Moody |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | workstations | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Sprint: | Dev Platform 2021-05-17 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
Proposed Solution Original Request It would be great if the workstations could protect against such compiles. This comment describes a potential approach to address forgetting icecream from the invocation. |
| Comments |
| Comment by Githook User [ 04/May/21 ] |
|
Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}Message: |
| Comment by Daniel Moody [ 26/Apr/21 ] |
|
Code Review: https://mongodbcr.appspot.com/766980004/ |
| Comment by Maria van Keulen [ 07/Apr/21 ] |
|
Gotcha, thanks for the additional context. Splitting the icecream-side work into another ticket sounds good to me. |
| Comment by Daniel Moody [ 06/Apr/21 ] |
|
maria.vankeulen That would be bit more tricky, because the ninja file after generation is static and is generated from a system state of some previous time, so icecream daemon state is subject to change. I think that would more appropriately be worked into the icecream interface instead. So currently there are icerun and icecc, but instead we could have our own internal icecream script which can test icecream functioning and forward the commands to the real icecream scripts and bins, otherwise hard fail. Actually icecream itself should be doing something like this, it's just not graceful in its own failures. I think something like that could spin off to another ticket, and this is scoped just to prevent large jobs from hanging the system when icecream was not selected. |
| Comment by Maria van Keulen [ 06/Apr/21 ] |
|
Sounds great, thank you daniel.moody! Is it feasible to also guard against the case of running a large build with the icecream daemon turned off? |
| Comment by Daniel Moody [ 06/Apr/21 ] |
|
Planned approach will be for pure scons build, if -j is not set, scons will check for icecream being set, and if its not set limit the jobs to number of cpus. An additional optional --force-jobs will be used to override this behavior. A message will always be printed saying scons is limiting the number of jobs if it takes such action. For the ninja side, If no icecream is in use, scons will generate all pools with a max depth of the max cpus, and it will generate an extra build edge which will be built first and always rebuilt. This can be accomplished with a build edge with no dependencies, a single input which is the ninja file itself, and does not produce its output, and is listed as the first build edge in the ninja file. This build edge will echo that the number of jobs is being limited due to no icecream and to regenerate if not desired. EDIT:
|
| Comment by Brooke Miller [ 19/Mar/21 ] |
|
robert.guo mentioned it'd probably be better to do this in the build system, so we've reassigned this to SDP. FYI april.schoffer |