[SERVER-37428] Sys-perf: linux builds using enterprise bits Created: 02/Oct/18 Updated: 29/Oct/23 Resolved: 26/Nov/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Performance |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.6, 4.1.6 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | David Daly | Assignee: | Max Hirschhorn |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | tig-evgconfig | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Requested: |
v4.0, v3.6, v3.4
|
||||||||||||||||
| Sprint: | STM 2018-12-03 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Story Points: | 2 | ||||||||||||||||
| Description |
|
Note: this only affects internal testing systems. The system_perf.yml file support for enterprise modules is doing the wrong thing for non-enterprise builds. There are three compile tasks in one variant, but they all get the enterprise module. The fix seems to be making the enterprise build more like the wtdevelop build, by checking out the enterprise module to a directory outside the mongo source directory, and actively copying it in for the enterprise build as is done in "use wiredtiger develop" function. |
| Comments |
| Comment by Githook User [ 21/Dec/18 ] |
|
Author: {'username': 'visemet', 'email': 'max.hirschhorn@mongodb.com', 'name': 'Max Hirschhorn'}Message: Build variants are able to define separate lists of modules whereas (cherry picked from commit d11e4a25014c2d6a4939481dbefa52fd11d8f286) |
| Comment by Githook User [ 26/Nov/18 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: Build variants are able to define separate lists of modules whereas |
| Comment by Henrik Ingo (Inactive) [ 19/Nov/18 ] |
|
Note that after EVG-727 / EVG-5379 this ticket changes in nature: We could now have a single compile task and use the enterprise version for everything. (Will still need a separate wtdevelop compile, of course.) |
| Comment by Henrik Ingo (Inactive) [ 05/Oct/18 ] |
|
Already commented on CR, only now seeing this thread...
This is a good point and IMO your proposal is also an acceptable solution to the same proposal. The reason the compile variant is the same is that we want all binaries to be compiled on the same linux and same toolchain to maintain ability to compare them. (More generally, until now we never targeted different OSes, compilers or libraries in sys-perf, focus was solely on MongoDB's own performance in an otherwise static environment.) So it was just natural to have a single variant for that. I tried to think about why I instinctively prefer the wtdevelop approach to the one seen in enterprise module, and came to this conclusion: In the wtdevelop compile, it's the same block of code that takes one parameter and:
For enterprise, it is now a completely separate input parameter/mechanism that clones enterprise source code into src/ and then a separate block of code to add -enterprise to the binary or not. When I look at the code, it seems just a matter of time before someone will again accidentally include enterprise module into the community compile, and there's nothing in the subsequent code that will fail hard or alert the user about this. If we wanted to go with your proposal, then we IMO need to change this code so that it doesn't take a parameter rather correctly determines whether the compile is enterprise or not. What should happen in the accidental use case is that the compile task uploads ...-enterprise.tgz and the task will try to download ...tgz and fail when it isn't there. (And also mongo --version should print the truth.) |
| Comment by Max Hirschhorn [ 05/Oct/18 ] |
|
There's another option of leaving the git.get_project command and build variants as-is and instead using the --modules command line option to SCons to get it to ignore the src/mongo/db/modules/enterprise/ directory in the compile and compile_wtdevelop tasks. |
| Comment by Max Hirschhorn [ 02/Oct/18 ] |
I don't agree with this proposal. The WiredTiger develop does what it does because there's an existing copy of the wiredtiger/wiredtiger repository vendored in the mongodb/mongo source tree - that isn't the case for the enterprise module. We shouldn't be specifying enterprise in the modules list for the build variant when we specifically are trying not to build with the enterprise module enabled. Is there a reason (e.g. how task dependencies work in Evergreen) that we chose to have 1 build variant with 3 compile* tasks rather than 3 build variants with 1 compile task each? |