[SERVER-49036] the compile_commands.json produced by ninja is incomplete. Created: 23/Jun/20 Updated: 29/Oct/23 Resolved: 07/Jul/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.1, 4.7.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Gabriel Russell (Inactive) | Assignee: | Daniel Moody |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Operating System: | ALL | ||||
| Backport Requested: |
v4.4
|
||||
| Sprint: | Dev Platform 2020-07-13 | ||||
| Participants: | |||||
| Description |
|
The compile_commands.json produced by scons is complete. For example, this is not enough information to compile this target:
|
| Comments |
| Comment by Githook User [ 13/Oct/20 ] |
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: Merging the following fixes into the stable version of the build tools
(cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0) |
| Comment by Githook User [ 07/Oct/20 ] |
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: Merging the following fixes into the stable version of the build tools
(cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0) |
| Comment by Githook User [ 06/Oct/20 ] |
|
Author: {'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}Message: Merging the following fixes into the stable version of the build tools |
| Comment by Githook User [ 06/Aug/20 ] |
|
Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}Message: (cherry picked from commit 5fe923a0aa312044062df044eb4eaa47951f70ec) |
| Comment by Andrew Morrow (Inactive) [ 02/Jul/20 ] |
|
daniel.moody - We will want a backport to v4.4 for this. |
| Comment by Githook User [ 02/Jul/20 ] |
|
Author: {'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}Message: |
| Comment by Daniel Moody [ 29/Jun/20 ] |
|
Alternatively, if we like the compilation database generated by SCons better, should we reconsider having Ninja generate at all, and have the Ninja `compiledb` simply re-invoke SCons?
I am curious how well that would work. Assuming we force compilation db to the last build step, all nodes could be built by ninja, and scons would finish up with a compilation db. I would wonder if they would generate compilation databases that we would consider equivalent? I would assume it would be slower, but not terribly slower, and probably not be reinvoked for iterative rebuilds. If the investigation pans out I think that would be the best path forward. |
| Comment by Daniel Moody [ 29/Jun/20 ] |
|
What released versions of Ninja contain that flag?
Looking at the tags in this commit: https://github.com/ninja-build/ninja/commit/26341ca0d652b6e50aa57fa176bf610480be5edd
Should we instead be telling people to use keeprsp as suggested in the Ninja PR?
The downside to that is that it will keep all response files hanging around. Its not terrible, but there are other bugs I have seen in older ninja (like dependency bugs), so I would prefer to force new ninjas if possible. If we roll it all up with python, then we can just add it to mongo requirements.txt for the build. |
| Comment by Daniel Moody [ 29/Jun/20 ] |
|
As of right now, we don't require use of any particular version of Ninja.
One tangental note, I would recommend making the build use ninja installed with pip. That makes it dependent on python which the build system is using, and you can use python module system to find the binary and the ninja_syntax.py file reliably. Then you don't need to worry about what version the system package manager is installing or other cross platform issues. |
| Comment by Andrew Morrow (Inactive) [ 29/Jun/20 ] |
|
What released versions of Ninja contain that flag? As of right now, we don't require use of any particular version of Ninja. Should we instead be telling people to use keeprsp as suggested in the Ninja PR? Alternatively, if we like the compilation database generated by SCons better, should we reconsider having Ninja generate at all, and have the Ninja `compiledb` simply re-invoke SCons? Our SCons level compilation_database actually works a little bit better in some ways: it has a dependency on the generated sources so ensures their existence. |
| Comment by Daniel Moody [ 29/Jun/20 ] |
|
ninja supports response file expansion with the -x option, although its not in their documentation directly: https://github.com/ninja-build/ninja/pull/1223
Also adding the -x option will still be an incomplete compilation database, because the substfiles and textfiles that the build uses can only be done in scons, and scons must be reinvoked to generate those.
One option would be to modifiy these scons only targets and write external scripts which ninja can use to generate them. This would allow the compdb that ninja spits out to represent the full build. (i.e. there would be no build using the TEMPLATE rule in the generated ninja file) |