[SERVER-50376] Ninja next does not see compiler changes Created: 18/Aug/20  Updated: 29/Oct/23  Resolved: 22/Sep/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.8.0, 4.4.2

Type: Bug Priority: Major - P3
Reporter: Daniel Moody Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File build.ninja-with-windows-build-broken.bz2    
Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Steps To Reproduce:

reproduced on master commit: 

2506e90e792f9dce299028d532fbabb48a44b556

 

Reproduces on Ubuntu 18.04 virtual workstation

 

Steps to reproduce:

  1. make a symlink to fakeout the compiler:
    sudo ln -s /usr/bin/gcc-8 /usr/bin/fakegcc
    sudo ln -s /usr/bin/g++-8 /usr/bin/fakeg++
  2. build with ninja:
    /usr/bin/python3.8 "buildscripts/scons.py" CC=/usr/bin/fakegcc CXX=/usr/bin/fakeg++ --build-tools=next --link-model=dynamic ICECC=icecc CCACHE=ccache --ninja generate-ninja
    ninja -j400 install-mongod
  3. now switch the compiler
    sudo rm /usr/bin/fakegcc
    sudo rm /usr/bin/fakeg++
    sudo ln -s /usr/bin/gcc-9 /usr/bin/fakegcc
    sudo ln -s /usr/bin/g++-9 /usr/bin/fakeg++
  4. rebuild with just ninja:
    ninja -j400 install-mongod
  5. ninja thinks there is nothing to do
Sprint: Dev Platform 2020-09-07, Dev Platform 2020-09-21, Dev Platform 2020-10-05
Participants:

 Description   

Doing a ninja build, and changing the under lying compiler does not cause a rebuild when just reinvoking with ninja.



 Comments   
Comment by Githook User [ 13/Oct/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-50363 Merge --build-tools=next into stable

Merging the following fixes into the stable version of the build tools
and migrate fully from --ninja=next to the --build-tools=next:

(cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0)
Branch: v4.4
https://github.com/mongodb/mongo/commit/e7f6696a96db5438a269c5e67a06ff9e05cb897a

Comment by Githook User [ 09/Oct/20 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-50376 Ninja should rebuild on compiler and tool changes

(cherry picked from commit 9a2d87bdd45c5a89af147308e03121b0e4ef9a83)
Branch: v4.4
https://github.com/mongodb/mongo/commit/393eef534e7f8ba9f880f5dd46196ce2e6dec23f

Comment by Githook User [ 07/Oct/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-50363 Merge --build-tools=next into stable

Merging the following fixes into the stable version of the build tools
and migrate fully from --ninja=next to the --build-tools=next:

(cherry picked from commit 78bb3f3c8a658a5a9fec8d55864e426382f68bd0)
Branch: v4.4
https://github.com/mongodb/mongo/commit/7b93770e0835087c023ae3e3f36eeb46d204767f

Comment by Githook User [ 06/Oct/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-50363 Merge --build-tools=next into stable

Merging the following fixes into the stable version of the build tools
and migrate fully from --ninja=next to the --build-tools=next:

Comment by Githook User [ 21/Sep/20 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-50376 Ninja should rebuild on compiler and tool changes

(cherry picked from commit 6cddc81cc79964a10161b95b5c9702cda1afa2c1)
Branch: v4.4
https://github.com/mongodb/mongo/commit/de5c2bf63a4ae08dff53c6b310f65e18adc9f8cb

Comment by Githook User [ 21/Sep/20 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-50376 Ninja should rebuild on compiler and tool changes
Branch: master
https://github.com/mongodb/mongo/commit/9a2d87bdd45c5a89af147308e03121b0e4ef9a83

Comment by Daniel Moody [ 16/Sep/20 ]

Code Review: https://mongodbcr.appspot.com/674830001/

Comment by Kaloian Manassiev [ 09/Sep/20 ]

ryan.egesdahl, I have attached one such build.ninja, which reproduces this problem.

Comment by Githook User [ 09/Sep/20 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: Revert "SERVER-50376 Ninja should rebuild on compiler and tool changes"

This reverts commit 58fd67fc2232a4ca591ff66443fd22213d4b5cac.
Branch: master
https://github.com/mongodb/mongo/commit/b397d8da37c2b7d54aed313d036d68c9ac992de8

Comment by Ryan Egesdahl (Inactive) [ 09/Sep/20 ]

kaloian.manassiev How strange. Do you still have the build.ninja from your error? If so, please post it so I can examine it. I get the feeling there is a complexity here that is specific to Windows.

Comment by Kaloian Manassiev [ 09/Sep/20 ]

Reverting the commit from the previous comment and reopening this ticket, because after it, ninja builds no longer work on Windows (MSVC):

// On a clean repo:
E:\workspace\mongo>buildscripts\scons.py --jlink=4 --ssl --win-version-min=win10 --detect-odr-violations --install-mode=hygienic --dbg=on --opt=off CPPPATH="D:/workspace/libs/snmp/include D:/workspace/libs/sasl/include" LIBPATH="D:/workspace/libs/snmp/lib D:/workspace/libs/sasl/lib" MONGO_GIT_HASH="unknown" MONGO_VERSION="0.0.0" VARIANT_DIR=ninja --ninja=next generate-ninja
scons: Reading SConscript files ...
scons: running with args C:\Python37\python.exe E:\workspace\mongo\buildscripts\scons.py --jlink=4 --ssl --win-version-min=win10 --detect-odr-violations --install-mode=hygienic --dbg=on --opt=off CPPPATH=D:/workspace/libs/snmp/include D:/workspace/libs/sasl/include LIBPATH=D:/workspace/libs/snmp/lib D:/workspace/libs/sasl/lib MONGO_GIT_HASH=unknown MONGO_VERSION=0.0.0 VARIANT_DIR=ninja --ninja=next generate-ninja
Mkdir("build\scons")
scons version: 3.1.2
python version: 3 7 7 'final' 0
CC is cl
cl found in $PATH at C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.EXE
...
scons: Building targets ...
Generating: build.ninja
scons: done building targets.
 
E:\workspace\mongo>ninja install-core
ninja: error: 'cl', needed by 'build/ninja/mongo/shell/mongo.obj', missing and no known rule to make it

Comment by Githook User [ 09/Sep/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-50376 Ninja should rebuild on compiler and tool changes

Allow Ninja to rebuild build.ninja any time a SCons tool or the compiler
is changed between iterative rebuilds. This change allows us to ensure
that we don't have any stale object files lying around that may have
been produced by an incompatible toolchain.
Branch: master
https://github.com/mongodb/mongo/commit/58fd67fc2232a4ca591ff66443fd22213d4b5cac

Comment by Githook User [ 04/Sep/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: Revert "SERVER-50376 Ninja should rebuild on compiler and tool changes"

This reverts commit 4d43e298fe956c57b3cb0310bcd0785c19f75ef7.
Branch: master
https://github.com/mongodb/mongo/commit/6ac9d725e9ef5e017f46a6d65c836286ca8f88d6

Comment by Githook User [ 03/Sep/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-50376 Ninja should rebuild on compiler and tool changes

Allow Ninja to rebuild build.ninja any time a SCons tool or the compiler
is changed between iterative rebuilds. This change allows us to ensure
that we don't have any stale object files lying around that may have
been produced by an incompatible toolchain.
Branch: master
https://github.com/mongodb/mongo/commit/4d43e298fe956c57b3cb0310bcd0785c19f75ef7

Comment by April Schoffer [ 03/Sep/20 ]

ryan.egesdahl to grab this and merge for daniel given it's approved.

Comment by Daniel Moody [ 18/Aug/20 ]

possibly related to 
SERVER-50010
 

Generated at Thu Feb 08 05:22:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.