[SERVER-58904] Escaped or single-quoted DESTDIR causes malformed .ninja and resmoke.ini Created: 28/Jul/21  Updated: 29/Oct/23  Resolved: 05/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.3, 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Bernard Gorman Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: Dev Platform 2021-08-09
Participants:

 Description   

Generate a build.ninja file which explicitly sets DESTDIR to $BUILD_ROOT/install using either escaping (i.e. DESTDIR=\$BUILD_ROOT/install) or single-quoting (i.e. DESTDIR='$BUILD_ROOT/install'). For instance:

python buildscripts/scons.py CCFLAGS='-gsplit-dwarf' --variables-files=etc/scons/developer_versions.vars --disable-warnings-as-errors --link-model=dynamic --opt=off VARIANT_DIR=ninja DESTDIR=\$BUILD_ROOT/install --variables-files=etc/scons/mongodbtoolchain_stable_clang.vars --ninja --modules= --install-mode=hygienic ICECC=icecc CCACHE=ccache NINJA_PREFIX=build-mongo NINJA_SUFFIX=ninja generate-ninja

After the file is generated, the SCONS invocation recorded in build.ninja and the contents of resmoke.ini both appear correct:

SCONS_INVOCATION = /home/ubuntu/python-env/mongo-python3/bin/python "buildscripts/scons.py" $
    "CCFLAGS=-gsplit-dwarf" "--variables-files=etc/scons/developer_versions.vars" $
    "--disable-warnings-as-errors" "--link-model=dynamic" "--opt=off" "VARIANT_DIR=ninja" $
    "DESTDIR=\$BUILD_ROOT/install" $
    "--variables-files=etc/scons/mongodbtoolchain_stable_clang.vars" "--ninja" "--modules=" $
    "--install-mode=hygienic" "ICECC=icecc" "CCACHE=ccache" "NINJA_PREFIX=build-mongo" $
    "NINJA_SUFFIX=ninja" __NINJA_NO=1 $out

[resmoke]
install_dir = build/install/bin

However, if you now build the server using ninja, it will cause the resmoke.ini file to become corrupted:

[resmoke]
install_dir = \\/install/bin

This will prevent any resmoke tests from being run:

buildscripts/resmoke.py run jstests/noPassthrough/change_stream_sharded_startafter_invalidate.js
...
[j0] Starting mongod on port 20000...
'PATH=/home/ubuntu/mongo-dev/mongo:/data/multiversion:/home/ubuntu/mongo-dev/mongo/\\/install/bin:/home/ubuntu/python-env/mongo-python3/bin:/home/ubuntu/.nvm/versions/node/v16.4.0/bin:/opt/go/bin:/home/ubuntu/mongo/build/install/bin
...
[j0] Failed to start mongod on port 20000: [Errno 2] No such file or directory: '/home/ubuntu/mongo-dev/mongo/\\\\/install/bin/mongod': '/home/ubuntu/mongo-dev/mongo/\\\\/install/bin/mongod'

If build.ninja is subsequently auto-regenerated due to any changes in SConstruct or Sconscript files, the build.ninja file will itself become similarly corrupted:

SCONS_INVOCATION = /home/ubuntu/python-env/mongo-python3/bin/python "buildscripts/scons.py" $
    "CCFLAGS=-gsplit-dwarf" "--variables-files=etc/scons/developer_versions.vars" $
    "--disable-warnings-as-errors" "--link-model=dynamic" "--opt=off" "VARIANT_DIR=ninja" $
    "DESTDIR=\\/install" "--variables-files=etc/scons/mongodbtoolchain_stable_clang.vars" $
    "--ninja" "--modules=" "--install-mode=hygienic" "ICECC=icecc" "CCACHE=ccache" $
    "NINJA_PREFIX=build-mongo" "NINJA_SUFFIX=ninja" __NINJA_NO=1 $out



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Daniel Moody [ 05/Aug/21 ]

upstream: https://github.com/SCons/scons/pull/3994

Comment by Githook User [ 04/Aug/21 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-58904 update ninja tool to support $ in scons command line args.

(cherry picked from commit dee0f733cdb3cab7714326574efc8a1ae4ec750f)
Branch: v5.0
https://github.com/mongodb/mongo/commit/2609bd519b21d96c23ef0489d835e7e743f59170

Comment by Githook User [ 04/Aug/21 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-58904 update ninja tool to support $ in scons command line args.
Branch: master
https://github.com/mongodb/mongo/commit/dee0f733cdb3cab7714326574efc8a1ae4ec750f

Generated at Thu Feb 08 05:45:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.