[SERVER-54262] Cannot use --build-tools=next with scons 4.1 Created: 03/Feb/21  Updated: 03/Mar/21  Resolved: 01/Mar/21

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Ryan Egesdahl (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Sprint: Dev Platform 2021-02-22, Dev Platform 2021-03-08
Participants:

 Description   

I wanted to do an experiment with SCons 4, and discovered that it doesn't work with --build-tools=next on master.

$VIRTUAL_ENV/bin/scons --variables-files=etc/scons/xcode_macosx.vars configure

Works on my macOS machine, but

$VIRTUAL_ENV/bin/scons --variables-files=etc/scons/xcode_macosx.vars configure --build-tools=next

does not. It fails during the configure checks:

scons: Reading SConscript files ...
scons: running with args /Users/acm/.virtualenvs/mongo/bin/python3 /Users/acm/.virtualenvs/mongo/bin/scons --variables-files=etc/scons/xcode_macosx.vars configure --build-tools=next
Using variable customization file /Users/acm/.scons/site_scons/mongo_custom_variables.py
Using variable customization file etc/scons/xcode_macosx.vars
Mkdir("build/scons")
scons version: 4.1.0.post1
python version: 3 9 1 'final' 0
CC is /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
CXX is /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ resolves to /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
Checking if C++ compiler "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++" is GCC... no
Checking if C++ compiler "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++" is clang... yes
Checking if C compiler "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" is clang... yes
Detected a x86_64 processor
Checking if target OS macOS is supported by the toolchain... yes
Checking if C compiler is clang 7.0 (or Apple XCode 10.2) or newer...yes
Checking if C++ compiler is clang 7.0 (or Apple XCode 10.2) or newer...yes
Checking for sufficient macOS target version minimum... yes
Checking if linker supports -fuse-ld=lld... no
Checking if linker supports -fuse-ld=gold... no
Checking whether the C compiler works... yes
Checking whether the C++ compiler works... yes
Checking that the C++ compiler can link a C++ program... no
C++ compiler /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ can't link C++ programs

Looking in config.log shows that SCons has failed to substitute something, expanding it to a python stringification rather than having called the function:

Compiling build/scons/opt/sconf_temp/conftest_737c003a897167642b70f8cdde8a0eaf_1_a86c4b69e2021206a6777b5b8a971ae1.o
scons: Configure: yes
 
scons: Configure: Checking that the C++ compiler can link a C++ program...
build/scons/opt/sconf_temp/conftest_ac30bfea77045c1c67c4f51773b51414_0.cpp <-
  |
  |#include <iostream>
  |#include <cstdlib>
  |
  |int main() {
  |    std::cout << "Hello, World" << std::endl;
  |    return EXIT_SUCCESS;
  |}
  |
Compiling build/scons/opt/sconf_temp/conftest_ac30bfea77045c1c67c4f51773b51414_0.o
Linking build/scons/opt/sconf_temp/conftest_ac30bfea77045c1c67c4f51773b51414_0_2e6d0cf7ce9340931b3db66029152374
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o "build/scons/opt/sconf_temp/conftest_ac30bfea77045c1c67c4f51773b51414_0_2e6d0cf7ce9340931b3db66029152374" -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -mmacosx-version-min=10.13 -target darwin17.0.0 -arch x86_64 -Wl,-bind_at_load -Wl,-fatal_warnings "build/scons/opt/sconf_temp/conftest_ac30bfea77045c1c67c4f51773b51414_0.o" functools.partial(<function get_syslibdeps at 0x1058eda60>, debug=None, shared=False) -lresolv'
scons: Configure: no

Note the functools.partial(<function get_syslibdeps at 0x1058eda60>, in there.



 Comments   
Comment by Ryan Egesdahl (Inactive) [ 01/Mar/21 ]

This was my PR: https://github.com/SCons/scons/pull/3889
My PR was as closed and reimplemented here: https://github.com/SCons/scons/pull/3895

Given that the new PR is exactly where I was going next, and the current codebase doesn't need any changes to work with our vendored SCons, I'm just going to drop this. When the new PR is merged, it will fix this problem without any intervention from us.

Comment by Ryan Egesdahl (Inactive) [ 17/Feb/21 ]

It looks like SCons now does an exact signature check when producing a subst for whether it's dealing with a generator or some other "user-defined" function. If your function does not have an exact signature for a generator, it ends up calling your subst with probably the wrong arguments. I can (and will) work around this, but it's probably not desirable behavior because it breaks some expectations developers might have while working in Python. It should either be checking a non-exhaustive list of arguments or allowing an override such as via a decorator to force the meaning of a function. I'll raise a bug report and PR later for to get the conversation started.

https://github.com/SCons/scons/blame/19efacf874704e95c3d1f7e65e505b90f7e2c60d/SCons/Subst.py#L423

Comment by Ryan Egesdahl (Inactive) [ 03/Feb/21 ]

It looks like SCons 4 may not be fully evaluating substs as callables anymore and is instead running something like _str_() on any second-level substs. In this particular case, it's happening in libdeps_next.py when $_SYSLIBDEPS is evaluated in $_LIBDEPS.

Generated at Thu Feb 08 05:33:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.