[SERVER-41129] Small changes on windows causes large recompile Created: 13/May/19  Updated: 10/Jun/19  Resolved: 21/May/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.1.12

Type: Bug Priority: Major - P3
Reporter: Jonathan Reams Assignee: bdbaddog#1
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:
  • Launch spawn host with source artifacts from a patch build.
  • Change some cpp files
  • Compile
  • Change same cpp files
  • Compile again - it will recompile way more than just the cpp files you changed
Sprint: Dev Tools 2019-05-20, Dev Tools 2019-06-03
Participants:

 Description   

When working on a spawn host, I've noticed that small changes in cpp files cause large re-compiles in windows.



 Comments   
Comment by Githook User [ 21/May/19 ]

Author:

{'email': 'bill@baddogconsulting.com', 'name': 'William Deegan', 'username': 'bdbaddog'}

Message: SERVER-41129 Bring enhanced --debug=explain logic over from SCons

Signed-off-by: Andrew Morrow <acm@mongodb.com>

Closes #1309
Branch: master
https://github.com/mongodb/mongo/commit/db55694f289d99db4f808a44643130818f3048a4

Comment by Githook User [ 21/May/19 ]

Author:

{'email': 'bill@baddogconsulting.com', 'name': 'William Deegan', 'username': 'bdbaddog'}

Message: SERVER-41129 - Needed to first check native windows path against dependency map, and then check normalized path. (We weren't checked win32 native path

Signed-off-by: Andrew Morrow <acm@mongodb.com>

Closes #1309
Branch: master
https://github.com/mongodb/mongo/commit/34cc3d7d40b5765c300dee97de85bd90062dfb37

Comment by Andrew Morrow (Inactive) [ 16/May/19 ]

bdbaddog - Here is a reproducing example for me on Windows against latest master:

python buildscripts/scons.py --ssl MONGO_DISTMOD=windows-64 CPPPATH="c:/sasl/include c:/snmp/include" LIBPATH="c:/sasl/lib c:/snmp/lib" -j17 --dynamic-windows build/opt/mongo/bson/mutable  --debug=explain

This will produce spurious rebuilds on the second build. Often, the initial build will also fail where a file that should be getting generated is not generated in time.

If I switch to using scons 3.0.5 installed via pip, the rebuilds go away, and I never observe a failure to generate the required file:

scons.bat --ssl MONGO_DISTMOD=windows-64 CPPPATH="c:/sasl/include c:/snmp/include" LIBPATH="c:/sasl/lib c:/snmp/lib" -j17 --dynamic-windows build/opt/mongo/bson/mutable  --debug=explain

Comment by Andrew Morrow (Inactive) [ 15/May/19 ]

Another workaround appears to be to continue to use the vendored SCons, but add --build-fast-and-loose=off on the command line.

Comment by Andrew Morrow (Inactive) [ 15/May/19 ]

I was able to reproduce this on my own spawn host. It appears that there is a windows specific defect in our vendored SCons 3.0.4 (or in one of the upstream fixes we have cherry-picked back to it). I've also verified that using SCons 3.0.5 (installed into the virtualenv with pip) doesn't reproduce the spurious rebuilds. I'll check in with bdbaddog to see if there is a known upstream fix in 3.0.5 that we can cherry-pick back.

Generated at Thu Feb 08 04:56:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.