[SERVER-46129] Ninja generation is slower when using hygienic builds Created: 13/Feb/20  Updated: 27/Oct/23  Resolved: 22/Apr/20

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: 4.4.0-rc3

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Andrew Morrow (Inactive)
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-41674 Turn hygienic builds on by default Closed
Backport Requested:
v4.4
Sprint: Dev Platform 2020-03-09, Dev Platform 2020-04-20, Dev Platform 2020-05-04
Participants:

 Description   

Enabling hygienic builds currently slows down Ninja generation. We should do some work to improve performance so the impact is lessened or eliminated.



 Comments   
Comment by Githook User [ 26/Apr/20 ]

Author:

{'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io', 'username': 'chasinglogic'}

Message: SERVER-46129 Fix various DAG inconsistencies

Some dependencies were getting wired when they should not or were
false dependencies.

(cherry picked from commit 727fe1181cc8ff8d3c6282662229d28651a927d9)
Branch: v4.4
https://github.com/mongodb/mongo/commit/76e7de4df5b74f1a1b9bf84b1d0384ae360dad32

Comment by Andrew Morrow (Inactive) [ 24/Apr/20 ]

The last commit of this ticket didn't make it in time for the v4.4 cut, but if we intend to make ninja_next consistent across both branches, we probably want the fixes there too.

Comment by Andrew Morrow (Inactive) [ 22/Apr/20 ]

I can't reproduce the claim of this ticket against the current state of the ninja generator on master. I tried both on my macOS machine and my (very old, very slow) linux machine, and I find no material difference when switching on -install-mode=hygienic, for either ninja or -ninja=next.

mac os, ninja, non-hygienic, static  : 26.95 real        23.14 user         2.99 sys
mac os, ninja, non-hygienic, dynamic : 33.53 real        29.82 user         3.26 sys
 
mac os, ninja, hygienic, static      : 27.21 real        23.88 user         2.89 sys
mac os, ninja, hygienic, dynamic     : 36.71 real        33.20 user         3.06 sys
 
 
mac os, ninja-next, non-hygienic, static  : 26.24 real        22.95 user         2.85 sys
mac os, ninja-next, non-hygienic, dynamic : 33.71 real        30.34 user         2.94 sys
 
mac os, ninja-next, hygienic, static      : 26.65 real        23.34 user         2.86 sys
mac os, ninja-next, hygienic, dynamic     : 37.45 real        34.17 user         2.88 sys
 
 
 
 
linux, ninja, non-hygienic, static  : 35.29user 2.93system 0:38.18elapsed 100%CPU (0avgtext+0avgdata 206768maxresident)k
linux, ninja, non-hygienic, dynamic : 45.09user 3.08system 0:48.13elapsed 100%CPU (0avgtext+0avgdata 247832maxresident)k
 
linux, ninja, hygienic, static      : 37.20user 2.76system 0:39.95elapsed 100%CPU (0avgtext+0avgdata 210908maxresident)k
linux, ninja, hygienic, dynamic     : 50.13user 3.09system 0:53.20elapsed 100%CPU (0avgtext+0avgdata 290632maxresident)k
 
 
linux, ninja-next, non-hygienic, static  : 34.35user 2.86system 0:37.18elapsed 100%CPU (0avgtext+0avgdata 209288maxresident)k
linux, ninja-next, non-hygienic, dynamic : 47.66user 3.02system 0:50.64elapsed 100%CPU (0avgtext+0avgdata 286364maxresident)k
 
linux, ninja-next, hygienic, static      : 35.35user 2.80system 0:38.10elapsed 100%CPU (0avgtext+0avgdata 215188maxresident)k
linux, ninja-next, hygienic, dynamic     : 53.07user 3.14system 0:56.18elapsed 100%CPU (0avgtext+0avgdata 330540maxresident)k

Comment by Githook User [ 06/Mar/20 ]

Author:

{'name': 'Mathew Robinson', 'username': 'chasinglogic', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Fix various DAG inconsistencies

Some dependencies were getting wired when they should not or were
false dependencies.
Branch: master
https://github.com/mongodb/mongo/commit/727fe1181cc8ff8d3c6282662229d28651a927d9

Comment by Githook User [ 28/Feb/20 ]

Author:

{'username': 'chasinglogic', 'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Correct function variable naming
Branch: master
https://github.com/mongodb/mongo/commit/15f7654d798f71f9ce58b322a48da66d508fbbcf

Comment by Githook User [ 27/Feb/20 ]

Author:

{'username': 'chasinglogic', 'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Ignore things that aren't builders when hooking into emitters for the ninja file
Branch: master
https://github.com/mongodb/mongo/commit/abcbc1ea9de96ac463720445c1e42080b417a8eb

Comment by Githook User [ 27/Feb/20 ]

Author:

{'username': 'chasinglogic', 'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Escape scons variable correctly in subst string
Branch: master
https://github.com/mongodb/mongo/commit/9f5493388564fb69d4098b0f6ff84610d412f0d5

Comment by Githook User [ 27/Feb/20 ]

Author:

{'username': 'chasinglogic', 'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Fix broken NINJA_*IX vars for ninja stable
Branch: master
https://github.com/mongodb/mongo/commit/1ba6e4193cfc27277e4ed65b033ee03a81b0c3e1

Comment by Githook User [ 22/Feb/20 ]

Author:

{'username': 'chasinglogic', 'name': 'Mathew Robinson', 'email': 'mathew@chasinglogic.io'}

Message: SERVER-46129 Make ninja faster and contain all targets all the time
Branch: master
https://github.com/mongodb/mongo/commit/60aed37d906100af704bcb5c4cb8f6080e1540fe

Generated at Thu Feb 08 05:10:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.