[SERVER-54232] Use install rather than cp to copy files in buildsystem Created: 03/Feb/21  Updated: 27/Oct/23  Resolved: 27/Oct/23

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: [DO NOT ASSIGN] Backlog - Server Development Platform Team (SDP) (Inactive)
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-48203 Support --install-action for Ninja bu... Closed
Assigned Teams:
Server Development Platform
Participants:

 Description   

One of the weird things I learned while working on ninja is that install is significantly faster than cp. I don't know why this is, but it is reproducible across many systems and filesystems, etc. For example, this is running on a virtual workstation (c5.4xl ubuntu 18.04 xfs):

ubuntu@ip-10-122-10-16:~$ dd if=/dev/zero of=src bs=1M count=100; sync;                          
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.189798 s, 552 MB/s
ubuntu@ip-10-122-10-16:~$ for i in `seq 10`; do time install src dest; done; sync;
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.064s
user    0m0.000s
sys     0m0.064s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.061s
user    0m0.000s
sys     0m0.061s
 
real    0m0.060s
user    0m0.000s
sys     0m0.061s
ubuntu@ip-10-122-10-16:~$ for i in `seq 10`; do time cp src dest; done; sync;
 
real    0m0.234s
user    0m0.000s
sys     0m0.071s
 
real    0m0.226s
user    0m0.004s
sys     0m0.067s
 
real    0m0.220s
user    0m0.000s
sys     0m0.073s
 
real    0m0.211s
user    0m0.000s
sys     0m0.070s
 
real    0m0.223s
user    0m0.000s
sys     0m0.075s
 
real    0m0.335s
user    0m0.000s
sys     0m0.075s
 
real    0m0.400s
user    0m0.000s
sys     0m0.073s
 
real    0m0.400s
user    0m0.004s
sys     0m0.066s
 
real    0m0.398s
user    0m0.000s
sys     0m0.073s
 
real    0m0.399s
user    0m0.000s
sys     0m0.070s

I have tried on my laptop (WSL2 + Arch ext4 on nvme) and the results were similar. Including the weirdness of later runs of cp being slower than earlier runs. �



 Comments   
Comment by Andrew Morrow (Inactive) [ 03/Feb/21 ]

redbeard0531 - Once we have something that works at all layers with --install-action, then we can make the default better, and potentially Ninja aware, so it won't need to be opt in.

Comment by Mathias Stearn [ 03/Feb/21 ]

I'm fine considering it part of SERVER-48203 work, but I think we should just make the copy action use the install tool (or cp --remove-dest if that is preferred), since I can't imagine anyone wanting to choose the current behavior instead. At the very least, we should make the faster and more correct option the default rather than expecting people to opt-in via --install-action.

Comment by Daniel Moody [ 03/Feb/21 ]

Well the CR out for SERVER-48203 is using --remove-destination, which maybe instead should be rm; cp for compatibility?

Comment by Andrew Morrow (Inactive) [ 03/Feb/21 ]

redbeard0531 - Ah right, I remember when we investigated that. OK, well, still think SERVER-48203 is going to provide the mechanism to achieve what you want here.

Comment by Mathias Stearn [ 03/Feb/21 ]

Ninja does not remove outputs prior to running commands, otherwise that would break things like restat=1. This is also why you need to remove static libs in the AR rule, because someone decided it would be a good idea for ar to be stateful: https://github.com/mongodb/mongo/blob/26a3e7bc2a773b992c51808577ae999fc0e66d0f/site_scons/site_tools/ninja.py#L489-L498.

Comment by Andrew Morrow (Inactive) [ 03/Feb/21 ]

redbeard0531 - We are working in SERVER-48203 to project the SCons level `--install-action` flag down into the generated Ninja so that the choice of copy mechanism becomes easily configurable. We could certainly add using install as one of the selectable copy mechanisms, which are currently limited to copy, hardlink, and symlink. Note that SCons will always remove targets before writing to them (unless tagged with .Precious) which is why hardlink is safe there. If Ninja doesn't do the same, then I agree that using cp without --remove-dest is risky. Let's see how SERVER-48203 progresses.

CC daniel.moody

Comment by Mathias Stearn [ 03/Feb/21 ]

I figured out why install is faster at https://superuser.com/a/229980. It removes the file then replaces it, whereas cp truncates and appends in place. Which means that not only is install faster, it is also probably more correct since it won't crash a running program if it is loaded. I verified that cp --remove-dest performs as well as install, but I think it is a gnu extension so it may not work everywhere.

Generated at Thu Feb 08 05:32:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.