Our test infrastructure copies data files for archival while a process is still running. The reason being that shutting down mongod may modify those files, making debugging more challenging.

If a checkpoint is active in WiredTiger, the data files will become completely inconsistent and unusable (e.g. copy a data file, then copy the WT metadata which can point to a new checkpoint absent in the already-copied data file). We should find a way to stop checkpoints, run fsyncLock or just SIGKILL the process before copying data files. I think SIGKILL is the simplest approach and would guarantee no files are modified before archival.

Here's a task where the data files are corrupt on node1 because the files were copied during an active checkpoint.