Related page: /tutorial/add-shards-to-shard-cluster/

I recently found out about the behaviour of the sharding.archiveMovedChunks option (still on by default) when I helped someone add a new shard.

The simplified case: There is a 2-shard cluster with 1 TB on each shard, but the disks were almost full (only 100GB of disk space left). Adding a third shard stopped the increase in db storage (and disk space) and all seemed well. But the disk usage increased on the older shards, quickly, as the balancer moved chunks to the other shards. In a well-balanced cluster each of the original two shards was in the process of migrating 330GB off and making movechunk archives of the same size. If the issue hadn't been noticed in time a disk-full error would have happened.

I had heard of the sharding.archiveMovedChunks (a.k.a. --moveParanoia) but didn't realize it would be on by default. That the migrated chunk's data was being saved outside of the extent files and consuming more diskspace was a real gotcha. As adding shards is something you often do to relieve disk space please add a warning explaining this in the addShard documentation.