Doing upserts on a sharded collection via a findAndModify will not initiate chunk splits (and subsequent balancing).
I've attached a javascript file that shows the correct behavior for a regular upsert and reproduces the bug behavior for a findAndModify upsert.
Before running, please start a sharded cluster using the mongo snippet with chunksize=1 or increase n.