A shard "r3" was replaced with a new server (same shard name). One mongos process recognised the change. The second and third mongos processes had a stale picture of which nodes made up rs3.
Once rs3 was re-added, the balancer (on one of the mongos nodes) started balancing to it's picture of "r3". Remaining two nodes kept looking for the old rs3.