It's unclear exactly why this is happening only on 5.0, but after shard removal, there's a ShardRegistry refresh process which will eventually run, and will cause RSM removal. If an addShard command immediately proceeds a removeShard, the addShard can occasionally try to use the removed RSM and fail with this error. We should be retrying this.