Primary vote for stepdown is unreasonably slow.

I setup a fresh v3.2 cluster with 3 mongos, 3 config(as a replica) and 9 mongod(each 3 for a shard). It's completely empty now with only auth info. When I issue stepDown on a primary of a shard, I have to wait about 2 seconds before see the new primary. It's much slower than v3.0. During this time, the shard isn't available. Since I issue the command manually on an empty replica set, I believe the new primary should be vote immediately(scheduled maintenance for example), it's different from detect a failure.