We cancel the election timeout on secondaries whenever the primary liveness is updated, which potentially happens on every oplog batch. We cancel the liveness timeout on primaries whenever the oldest secondary liveness is updated, which potentially happens on every replSetUpdatePosition. It turns out cancelling a timer, at least on Linux, is quite expensive (likely system call overhead), and we do this in the replication lock, which increases contention on that already-hot mutex.
We can greatly reduce this with a class which handles "cancel and reschedule" by keeping track of the latest time of the reschedule, and then when the timeout occurs, reschedules at that point instead of immediately. This means we get no cancels and one reschedule every timeout interval (not every miniscule bump forward of the timer)
- is related to
SERVER-59776 50% regression in single multi-update