[SERVER-66023] Do not constantly reset election and liveness timers Created: 27/Apr/22 Updated: 29/Oct/23 Resolved: 18/May/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 6.0.1, 5.0.11, 6.1.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Matthew Russotto | Assignee: | Matthew Russotto |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Backport Requested: |
v6.0, v5.3, v5.0, v4.4
|
||||||||||||||||
| Sprint: | Repl 2022-05-02, Repl 2022-05-16, Repl 2022-05-30 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 135 | ||||||||||||||||
| Description |
|
We cancel the election timeout on secondaries whenever the primary liveness is updated, which potentially happens on every oplog batch. We cancel the liveness timeout on primaries whenever the oldest secondary liveness is updated, which potentially happens on every replSetUpdatePosition. It turns out cancelling a timer, at least on Linux, is quite expensive (likely system call overhead), and we do this in the replication lock, which increases contention on that already-hot mutex. We can greatly reduce this with a class which handles "cancel and reschedule" by keeping track of the latest time of the reschedule, and then when the timeout occurs, reschedules at that point instead of immediately. This means we get no cancels and one reschedule every timeout interval (not every miniscule bump forward of the timer) |
| Comments |
| Comment by Githook User [ 27/Jul/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: (cherry picked from commit 94f976f475b7d26e75f60d0e0733aed99bb6f272) |
| Comment by Githook User [ 27/Jul/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: (cherry picked from commit 466ddbc1784709f3421ed9ccc485210b1fe94e6e) |
| Comment by Githook User [ 27/Jul/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: (cherry picked from commit 94f976f475b7d26e75f60d0e0733aed99bb6f272) |
| Comment by Githook User [ 27/Jul/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: (cherry picked from commit 466ddbc1784709f3421ed9ccc485210b1fe94e6e) |
| Comment by Githook User [ 18/May/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: |
| Comment by Githook User [ 18/May/22 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}Message: |