[SERVER-12163] Replica Set failover time is more than 10 sec Created: 19/Dec/13 Updated: 06/Apr/23 Resolved: 19/Dec/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | 2.4.6 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Amit Wankhede | Assignee: | Matt Dannenberg |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
CentOS |
||
| Issue Links: |
|
||||||||||||
| Operating System: | Linux | ||||||||||||
| Steps To Reproduce: | Power the primary DB VM and verify the secondary DB logs. |
||||||||||||
| Participants: | |||||||||||||
| Description |
|
We have three member replica set primary+secondary+arbiter across different virtual machines. To reproduced the issue we power the primary DB. We have observed that failover time is more than 10 secs. Thu Dec 19 02:15:55.994 [rsSyncNotifier] replset setting oplog notifier to sessionmgr01:27717 |
| Comments |
| Comment by Gianfranco Palumbo [ 23/Dec/13 ] |
|
The ticket Matt is referring is This is currently scheduled for 2.7.x (the development version of 2.8) Please click on "Start watching this issue" to receive email updates on the status of the feature request. |
| Comment by Amit Wankhede [ 20/Dec/13 ] |
|
Hi Matt, Can you please provide more insights on this. |
| Comment by Matt Dannenberg [ 20/Dec/13 ] |
|
Turns out there is a "speed up failovers" ticket. It is now linked. |
| Comment by Matt Dannenberg [ 20/Dec/13 ] |
|
At this time, there is no concrete plan to improve replica set failover time specifically. We are planning to rework replica set internals in the near future. We anticipate that this will have a positive effect on failover time. |
| Comment by Amit Wankhede [ 20/Dec/13 ] |
|
Hi Matt, Thanks for your comments. In which future release we will get the fix for failover time? regards, |
| Comment by Matt Dannenberg [ 19/Dec/13 ] |
|
Failover time is not tunable. The way we detect a downed node is by a loss of heartbeats and heartbeat responses. Heartbeat responses time out after 10 seconds and then if we have not received a heartbeat from them in the past two seconds (they are sent every two seconds), we mark them as down. So it is common for the election process to take 10 seconds before it starts. There are many other variables such as the latency between nodes and where or not the first election is successful. As a result, we make no guarantees with regard to how long a failover will take. In a future release, we will be heavily reworking the internals of replication and a side effect of that should be reduced fail over time. |