Consider enforcing more strict requirements for liveliness checks

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Replication
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Currently a secondary will postpone its election timeout if we receive a heartbeat response from the primary node. This means if the primary is in a bad state and is unable to send heartbeat requests to secondaries, but it can respond to heartbeats, it will never step down.

      This can lead to a scenario where a mongod that cannot do any work (due to a host issue leading to CPU / scheduling issues) is still alive and stays primary, when a failover would bring the replica set back to being healthy and accepting incoming connections.

      One potential solution is to have secondaries also check if they have received a heartbeat request from the primary within their election timeout

      The consequences of doing nothing here is leaving open a scenario where we lose availability on the cluster until there is manual intervention done via a forced stepup / failover on the cluster

            Assignee:
            Unassigned
            Reporter:
            Sean Zimmerman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: