Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32802

LVM snapshot restored node is added as member with invalid config error

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.4.2
    • Component/s: Replication
    • Labels:
      None

      Hi Team,
      Below is the issue :-
      --> Lets say NODE 2 is restored using lvm sanpshot backup of primary
      --> If mongodb is running on primary, lets say NODE1. Then Node2 is added as member of replica set. Then below error is thrown

      REPL [ReplicationExecutor] Error in heartbeat request to NODE2 ; InvalidReplicaSetConfig: Our replica set configuration is invalid or does not include us

      and primary NODE1 starts initial sync with secondary despite of fact that data is already present in NODE 2 data directory as it is snapshot backup of NODE1
      and NODE2 is in startup phase though data vol is increasing because of initial SYNC

      ---> If mongodb is running on primary, lets say NODE1.We shutdown NODE1 using db.shutdowServer(

      {force:true}

      ) command. Then, We, start NODE1 and add NODE2 as secondary, It becomes Secondary and inital SYNC is avoided, but throw below error

      018-01-19T12:57:50.229-0500 I NETWORK [thread1] connection accepted from NODE2:60570 #49 (5 connections now open)
      2018-01-19T12:57:50.229-0500 I REPL [ReplicationExecutor] Error in heartbeat request to NODE2:27021; InvalidReplicaSetConfig: Our replica set configuration is invalid or does not include us
      2018-01-19T12:57:50.230-0500 I REPL [ReplicationExecutor] Error in heartbeat request to NODE2:27021; InvalidReplicaSetConfig: Our replica set configuration is invalid or does not include us
      2018-01-19T12:57:50.231-0500 I - [conn49] end connection NODE2:60570 (6 connections now open)
      2018-01-19T12:57:50.231-0500 I NETWORK [thread1] connection accepted from NODE2:60574 #50 (5 connections now open)
      2018-01-19T12:57:50.231-0500 I REPL [ReplicationExecutor] Error in heartbeat request to NODE2:27021; InvalidReplicaSetConfig: Our replica set configuration is invalid or does not include us
      2018-01-19T12:57:50.233-0500 I - [conn50] end connection NODE2:60574 (6 connections now open)
      2018-01-19T12:57:50.236-0500 I NETWORK [thread1] connection accepted from NODE2 #51 (5 connections now open)
      2018-01-19T12:57:50.237-0500 I - [conn51] end connection NODE2:60578 (6 connections now open)
      2018-01-19T12:57:52.232-0500 I REPL [ReplicationExecutor] Member NODE2:27021 is now in state SECONDARY

      The question is why this error is comming in mongo always and and what can be done to avoid that.
      REPL [ReplicationExecutor] Error in heartbeat request to NODE2 ; InvalidReplicaSetConfig: Our replica set configuration is invalid or does not include us

      Is there a way i can add node as secondary without doing shutdownServer command and intialsync using LVM SNAPSHOT BACKUP

            Assignee:
            mark.agarunov Mark Agarunov
            Reporter:
            ashutoshpandey89 Ashutosh
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: