Cannot select sync source because we are a voter and it is not

XMLWordPrintableJSON

    • Replication
    • Fully Compatible
    • v8.0, v7.0, v6.0, v5.0
    • Repl 2024-02-19, Repl 2024-03-04, Repl 2024-03-18, Repl 2024-04-01, Repl 2024-05-13, Repl 2024-05-27, Repl 2024-06-10, Repl 2024-10-14, Repl 2024-10-28
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      After upgrade RS from 6 to 7.0.5 we are getting this error on one replica:

      {"t":{"$date":"2024-01-30T09:22:47.768+00:00"},"s":"I",  "c":"REPL",     "id":3873108, "ctx":"ReplCoordExtern-0","msg":"Cannot select sync source because we are a voter and it is not","attr":{"syncSourceCandidate":"example5.com:27017"}}

      nodes with votes: 1, 2, 3
      I see no other problems: lag is 0, replica works ok. But there is constant spam with this message into log (around 30 messages per second)

      result of rs.conf() command from example3.com (problem node)

      {
  set: 'exampleRS',
  date: ISODate('2024-01-30T09:24:24.943Z'),
  myState: 2,
  term: Long('161'),
  syncSourceHost: 'example2.com:27017',
  syncSourceId: 1,
  heartbeatIntervalMillis: Long('2000'),
  majorityVoteCount: 2,
  writeMajorityCount: 2,
  votingMembersCount: 3,
  writableVotingMembersCount: 3,
  optimes: {
    lastCommittedOpTime: { ts: Timestamp({ t: 1706606664, i: 5 }), t: Long('161') },
    lastCommittedWallTime: ISODate('2024-01-30T09:24:24.676Z'),
    readConcernMajorityOpTime: { ts: Timestamp({ t: 1706606664, i: 5 }), t: Long('161') },
    appliedOpTime: { ts: Timestamp({ t: 1706606664, i: 5 }), t: Long('161') },
    durableOpTime: { ts: Timestamp({ t: 1706606664, i: 5 }), t: Long('161') },
    lastAppliedWallTime: ISODate('2024-01-30T09:24:24.676Z'),
    lastDurableWallTime: ISODate('2024-01-30T09:24:24.676Z')
  },
  lastStableRecoveryTimestamp: Timestamp({ t: 1706606606, i: 7 }),
  members: [
    {
      _id: 0,
      name: 'example1.com:27017',
      health: 1,
      state: 1,
      stateStr: 'PRIMARY',
      uptime: 662,
      optime: { ts: Timestamp({ t: 1706606663, i: 2 }), t: Long('161') },
      optimeDurable: { ts: Timestamp({ t: 1706606663, i: 2 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:23.000Z'),
      optimeDurableDate: ISODate('2024-01-30T09:24:23.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:23.414Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:23.414Z'),
      lastHeartbeat: ISODate('2024-01-30T09:24:23.925Z'),
      lastHeartbeatRecv: ISODate('2024-01-30T09:24:23.062Z'),
      pingMs: Long('178'),
      lastHeartbeatMessage: '',
      syncSourceHost: '',
      syncSourceId: -1,
      infoMessage: '',
      electionTime: Timestamp({ t: 1706519832, i: 1 }),
      electionDate: ISODate('2024-01-29T09:17:12.000Z'),
      configVersion: 20,
      configTerm: 161
    },
    {
      _id: 1,
      name: 'example2.com:27017',
      health: 1,
      state: 2,
      stateStr: 'SECONDARY',
      uptime: 662,
      optime: { ts: Timestamp({ t: 1706606662, i: 20 }), t: Long('161') },
      optimeDurable: { ts: Timestamp({ t: 1706606662, i: 20 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:22.000Z'),
      optimeDurableDate: ISODate('2024-01-30T09:24:22.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:22.626Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:22.626Z'),
      lastHeartbeat: ISODate('2024-01-30T09:24:22.925Z'),
      lastHeartbeatRecv: ISODate('2024-01-30T09:24:23.752Z'),
      pingMs: Long('176'),
      lastHeartbeatMessage: '',
      syncSourceHost: 'example1.com:27017',
      syncSourceId: 0,
      infoMessage: '',
      configVersion: 20,
      configTerm: 161
    },
    {
      _id: 2,
      name: 'example3.com:27017',
      health: 1,
      state: 2,
      stateStr: 'SECONDARY',
      uptime: 666,
      optime: { ts: Timestamp({ t: 1706606664, i: 5 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:24.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:24.676Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:24.676Z'),
      syncSourceHost: 'example2.com:27017',
      syncSourceId: 1,
      infoMessage: '',
      configVersion: 20,
      configTerm: 161,
      self: true,
      lastHeartbeatMessage: ''
    },
    {
      _id: 3,
      name: 'example4.com:27017',
      health: 1,
      state: 2,
      stateStr: 'SECONDARY',
      uptime: 662,
      optime: { ts: Timestamp({ t: 1706606663, i: 2 }), t: Long('161') },
      optimeDurable: { ts: Timestamp({ t: 1706606663, i: 2 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:23.000Z'),
      optimeDurableDate: ISODate('2024-01-30T09:24:23.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:23.414Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:23.414Z'),
      lastHeartbeat: ISODate('2024-01-30T09:24:23.506Z'),
      lastHeartbeatRecv: ISODate('2024-01-30T09:24:23.999Z'),
      pingMs: Long('175'),
      lastHeartbeatMessage: '',
      syncSourceHost: 'example2.com:27017',
      syncSourceId: 1,
      infoMessage: '',
      configVersion: 20,
      configTerm: 161
    },
    {
      _id: 4,
      name: 'example5.com:27017',
      health: 1,
      state: 2,
      stateStr: 'SECONDARY',
      uptime: 662,
      optime: { ts: Timestamp({ t: 1706606663, i: 1 }), t: Long('161') },
      optimeDurable: { ts: Timestamp({ t: 1706606663, i: 1 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:23.000Z'),
      optimeDurableDate: ISODate('2024-01-30T09:24:23.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:23.237Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:23.237Z'),
      lastHeartbeat: ISODate('2024-01-30T09:24:23.385Z'),
      lastHeartbeatRecv: ISODate('2024-01-30T09:24:23.248Z'),
      pingMs: Long('115'),
      lastHeartbeatMessage: '',
      syncSourceHost: 'example2.com:27017',
      syncSourceId: 1,
      infoMessage: '',
      configVersion: 20,
      configTerm: 161
    },
    {
      _id: 5,
      name: 'example6.com:27017',
      health: 1,
      state: 2,
      stateStr: 'SECONDARY',
      uptime: 661,
      optime: { ts: Timestamp({ t: 1706606664, i: 2 }), t: Long('161') },
      optimeDurable: { ts: Timestamp({ t: 1706606664, i: 2 }), t: Long('161') },
      optimeDate: ISODate('2024-01-30T09:24:24.000Z'),
      optimeDurableDate: ISODate('2024-01-30T09:24:24.000Z'),
      lastAppliedWallTime: ISODate('2024-01-30T09:24:24.251Z'),
      lastDurableWallTime: ISODate('2024-01-30T09:24:24.251Z'),
      lastHeartbeat: ISODate('2024-01-30T09:24:24.729Z'),
      lastHeartbeatRecv: ISODate('2024-01-30T09:24:24.891Z'),
      pingMs: Long('328'),
      lastHeartbeatMessage: '',
      syncSourceHost: 'example2.com:27017',
      syncSourceId: 1,
      infoMessage: '',
      configVersion: 20,
      configTerm: 161
    }
  ]

      example5.com is not a voting member, just secondary node. I tried to restart problem member, but message still present

            Assignee:
            Ruchitha Rajaghatta
            Reporter:
            Andrii K
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: