Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-35738

Mongodb bad performance causes heartbeat lose

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • 3.2.8
    • Replication
    • None
    • ALL
    • Hide

      we don't konw

      Show
      we don't konw

    Description

      Secondary became primary through election when secondary can not find primary.

      Before this time, many time-consuming operations happened at the primary node, which can be seen in the attach file.

      primary 10.0.208.149,important log:
      2018-06-17T04:06:44.004 end connection 10.0.189.100:42238 (2829 connections now open)
      2018-06-17T04:06:45.770 end connection 10.0.189.100:36520 (2827 connections now open)
      2018-06-17T04:06:51.82 [ReplicationExecutor] Error in heartbeat request to 10.0.189.100:27017; ExceededTimeLimit: Operation timed out

      2018-06-17T04:06:58.242 connection accepted from 10.0.189.100:39064 #206971
      2018-06-17T04:06:58.242 [NetworkInterfaceASIO-Replication-0] Successfully connected to 10.0.189.100:27017

      2018-06-17T04:06:59.888 [ReplicationExecutor] Member 10.0.189.100:27017 is now in state SECONDARY
      2018-06-17T04:06:59.890 [initandlisten] connection accepted from 10.0.189.100:39066 #2069721 (2831 connections now open)
      2018-06-17T04:06:59.890 end connection 10.0.189.100:39065 (2830 connections now open)

      secondry 10.0.189.100,important log

      2018-06-17T04:06:41.460 [ReplicationExecutor] could not find member to sync from

      2018-06-17T04:06:58.239+0000 [ReplicationExecutor] Starting an election, since we've seen no PRIMARY in the past 10000ms
      2018-06-17T04:06:58.240+0000 I REPL [ReplicationExecutor] conducting a dry run election to see if we could be elected

      2018-06-17T04:06:58.243+0000 I REPL [ReplicationExecutor] VoteRequester: Got no vote from 10.0.208.149:27017 because: candidate's data is staler than mine, resp:

      { term: 95, voteGranted: false, reason: "candidate's data is staler than mine", ok: 1.0 }

      Attachments

        1. 2018-06-17T04-primary.log
          3.27 MB
        2. 2018-06-17T04-secondary.log
          3.11 MB
        3. 2018-06-17T04-secondary-with-priority-0.log
          41 kB
        4. backup.zip
          6.44 MB
        5. primary.zip
          3.43 MB
        6. secondary.zip
          8.73 MB

        Activity

          People

            nick.brewer Nick Brewer
            cuizihang Zihang Cui
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: