Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13233

Arbiter-Only note becomes Secondary

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Incomplete
    • Icon: Minor - P4 Minor - P4
    • None
    • None
    • Replication
    • None
    • ALL

    Description

      This happened in a three member replica set:
      1. Member_1: Data node
      2. Member_2 : Data node
      3. Member_3: Arbiter-Only node
      At some point of time when Member_1 was Primary and Member_2 was Secondary and Member_3 was Arbiter I brought down Member_2 node to do some hardware repairs and upgrades.
      Member_2 was down for a couple of hours. What I noticed just before bringing Member_2 node up again is that Member_1 is, as expected, Primary, but Member_3 is not Arbiter anymore but rather SECONDAY!
      Somehow it is promoted itself. Its Secondary status was evident from connecting to it from the shell: it would display a prompt with “SECONDARY” label in it. Also it would show up as Secondary when rs.status() command was executed.
      Furthermore, the data directory if Member_3 was showing some new files which were collections data files from the rest of the replica set. The total size of these files was 10 GB at that time.
      I brought up Member_2 and continued monitoring the situation. After several hours my replica set had one Primary and two Secondaries: Member_3 continued to be secondary.
      Furthermore, the data directory for Member_3 continued to grow to 60GB (which is the size of Member_1 data directory). Which means that by that time it fully synced with the Primary.
      And so Member_3 continued to be secondary until I removed it from replica set, restarted Member_3 instance and added it back to the Replica Set. After that it carried on as Arbiter.

      Attachments

        Activity

          People

            Unassigned Unassigned
            resheto@yahoo.com Henry Resheto
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: