Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-25842

Secondary started accepting queries before syncing(replSet still syncing, not yet to minValid optime)

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Works as Designed
    • 2.6.11
    • None
    • Querying, Replication
    • None
    • ALL
    • Hide
      1. Add a replica which is out of sync with the primary(2 days in this case)
      2. Observe in the logs of new replica for:

        replSet still syncing, not yet to minValid optime 57bb0c9d:f

      3. Also check the rs.status() of the replicaset whoich will show the state of new replica member as "SECONDARY" (which I guess should be "STARTUP2" or "RECOVERY")
      4. Any application using this replicaset will now start sending queries to this member
      Show
      Add a replica which is out of sync with the primary(2 days in this case) Observe in the logs of new replica for: replSet still syncing, not yet to minValid optime 57bb0c9d:f Also check the rs.status() of the replicaset whoich will show the state of new replica member as "SECONDARY" (which I guess should be "STARTUP2" or "RECOVERY") Any application using this replicaset will now start sending queries to this member

    Description

      I have brought up a new replica from EBS snapshot(2 days old) of an existing secondary and then added it to the replicaset using rs.add(). But once added it started serving queries immediately even before fully syncing with the primary.

      rs.status()

      also shows status of this member as "SECONDARY"

      Logs:

      2016-08-23T05:32:42.470+0000 [rsBackgroundSync] replSet syncing to: 10.0.1.211:27017
      2016-08-23T05:32:42.470+0000 [rsSync] replSet still syncing, not yet to minValid optime 57bb0c9d:f
      2016-08-23T05:32:42.471+0000 [rsBackgroundSync] replset setting syncSourceFeedback to 10.0.1.211:27017
      2016-08-23T05:32:43.635+0000 [initandlisten] connection accepted from 127.0.0.1:43066 #104 (104 connections now open)
      2016-08-23T05:32:43.636+0000 [conn104]  authenticate db: admin { authenticate: 1, nonce: "xxx", user: "admin", key: "xxx" }
      2016-08-23T05:32:51.112+0000 [rsSync] replSet SECONDARY
      2016-08-23T05:32:51.182+0000 [initandlisten] connection accepted from 10.0.0.31:39528 #105 (105 connections now open)
      2016-08-23T05:32:51.188+0000 [conn105]  authenticate db: <db> { authenticate: 1, user: <user>, nonce: "xxx", key: "xxx" }
      2016-08-23T05:32:51.190+0000 [initandlisten] connection accepted from 10.0.0.31:39530 #106 (106 connections now open)
      2016-08-23T05:32:51.190+0000 [conn106]  authenticate db: <db> { authenticate: 1, user: <user>, nonce: "xxx", key: "xxx" }
      2016-08-23T05:32:51.357+0000 [initandlisten] connection accepted from 10.0.0.31:39542 #107 (107 connections now open)
      2016-08-23T05:32:51.357+0000 [initandlisten] connection accepted from 10.0.0.31:39544 #108 (108 connections now open)
      2016-08-23T05:32:51.357+0000 [conn107]  authenticate db: <db> { authenticate: 1, user: <user>, nonce: "xxx", key: "xxx" }
      2016-08-23T05:32:51.357+0000 [conn108]  authenticate db: <db> { authenticate: 1, user: <user>, nonce: "xxx", key: "xxx" }
      2016-08-23T05:32:51.431+0000 [conn105] query <db.coll> query: { <query> } planSummary: IDHACK ntoskip:0 nscanned:1 nscannedObjects:1 idhack:1 keyUpdates:0 numYields:1 locks(micros) r:336 nreturned:1 reslen:2081 131ms
      2016-08-23T05:32:51.455+0000 [initandlisten] connection accepted from 10.0.0.31:39558 #109 (109 connections now open)
      2016-08-23T05:32:51.456+0000 [conn109]  authenticate db: <db> { authenticate: 1, user: <user>, nonce: "xxx", key: "xxx" }
      2016-08-23T05:32:51.463+0000 [conn107] query <db.coll> query: { <query> } planSummary: IXSCAN { <index> } ntoreturn:0 ntoskip:0 nscanned:1 nscannedObjects:1 keyUpdates:0 numYields:1 locks(micros) r:206862 nreturned:1 reslen:367 104ms
      2016-08-23T05:32:51.534+0000 [initandlisten] connection accepted from 10.0.0.41:46923 #110 (110 connections now open)
      

      Made a workaround by removing and making a hidden secondary till it got synced.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              vivek_jonam Manoj Vivek
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: