Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18816

rollback messages in logs

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Replication
    • Labels:
      None

      We start seeing the following messages in the logs and was just wondering why it's happening:

      2015-05-21T10:07:24.053-0700 I REPL     [ReplicationExecutor] Error in heartbeat request to ir-mngs3-l002w.sfoprod.local:27017; Location18915 Failed attempt to connect to ir-mngs3-l002w.sfoprod.local:27017; couldn't connect to server ir-mngs3-l002w.sfoprod.local:27017 (172.16.128.38), connection attempt failed
      2015-05-21T10:07:25.440-0700 W NETWORK  [LockPinger] Failed to connect to 172.16.128.61:27017 after 5000 milliseconds, giving up.
      2015-05-21T10:07:25.440-0700 I NETWORK  [LockPinger] SyncClusterConnection connect fail to: ir-mngc-l002w.sfoprod.local:27017 errmsg: couldn't connect to server ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61), connection attempt failed
      2015-05-21T10:07:25.440-0700 I NETWORK  [LockPinger] SyncClusterConnection connecting to [ir-mngc-l002c.sfoprod.local:27017]
      2015-05-21T10:07:25.443-0700 I NETWORK  [LockPinger] SyncClusterConnection connecting to [ir-mngc-l002e.sfoprod.local:27017]
      2015-05-21T10:07:25.480-0700 I NETWORK  [LockPinger] trying reconnect to ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61) failed
      2015-05-21T10:07:27.139-0700 I REPL     [ReplicationExecutor] syncing from: ir-mngs3-l002e.sfoprod.local:27017
      2015-05-21T10:07:27.208-0700 I REPL     [SyncSourceFeedback] replset setting syncSourceFeedback to ir-mngs3-l002e.sfoprod.local:27017
      2015-05-21T10:07:27.309-0700 I REPL     [rsBackgroundSync] replSet our last op time fetched: May 21 10:06:50:9f
      2015-05-21T10:07:27.309-0700 I REPL     [rsBackgroundSync] replset source's GTE: May 21 10:07:25:1
      2015-05-21T10:07:27.309-0700 I REPL     [rsBackgroundSync] beginning rollback
      2015-05-21T10:07:27.309-0700 I REPL     [rsBackgroundSync] rollback 0
      2015-05-21T10:07:27.310-0700 I REPL     [ReplicationExecutor] transition to ROLLBACK
      2015-05-21T10:07:27.310-0700 I REPL     [rsBackgroundSync] rollback 1
      2015-05-21T10:07:27.310-0700 I REPL     [rsBackgroundSync] rollback 2 FindCommonPoint
      2015-05-21T10:07:27.379-0700 I REPL     [rsBackgroundSync] replSet info rollback our last optime:   May 21 10:06:50:9f
      2015-05-21T10:07:27.379-0700 I REPL     [rsBackgroundSync] replSet info rollback their last optime: May 21 10:07:25:14
      2015-05-21T10:07:27.379-0700 I REPL     [rsBackgroundSync] replSet info rollback diff in end of log times: -35 seconds
      2015-05-21T10:07:27.434-0700 I REPL     [rsBackgroundSync] replSet rollback found matching events at May 21 10:06:29:255
      2015-05-21T10:07:27.434-0700 I REPL     [rsBackgroundSync] replSet rollback findcommonpoint scanned : 8873
      2015-05-21T10:07:27.434-0700 I REPL     [rsBackgroundSync] replSet rollback 3 fixup
      2015-05-21T10:07:28.628-0700 W NETWORK  [ReplicaSetMonitorWatcher] Failed to connect to 172.16.128.36:27017, reason: errno:115 Operation now in progress
      2015-05-21T10:07:28.629-0700 W NETWORK  [ReplicaSetMonitorWatcher] No primary detected for set s2
      2015-05-21T10:07:29.049-0700 W NETWORK  [ReplExecNetThread-130] Failed to connect to 172.16.128.38:27017, reason: errno:115 Operation now in progress
      2015-05-21T10:07:29.049-0700 I REPL     [ReplicationExecutor] Error in heartbeat request to ir-mngs3-l002w.sfoprod.local:27017; Location18915 Failed attempt to connect to ir-mngs3-l002w.sfoprod.local:27017; couldn't connect to server ir-mngs3-l002w.sfoprod.local:27017 (172.16.128.38), connection attempt failed
      2015-05-21T10:07:29.672-0700 I NETWORK  [initandlisten] connection accepted from 172.17.128.94:44923 #9766 (38 connections now open)
      2015-05-21T10:07:30.484-0700 W NETWORK  [LockPinger] Failed to connect to 172.16.128.61:27017 after 5000 milliseconds, giving up.
      2015-05-21T10:07:30.484-0700 I NETWORK  [LockPinger] reconnect ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61) failed failed couldn't connect to server ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61), connection attempt failed
      2015-05-21T10:07:30.594-0700 I NETWORK  [LockPinger] scoped connection to ir-mngc-l002w.sfoprod.local:27017,ir-mngc-l002c.sfoprod.local:27017,ir-mngc-l002e.sfoprod.local:27017 not being returned to the pool
      

            Assignee:
            sam.kleinman Sam Kleinman (Inactive)
            Reporter:
            van.pham@wbgames.com Van Pham
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: