Details
-
Question
-
Resolution: Done
-
Major - P3
-
None
-
None
-
None
Description
We start seeing the following messages in the logs and was just wondering why it's happening:
2015-05-21T10:07:24.053-0700 I REPL [ReplicationExecutor] Error in heartbeat request to ir-mngs3-l002w.sfoprod.local:27017; Location18915 Failed attempt to connect to ir-mngs3-l002w.sfoprod.local:27017; couldn't connect to server ir-mngs3-l002w.sfoprod.local:27017 (172.16.128.38), connection attempt failed
|
2015-05-21T10:07:25.440-0700 W NETWORK [LockPinger] Failed to connect to 172.16.128.61:27017 after 5000 milliseconds, giving up.
|
2015-05-21T10:07:25.440-0700 I NETWORK [LockPinger] SyncClusterConnection connect fail to: ir-mngc-l002w.sfoprod.local:27017 errmsg: couldn't connect to server ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61), connection attempt failed
|
2015-05-21T10:07:25.440-0700 I NETWORK [LockPinger] SyncClusterConnection connecting to [ir-mngc-l002c.sfoprod.local:27017]
|
2015-05-21T10:07:25.443-0700 I NETWORK [LockPinger] SyncClusterConnection connecting to [ir-mngc-l002e.sfoprod.local:27017]
|
2015-05-21T10:07:25.480-0700 I NETWORK [LockPinger] trying reconnect to ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61) failed
|
2015-05-21T10:07:27.139-0700 I REPL [ReplicationExecutor] syncing from: ir-mngs3-l002e.sfoprod.local:27017
|
2015-05-21T10:07:27.208-0700 I REPL [SyncSourceFeedback] replset setting syncSourceFeedback to ir-mngs3-l002e.sfoprod.local:27017
|
2015-05-21T10:07:27.309-0700 I REPL [rsBackgroundSync] replSet our last op time fetched: May 21 10:06:50:9f
|
2015-05-21T10:07:27.309-0700 I REPL [rsBackgroundSync] replset source's GTE: May 21 10:07:25:1
|
2015-05-21T10:07:27.309-0700 I REPL [rsBackgroundSync] beginning rollback
|
2015-05-21T10:07:27.309-0700 I REPL [rsBackgroundSync] rollback 0
|
2015-05-21T10:07:27.310-0700 I REPL [ReplicationExecutor] transition to ROLLBACK
|
2015-05-21T10:07:27.310-0700 I REPL [rsBackgroundSync] rollback 1
|
2015-05-21T10:07:27.310-0700 I REPL [rsBackgroundSync] rollback 2 FindCommonPoint
|
2015-05-21T10:07:27.379-0700 I REPL [rsBackgroundSync] replSet info rollback our last optime: May 21 10:06:50:9f
|
2015-05-21T10:07:27.379-0700 I REPL [rsBackgroundSync] replSet info rollback their last optime: May 21 10:07:25:14
|
2015-05-21T10:07:27.379-0700 I REPL [rsBackgroundSync] replSet info rollback diff in end of log times: -35 seconds
|
2015-05-21T10:07:27.434-0700 I REPL [rsBackgroundSync] replSet rollback found matching events at May 21 10:06:29:255
|
2015-05-21T10:07:27.434-0700 I REPL [rsBackgroundSync] replSet rollback findcommonpoint scanned : 8873
|
2015-05-21T10:07:27.434-0700 I REPL [rsBackgroundSync] replSet rollback 3 fixup
|
2015-05-21T10:07:28.628-0700 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 172.16.128.36:27017, reason: errno:115 Operation now in progress
|
2015-05-21T10:07:28.629-0700 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set s2
|
2015-05-21T10:07:29.049-0700 W NETWORK [ReplExecNetThread-130] Failed to connect to 172.16.128.38:27017, reason: errno:115 Operation now in progress
|
2015-05-21T10:07:29.049-0700 I REPL [ReplicationExecutor] Error in heartbeat request to ir-mngs3-l002w.sfoprod.local:27017; Location18915 Failed attempt to connect to ir-mngs3-l002w.sfoprod.local:27017; couldn't connect to server ir-mngs3-l002w.sfoprod.local:27017 (172.16.128.38), connection attempt failed
|
2015-05-21T10:07:29.672-0700 I NETWORK [initandlisten] connection accepted from 172.17.128.94:44923 #9766 (38 connections now open)
|
2015-05-21T10:07:30.484-0700 W NETWORK [LockPinger] Failed to connect to 172.16.128.61:27017 after 5000 milliseconds, giving up.
|
2015-05-21T10:07:30.484-0700 I NETWORK [LockPinger] reconnect ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61) failed failed couldn't connect to server ir-mngc-l002w.sfoprod.local:27017 (172.16.128.61), connection attempt failed
|
2015-05-21T10:07:30.594-0700 I NETWORK [LockPinger] scoped connection to ir-mngc-l002w.sfoprod.local:27017,ir-mngc-l002c.sfoprod.local:27017,ir-mngc-l002e.sfoprod.local:27017 not being returned to the pool
|
Attachments
Issue Links
- is duplicated by
-
SERVER-18817 rollback problem
-
- Closed
-