2014-11-28T14:22:48.063+0000 [rsHealthPoll] couldn't connect to mongo-arb-6ovf:27017: couldn't connect to server mongo-arb-6ovf:27017 (w.x.y.z) failed, connection attempt failed
|
2014-11-28T14:22:48.103+0000 [rsHealthPoll] warning: Failed to connect to w.x.y.z:27017, reason: errno:111 Connection refused
|
2014-11-28T14:22:48.103+0000 [rsHealthPoll] couldn't connect to mongo-arb-6ovf:27017: couldn't connect to server mongo-arb-6ovf:27017 (w.x.y.z) failed, connection attempt failed
|
2014-11-28T14:22:48.485+0000 [conn24566] replSet replSetReconfig config object parses ok, 5 members specified
|
2014-11-28T14:22:48.578+0000 [conn24566] replSet replSetReconfig [2]
|
2014-11-28T14:22:48.578+0000 [conn24566] replSet info saving a newer config version to local.system.replset: { _id: "wsyrs", version: 8, members: [ { _id: 0, host: "rs0.wsy:27017" }, { _id: 1, host: "rs1.wsy:27017" }, { _id: 3, host: "arb.wsy:27017", arbiterOnly: true }, { _id: 4, host: "mongo-db-6m7q:27017", priority: 0.0 }, { _id: 5, host: "mongo-db-td6n:27017", priority: 0.0 } ] }
|
2014-11-28T14:22:48.579+0000 [conn24566] replSet saveConfigLocally done
|
2014-11-28T14:22:48.580+0000 [conn24566] replSet relinquishing primary state
|
2014-11-28T14:22:48.580+0000 [conn24566] replSet SECONDARY
|
2014-11-28T14:22:48.580+0000 [conn24566] replSet closing client sockets after relinquishing primary
|
2014-11-28T14:22:48.580+0000 [conn24540] end connection 10.128.229.15:59530 (28 connections now open)
|
2014-11-28T14:22:48.580+0000 [conn22615] end connection 10.128.229.18:55955 (28 connections now open)
|
2014-11-28T14:22:48.581+0000 [conn24559] end connection w3.x.y.z:46655 (28 connections now open)
|
2014-11-28T14:22:48.582+0000 [initandlisten] connection accepted from 10.128.229.15:33396 #24654 (30 connections now open)
|
2014-11-28T14:22:48.581+0000 [conn23587] end connection 10.128.229.18:56730 (28 connections now open)
|
2014-11-28T14:22:48.583+0000 [conn24602] end connection w2.x.y.z:48989 (28 connections now open)
|
2014-11-28T14:22:48.584+0000 [conn23598] end connection 10.128.229.18:56748 (28 connections now open)
|
2014-11-28T14:22:48.584+0000 [conn23597] end connection 10.128.229.18:56745 (28 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23603] end connection 10.128.229.18:56771 (28 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23613] end connection 10.128.229.18:56792 (28 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23582] end connection 10.128.229.18:56722 (28 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23599] end connection 10.128.229.18:56751 (28 connections now open)
|
2014-11-28T14:22:48.586+0000 [conn23600] end connection 10.128.229.18:56760 (28 connections now open)
|
2014-11-28T14:22:48.586+0000 [conn24574] end connection 10.128.229.18:57240 (28 connections now open)
|
2014-11-28T14:22:48.587+0000 [conn23621] end connection 10.128.229.18:56816 (27 connections now open)
|
2014-11-28T14:22:48.588+0000 [conn23638] end connection 10.128.229.18:56849 (26 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23605] end connection 10.128.229.18:56775 (28 connections now open)
|
2014-11-28T14:22:48.585+0000 [conn23615] end connection 10.128.229.18:56796 (28 connections now open)
|
2014-11-28T14:22:48.589+0000 [initandlisten] connection accepted from 127.0.0.1:48476 #24655 (23 connections now open)
|
2014-11-28T14:22:48.587+0000 [conn23580] end connection 10.128.229.18:56717 (27 connections now open)
|
2014-11-28T14:22:48.587+0000 [conn23617] end connection 10.128.229.18:56803 (27 connections now open)
|
2014-11-28T14:22:48.587+0000 [conn23645] end connection 10.128.229.18:56860 (27 connections now open)
|
2014-11-28T14:22:48.587+0000 [conn23586] end connection 10.128.229.18:56726 (27 connections now open)
|
2014-11-28T14:22:48.590+0000 [conn23579] end connection 10.128.229.18:56714 (17 connections now open)
|
2014-11-28T14:22:48.594+0000 [conn24539] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [10.128.229.15:59529]
|
2014-11-28T14:22:48.590+0000 [conn24557] end connection w3.x.y.z:46654 (15 connections now open)
|
2014-11-28T14:22:48.594+0000 [conn24601] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [w2.x.y.z:48981]
|
2014-11-28T14:22:48.595+0000 [conn24566] replSet PRIMARY
|
2014-11-28T14:22:48.596+0000 [conn24654] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.597+0000 [conn24655] authenticate db: admin { authenticate: 1, nonce: "xxx", user: "uuuuuuu", key: "xxx" }
|
2014-11-28T14:22:48.598+0000 [conn24566] replSet replSetReconfig new config saved locally
|
2014-11-28T14:22:48.598+0000 [conn24566] command admin.$cmd command: replSetReconfig { replSetReconfig: { _id: "wsyrs", version: 8, members: [ { _id: 0, host: "rs0.wsy:27017" }, { _id: 1, host: "rs1.wsy:27017" }, { _id: 3, host: "arb.wsy:27017", arbiterOnly: true }, { _id: 4, host: "mongo-db-6m7q:27017", priority: 0.0 }, { _id: 5, host: "mongo-db-td6n:27017", priority: 0.0 } ] } } keyUpdates:0 numYields:0 locks(micros) W:6321 w:594 reslen:37 113ms
|
2014-11-28T14:22:48.599+0000 [conn24566] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [127.0.0.1:46724]
|
2014-11-28T14:22:48.599+0000 [rsHealthPoll] replset info arb.wsy:27017 heartbeat failed, retrying
|
2014-11-28T14:22:48.600+0000 [initandlisten] connection accepted from 10.128.229.15:33397 #24656 (7 connections now open)
|
2014-11-28T14:22:48.599+0000 [rsHealthPoll] replSet member rs0.wsy:27017 is up
|
2014-11-28T14:22:48.601+0000 [rsHealthPoll] replSet member rs0.wsy:27017 is now in state SECONDARY
|
2014-11-28T14:22:48.601+0000 [rsMgr] can't see a majority of the set, relinquishing primary
|
2014-11-28T14:22:48.601+0000 [rsMgr] replSet relinquishing primary state
|
2014-11-28T14:22:48.601+0000 [rsMgr] replSet SECONDARY
|
2014-11-28T14:22:48.601+0000 [rsMgr] replSet closing client sockets after relinquishing primary
|
2014-11-28T14:22:48.602+0000 [conn24654] end connection 10.128.229.15:33396 (5 connections now open)
|
2014-11-28T14:22:48.602+0000 [conn24655] end connection 127.0.0.1:48476 (5 connections now open)
|
2014-11-28T14:22:48.603+0000 [conn24656] end connection 10.128.229.15:33397 (3 connections now open)
|
2014-11-28T14:22:48.607+0000 [rsHealthPoll] replSet member arb.wsy:27017 is up
|
2014-11-28T14:22:48.608+0000 [initandlisten] connection accepted from 10.128.229.15:33398 #24657 (4 connections now open)
|
2014-11-28T14:22:48.609+0000 [conn24657] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.611+0000 [initandlisten] connection accepted from 10.128.229.15:33399 #24658 (5 connections now open)
|
2014-11-28T14:22:48.612+0000 [conn24658] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.632+0000 [rsHealthPoll] replSet member mongo-db-td6n:27017 is up
|
2014-11-28T14:22:48.633+0000 [rsHealthPoll] replSet member mongo-db-td6n:27017 is now in state SECONDARY
|
2014-11-28T14:22:48.647+0000 [rsHealthPoll] replSet member mongo-db-6m7q:27017 is up
|
2014-11-28T14:22:48.647+0000 [rsHealthPoll] replSet member mongo-db-6m7q:27017 is now in state SECONDARY
|
2014-11-28T14:22:48.652+0000 [initandlisten] connection accepted from w3.x.y.z:48730 #24659 (6 connections now open)
|
2014-11-28T14:22:48.654+0000 [initandlisten] connection accepted from w3.x.y.z:48731 #24660 (7 connections now open)
|
2014-11-28T14:22:48.661+0000 [initandlisten] connection accepted from w2.x.y.z:50208 #24661 (8 connections now open)
|
2014-11-28T14:22:48.662+0000 [rsHealthPoll] warning: Failed to connect to w.x.y.z:27017, reason: errno:111 Connection refused
|
2014-11-28T14:22:48.683+0000 [conn24659] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.685+0000 [conn24660] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.701+0000 [conn24661] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.703+0000 [rsHealthPoll] warning: Failed to connect to w.x.y.z:27017, reason: errno:111 Connection refused
|
2014-11-28T14:22:48.704+0000 [rsHealthPoll] couldn't connect to mongo-arb-6ovf:27017: couldn't connect to server mongo-arb-6ovf:27017 (w.x.y.z) failed, connection attempt failed
|
2014-11-28T14:22:48.806+0000 [conn24659] end connection w3.x.y.z:48730 (7 connections now open)
|
2014-11-28T14:22:48.818+0000 [initandlisten] connection accepted from w2.x.y.z:50209 #24662 (8 connections now open)
|
2014-11-28T14:22:48.851+0000 [initandlisten] connection accepted from w3.x.y.z:48732 #24663 (9 connections now open)
|
2014-11-28T14:22:48.856+0000 [conn24662] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.889+0000 [conn24663] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:48.972+0000 [conn24661] end connection w2.x.y.z:50208 (8 connections now open)
|
2014-11-28T14:22:49.026+0000 [conn24660] end connection w3.x.y.z:48731 (7 connections now open)
|
2014-11-28T14:22:50.032+0000 [initandlisten] connection accepted from 10.128.229.18:57380 #24664 (8 connections now open)
|
2014-11-28T14:22:50.034+0000 [conn24664] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:22:50.037+0000 [rsMgr] not electing self, mongo-db-td6n:27017 would veto with 'I don't think rs1.wsy:27017 is electable'
|
2014-11-28T14:22:50.120+0000 [conn24653] replSet info voting yea for rs0.wsy:27017 (0)
|
2014-11-28T14:22:50.608+0000 [rsHealthPoll] replSet member arb.wsy:27017 is now in state REMOVED
|
2014-11-28T14:22:51.083+0000 [rsMgr] replSet not trying to elect self as responded yea to someone else recently
|
2014-11-28T14:22:51.247+0000 [rsMgr] replSet not trying to elect self as responded yea to someone else recently
|
2014-11-28T14:22:51.854+0000 [rsMgr] replSet not trying to elect self as responded yea to someone else recently
|
2014-11-28T14:22:53.632+0000 [conn24657] end connection 10.128.229.15:33398 (7 connections now open)
|
2014-11-28T14:22:54.605+0000 [rsHealthPoll] replSet member rs0.wsy:27017 is now in state PRIMARY
|
2014-11-28T14:22:54.610+0000 [rsHealthPoll] replSet member arb.wsy:27017 is now in state ARBITER
|
2014-11-28T14:22:55.039+0000 [rsBackgroundSync] replSet syncing to: rs0.wsy:27017
|
2014-11-28T14:22:55.042+0000 [rsBackgroundSync] replset setting syncSourceFeedback to rs0.wsy:27017
|
2014-11-28T14:22:56.132+0000 [conn24662] end connection w2.x.y.z:50209 (6 connections now open)
|
2014-11-28T14:22:56.203+0000 [conn24663] end connection w3.x.y.z:48732 (5 connections now open)
|
2014-11-28T14:22:59.078+0000 [conn24653] end connection 10.128.229.15:33372 (4 connections now open)
|
2014-11-28T14:22:59.079+0000 [initandlisten] connection accepted from 10.128.229.15:33410 #24665 (5 connections now open)
|
2014-11-28T14:22:59.080+0000 [conn24665] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|
2014-11-28T14:23:01.906+0000 [initandlisten] connection accepted from 10.128.229.18:57385 #24666 (6 connections now open)
|
2014-11-28T14:23:02.315+0000 [conn24666] end connection 10.128.229.18:57385 (5 connections now open)
|
2014-11-28T14:23:03.138+0000 [conn24651] end connection w2.x.y.z:50150 (4 connections now open)
|
2014-11-28T14:23:03.175+0000 [initandlisten] connection accepted from w2.x.y.z:50216 #24667 (5 connections now open)
|
2014-11-28T14:23:03.215+0000 [conn24667] authenticate db: local { authenticate: 1, nonce: "xxx", user: "__system", key: "xxx" }
|