Wed Mar 20 16:34:58 [conn114] query bq2.keyword query: { query: { path: { $regex: "^[object]" } }, orderby: { nm: 1 } } ntoreturn:0 ntoskip:0 nscanned:32118 scanAndOrder:1 keyUpdates:0 locks(micros) r:110984 nreturned:8589 reslen:806643 110ms
Wed Mar 20 18:02:07 [rsHealthPoll] DBClientCursor::init call() failed
Wed Mar 20 18:02:07 [rsHealthPoll] replSet info o16.servername.com:27017 is down (or slow to respond): DBClientBase::findN: transport error: o16.servername.com:27017 ns: admin.$cmd query: { replSetHeartbeat: "bqset", v: 252399, pv: 1, checkEmpty: false, from: "o15.servername.com:27017", $auth: {} }
Wed Mar 20 18:02:07 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state DOWN
Wed Mar 20 18:02:07 [rsMgr] replSet info electSelf 1
Wed Mar 20 18:02:07 [rsMgr] replSet PRIMARY
Wed Mar 20 18:02:22 [rsHealthPoll] replSet member o16.servername.com:27017 is up
Wed Mar 20 18:02:22 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state PRIMARY
Wed Mar 20 18:02:22 [rsMgr] replSet relinquishing primary state
Wed Mar 20 18:02:22 [rsMgr] replSet SECONDARY
Wed Mar 20 18:02:22 [rsMgr] replSet closing client sockets after relinquishing primary
Wed Mar 20 18:02:24 [rsHealthPoll] DBClientCursor::init call() failed
Wed Mar 20 18:02:24 [rsHealthPoll] replSet info o16.servername.com:27017 is down (or slow to respond): DBClientBase::findN: transport error: o16.servername.com:27017 ns: admin.$cmd query: { replSetHeartbeat: "bqset", v: 252399, pv: 1, checkEmpty: false, from: "o15.servername.com:27017", $auth: {} }
Wed Mar 20 18:02:24 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state DOWN
Wed Mar 20 18:02:24 [rsMgr] not electing self, 14.servername.com:27017 would veto
Wed Mar 20 18:02:27 [rsHealthPoll] replSet member o16.servername.com:27017 is up
Wed Mar 20 18:02:27 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state PRIMARY
Wed Mar 20 18:02:42 [rsBackgroundSync] Socket recv() errno:0 Success 128.121.86.42:27017
Wed Mar 20 18:02:42 [rsBackgroundSync] SocketException: remote: 128.121.86.42:27017 error: 9001 socket exception [1] server [128.121.86.42:27017] 
Wed Mar 20 18:02:42 [rsBackgroundSync] replSet db exception in producer: 10278 dbclient error communicating with server: o16.servername.com:27017
Wed Mar 20 18:02:43 [conn7437] OpCounters::gotOp unknown op: 0
Wed Mar 20 18:02:43 [conn7437]     operation isn't supported: 0
Wed Mar 20 18:02:43 [conn7437] none  keyUpdates:0  0ms
Wed Mar 20 18:02:52 [rsBackgroundSync] replSet syncing to: o16.servername.com:27017
Wed Mar 20 18:05:29 [rsHealthPoll] DBClientCursor::init call() failed
Wed Mar 20 18:05:29 [rsHealthPoll] replSet info o16.servername.com:27017 is down (or slow to respond): DBClientBase::findN: transport error: o16.servername.com:27017 ns: admin.$cmd query: { replSetHeartbeat: "bqset", v: 252399, pv: 1, checkEmpty: false, from: "o15.servername.com:27017", $auth: {} }
Wed Mar 20 18:05:29 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state DOWN
Wed Mar 20 18:05:29 [rsMgr] not electing self, 14.servername.com:27017 would veto
Wed Mar 20 18:05:35 [rsMgr] replSet info electSelf 1
Wed Mar 20 18:05:35 [rsMgr] replSet PRIMARY
Wed Mar 20 18:06:22 [conn7463] recv(): message len 0 is too large0
Wed Mar 20 18:06:22 [rsHealthPoll] replSet member o16.servername.com:27017 is up
Wed Mar 20 18:06:22 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state SECONDARY
Wed Mar 20 18:06:24 [conn7479] recv(): message len 0 is too large0
Wed Mar 20 18:06:27 [conn7497] recv(): message len 0 is too large0
Wed Mar 20 18:06:27 [rsMgr] stepping down o15.servername.com:27017 (priority 1), o16.servername.com:27017 is priority 2 and 0 seconds behind
Wed Mar 20 18:06:27 [rsMgr] replSet relinquishing primary state
Wed Mar 20 18:06:27 [rsMgr] replSet SECONDARY
Wed Mar 20 18:06:27 [rsMgr] replSet closing client sockets after relinquishing primary
Wed Mar 20 18:06:28 [conn7488] replSet info voting yea for o16.servername.com:27017 (0)
Wed Mar 20 18:06:28 [conn7506] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "renee_fleming" }
Wed Mar 20 18:06:28 [conn7506]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:28 [conn7506] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:28 [conn7507] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "mark_twain" }
Wed Mar 20 18:06:28 [conn7507]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:28 [conn7507] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:28 [conn7507] recv(): message len 0 is too large0
Wed Mar 20 18:06:28 [conn7508] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "renee_fleming" }
Wed Mar 20 18:06:28 [conn7508]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:28 [conn7508] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:28 [slaveTracking] build index local.slaves { _id: 1 }
Wed Mar 20 18:06:28 [slaveTracking] build index done.  scanned 0 total records. 0.027 secs
Wed Mar 20 18:06:28 [slaveTracking] update local.slaves query: { _id: ObjectId('50b911890e709bc7a198f0df'), host: "128.121.86.42", ns: "local.oplog.rs" } update: { $set: { syncedTo: Timestamp 1363740726000|1 } } nscanned:0 nupdated:1 fastmodinsert:1 keyUpdates:0 locks(micros) w:149577 149ms
Wed Mar 20 18:06:28 [conn7509] recv(): message len 0 is too large0
Wed Mar 20 18:06:28 [rsHealthPoll] DBClientCursor::init call() failed
Wed Mar 20 18:06:28 [rsHealthPoll] replSet info o16.servername.com:27017 is down (or slow to respond): DBClientBase::findN: transport error: o16.servername.com:27017 ns: admin.$cmd query: { replSetHeartbeat: "bqset", v: 252399, pv: 1, checkEmpty: false, from: "o15.servername.com:27017", $auth: {} }
Wed Mar 20 18:06:28 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state DOWN
Wed Mar 20 18:06:28 [MultiCommandJob] SSL Error ret: -1 err: 1 error:00000000:lib(0):func(0):reason(0)
Wed Mar 20 18:06:28 [rsMgr] replSet not trying to elect self as responded yea to someone else recently
Wed Mar 20 18:06:29 [conn7511] recv(): message len 0 is too large0
Wed Mar 20 18:06:29 [conn7513] recv(): message len 4 is too large4
Wed Mar 20 18:06:29 [conn7515] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "visual" }
Wed Mar 20 18:06:29 [conn7515]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:29 [conn7515] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:29 [conn7516] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "visual" }
Wed Mar 20 18:06:29 [conn7516]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:29 [conn7516] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:29 [conn7517] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "visual" }
Wed Mar 20 18:06:29 [conn7517]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:29 [conn7517] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:29 [conn7515] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "visual" }
Wed Mar 20 18:06:29 [conn7515]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:29 [conn7515] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:29 [conn7519] recv(): message len 0 is too large0
Wed Mar 20 18:06:29 [conn7520] recv(): message len 0 is too large0
Wed Mar 20 18:06:29 [conn7521] recv(): message len 4 is too large4
Wed Mar 20 18:06:30 [conn7522] recv(): message len 0 is too large0
Wed Mar 20 18:06:30 [conn7523] recv(): message len 4 is too large4
Wed Mar 20 18:06:30 [conn7524] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "annoyance" }
Wed Mar 20 18:06:30 [conn7524]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:30 [conn7524] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:30 [conn7524] recv(): message len 0 is too large0
Wed Mar 20 18:06:30 [conn7525] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "frank_iero" }
Wed Mar 20 18:06:30 [conn7525]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:30 [conn7525] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:30 [conn7528] recv(): message len 4 is too large4
Wed Mar 20 18:06:30 [conn7526] recv(): message len 0 is too large0
Wed Mar 20 18:06:31 [conn7529] assertion 13435 not master and slaveOk=false ns:bq2.keyword query:{ path: "aim" }
Wed Mar 20 18:06:31 [conn7529]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:31 [conn7529] problem detected during query over bq2.keyword : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:31 [conn7530] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "lester_b_pearson" }
Wed Mar 20 18:06:31 [conn7530]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:31 [conn7530] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:31 [rsHealthPoll] replSet member o16.servername.com:27017 is up
Wed Mar 20 18:06:31 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state PRIMARY
Wed Mar 20 18:06:31 [conn7531] assertion 13435 not master and slaveOk=false ns:bq2.author query:{ path: "lester_b_pearson" }
Wed Mar 20 18:06:31 [conn7531]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:31 [conn7531] problem detected during query over bq2.author : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:31 [conn7531] recv(): message len 0 is too large0
Wed Mar 20 18:06:31 [conn7532]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:31 [conn7532] problem detected during query over bq2.quote : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:31 [conn7533]  ntoskip:0 ntoreturn:-1
Wed Mar 20 18:06:31 [conn7533] problem detected during query over bq2.quote : { $err: "not master and slaveOk=false", code: 13435 }
Wed Mar 20 18:06:31 [conn7534] recv(): message len 4 is too large4
Wed Mar 20 18:06:31 [conn7538] recv(): message len 0 is too large0
Wed Mar 20 18:06:31 [conn7539] recv(): message len 0 is too large0
Wed Mar 20 18:06:32 [conn7540] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7493] SSL Error ret: -1 err: 5 error:00000000:lib(0):func(0):reason(0)
Wed Mar 20 18:06:32 [conn7493] SocketException handling request, closing client connection: 9001 socket exception [2] server [128.121.86.42:58414] 
Wed Mar 20 18:06:32 [conn7543] recv(): message len 0 is too large0
Wed Mar 20 18:06:32 [conn7544] recv(): message len 0 is too large0
Wed Mar 20 18:06:32 [conn7545] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7546] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7547] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7499] SSL Error ret: -1 err: 5 error:00000000:lib(0):func(0):reason(0)
Wed Mar 20 18:06:32 [conn7499] SocketException handling request, closing client connection: 9001 socket exception [2] server [128.121.86.42:58421] 
Wed Mar 20 18:06:32 [conn7548] recv(): message len 0 is too large0
Wed Mar 20 18:06:32 [conn7549] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7550] recv(): message len 4 is too large4
Wed Mar 20 18:06:32 [conn7551] recv(): message len 4 is too large4
Wed Mar 20 18:06:33 [rsBackgroundSync] Socket recv() errno:9 Bad file descriptor 128.121.86.42:27017
Wed Mar 20 18:06:33 [rsBackgroundSync] SocketException: remote: 128.121.86.42:27017 error: 9001 socket exception [1] server [128.121.86.42:27017] 
Wed Mar 20 18:06:33 [rsBackgroundSync] SSL Error ret: -1 err: 5 error:00000000:lib(0):func(0):reason(0)
Wed Mar 20 18:06:33 [rsBackgroundSync] Socket flush send() errno:9 Bad file descriptor 128.121.86.42:27017
Wed Mar 20 18:06:33 [rsBackgroundSync]   caught exception (socket exception [SEND_ERROR] for 128.121.86.42:27017) in destructor (~PiggyBackData)
Wed Mar 20 18:06:33 [rsBackgroundSync] replSet db exception in producer: 10278 dbclient error communicating with server: o16.servername.com:27017
Wed Mar 20 18:06:37 [conn7555] recv(): message len 0 is too large0
Wed Mar 20 18:06:41 [rsHealthPoll] DBClientCursor::init call() failed
Wed Mar 20 18:06:41 [rsHealthPoll] replSet info o16.servername.com:27017 is down (or slow to respond): DBClientBase::findN: transport error: o16.servername.com:27017 ns: admin.$cmd query: { replSetHeartbeat: "bqset", v: 252399, pv: 1, checkEmpty: false, from: "o15.servername.com:27017", $auth: {} }
Wed Mar 20 18:06:41 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state DOWN
Wed Mar 20 18:06:41 [rsMgr] not electing self, 14.servername.com:27017 would veto
Wed Mar 20 18:06:43 [conn7557] recv(): message len 0 is too large0
Wed Mar 20 18:06:44 [rsHealthPoll] replSet member o16.servername.com:27017 is up
Wed Mar 20 18:06:44 [rsHealthPoll] replSet member o16.servername.com:27017 is now in state PRIMARY
Wed Mar 20 18:06:44 [rsBackgroundSync] replSet syncing to: o16.servername.com:27017
Wed Mar 20 18:06:48 [conn7558] recv(): message len 0 is too large0
Wed Mar 20 19:50:28 [rsBackgroundSync] Socket recv() errno:0 Success 128.121.86.42:27017
Wed Mar 20 19:50:28 [rsBackgroundSync] SocketException: remote: 128.121.86.42:27017 error: 9001 socket exception [1] server [128.121.86.42:27017] 
Wed Mar 20 19:50:28 [rsBackgroundSync] replSet db exception in producer: 10278 dbclient error communicating with server: o16.servername.com:27017
Wed Mar 20 19:50:38 [rsBackgroundSync] replSet syncing to: o16.servername.com:27017