Possibly related to SERVER-2485

Wed Feb 16 10:27:40 [conn113104] remove sd.metrics query:

{ _id: ObjectId('4d5707c2160ba00d6a0011da'), accId: 3567, sId: 5 }

138ms
Wed Feb 16 10:27:40 [conn113508] query admin.$cmd ntoreturn:1 command:

{ setShardVersion: "sd.metricsLatest", configdb: "config1:27019", version: Timestamp 0|0, serverID: ObjectId('4d596016fe9bde831c61e013'), shard: "shard2", shardHost: "set2/rs2b:27018" }

reslen:53 116ms
Wed Feb 16 10:27:40 [conn113255] query admin.$cmd ntoreturn:1 command:

{ setShardVersion: "sd.metricsLatest", configdb: "config1:27019", version: Timestamp 0|0, serverID: ObjectId('4d596016fe9bde831c61e013'), shard: "shard2", shardHost: "set2/rs2b:27018" }

reslen:53 120ms
Wed Feb 16 10:27:40 [conn113357] query admin.$cmd ntoreturn:1 command:

{ setShardVersion: "sd.metricsLatest", configdb: "config1:27019", version: Timestamp 0|0, serverID: ObjectId('4d596016fe9bde831c61e013'), shard: "shard2", shardHost: "set2/rs2b:27018" }

reslen:53 115ms
Wed Feb 16 10:27:41 [conn113859] query admin.$cmd ntoreturn:1 command:

{ writebacklisten: ObjectId('4d5ba56021d7d00ef5bbe3b7') }

reslen:60 300053ms
Wed Feb 16 10:27:46 [ReplSetHealthPollTask] DBClientCursor::init call() failed
Wed Feb 16 10:27:46 [ReplSetHealthPollTask] replSet info rs2d:27018 is down (or slow to respond): DBClientBase::findOne: transport error: rs2d:27018 query:

{ replSetHeartbeat: "set2", v: 7, pv: 1, checkEmpty: false, from: "rs2a:27018" }

Wed Feb 16 10:27:53 [conn113104] query sd.metrics ntoreturn:1000 reslen:146469 nscanned:3571817 { $query: { $nor: [ { accId: 2461, tA:

{ $gt: new Date(1295260061132) }

}, { accId: 2805, tA:

{ $gt: new Date(1297247261132) }

}, { accId: 2424, tA:

{ $gt: new Date(1295260061135) }

}, { tA:

{ $gt: new Date(1297592861135) }

} ] }, $orderby:

{ _id: 1 }

} nreturned:1000 12267ms
Wed Feb 16 10:28:05 [conn113377] query admin.$cmd ntoreturn:1 command:

{ setShardVersion: "sd.servers", configdb: "config1:27019", version: Timestamp 0|0, serverID: ObjectId('4d596016fe9bde831c61e013'), shard: "shard2", shardHost: "set2/rs2b:27018" }

reslen:53 227ms
Wed Feb 16 10:28:06 [conn113881] Assertion: 13548:BufBuilder grow() > 64MB
Wed Feb 16 10:28:06 [conn113220] query admin.$cmd ntoreturn:1 command:

{ setShardVersion: "sd.metricsLatest", configdb: "config1:27019", version: Timestamp 0|0, serverID: ObjectId('4d596016fe9bde831c61e013'), shard: "shard2", shardHost: "set2/rs2b:27018" }

reslen:53 117ms
0x560819 0x4e87a7 0x58aa7c 0x6d08e5 0x6cbdb4 0x7d518a 0x7d64e1 0x63fdc5 0x645e85 0x74dc0c 0x75399d 0x89a15e 0x8ad550 0x3f7b60673d 0x3f7aad3d1d
/home/david/mongodb/latest/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x129) [0x560819]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo10BufBuilder15grow_reallocateEv+0x47) [0x4e87a7]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo14BSONObjBuilder6appendERKNS_10StringDataEPKci+0xbc) [0x58aa7c]
/home/david/mongodb/latest/bin/mongod(_ZNK5mongo11ReplSetImpl16_summarizeStatusERNS_14BSONObjBuilderE+0xad5) [0x6d08e5]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo19CmdReplSetGetStatus3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x114) [0x6cbdb4]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x58a) [0x7d518a]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x851) [0x7d64e1]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x35) [0x63fdc5]
/home/david/mongodb/latest/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x3005) [0x645e85]
/home/david/mongodb/latest/bin/mongod [0x74dc0c]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x5ad) [0x75399d]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x21e) [0x89a15e]
/home/david/mongodb/latest/bin/mongod(thread_proxy+0x80) [0x8ad550]
/lib64/libpthread.so.0 [0x3f7b60673d]
/lib64/libc.so.6(clone+0x6d) [0x3f7aad3d1d]
Wed Feb 16 10:28:06 Got signal: 6 (Aborted).

Wed Feb 16 10:28:06 Backtrace:
0x899789 0x3f7aa302d0 0x3f7aa30265 0x3f7aa31d10 0x3f7aa6a84b 0x3f7aa7230f 0x3f7aa7276b 0x3f81a9db6a 0x6d13f3 0x6cbdb4 0x7d518a 0x7d64e1 0x63fdc5 0x645e85 0x74dc0c 0x75399d 0x89a15e 0x8ad550 0x3f7b60673d 0x3f7aad3d1d
/home/david/mongodb/latest/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x899789]
/lib64/libc.so.6 [0x3f7aa302d0]
/lib64/libc.so.6(gsignal+0x35) [0x3f7aa30265]
/lib64/libc.so.6(abort+0x110) [0x3f7aa31d10]
/lib64/libc.so.6 [0x3f7aa6a84b]
/lib64/libc.so.6 [0x3f7aa7230f]
/lib64/libc.so.6(cfree+0x4b) [0x3f7aa7276b]
/usr/lib64/libstdc++.so.6(_ZNSsD1Ev+0x3a) [0x3f81a9db6a]
/home/david/mongodb/latest/bin/mongod(_ZNK5mongo11ReplSetImpl16_summarizeStatusERNS_14BSONObjBuilderE+0x15e3) [0x6d13f3]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo19CmdReplSetGetStatus3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x114) [0x6cbdb4]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x58a) [0x7d518a]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x851) [0x7d64e1]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x35) [0x63fdc5]
/home/david/mongodb/latest/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x3005) [0x645e85]
/home/david/mongodb/latest/bin/mongod [0x74dc0c]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x5ad) [0x75399d]
/home/david/mongodb/latest/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x21e) [0x89a15e]
/home/david/mongodb/latest/bin/mongod(thread_proxy+0x80) [0x8ad550]
/lib64/libpthread.so.0 [0x3f7b60673d]
/lib64/libc.so.6(clone+0x6d) [0x3f7aad3d1d]

Wed Feb 16 10:28:06 dbexit:
Wed Feb 16 10:28:06 [conn113881] shutdown: going to close listening sockets...
Wed Feb 16 10:28:06 [conn113881] closing listening socket: 5
Wed Feb 16 10:28:06 [conn113881] closing listening socket: 6
Wed Feb 16 10:28:06 [conn113881] closing listening socket: 8
Wed Feb 16 10:28:06 [conn113881] closing listening socket: 9
Wed Feb 16 10:28:06 [conn113881] removing socket file: /tmp/mongodb-27018.sock
Wed Feb 16 10:28:06 [conn113246] got request after shutdown()
Wed Feb 16 10:28:06 [conn113671] got request after shutdown()
Wed Feb 16 10:28:06 [conn113168] got request after shutdown()
Wed Feb 16 10:28:06 [conn113394] got request after shutdown()
Wed Feb 16 10:28:06 [conn112133] got request after shutdown()
Wed Feb 16 10:28:06 [conn113350] got request after shutdown()
Wed Feb 16 10:28:06 [conn113508] got request after shutdown()
Wed Feb 16 10:28:06 [conn113723] got request after shutdown()
Wed Feb 16 10:28:06 [conn113881] removing socket file: /tmp/mongodb-28018.sock
Wed Feb 16 10:28:06 [conn113881] shutdown: going to flush diaglog...
Wed Feb 16 10:28:06 [conn113447] got request after shutdown()
Wed Feb 16 10:28:06 [conn113881] shutdown: going to close sockets...
Wed Feb 16 10:28:06 [conn113220] got request after shutdown()
Wed Feb 16 10:28:06 [conn113126] got request after shutdown()
Wed Feb 16 10:28:06 [conn96111] got request after shutdown()
Wed Feb 16 10:28:06 ERROR: Client::shutdown not called: slaveTracking
Wed Feb 16 10:28:06 [conn113212] got request after shutdown()
Wed Feb 16 10:28:06 [conn112761] got request after shutdown()
Wed Feb 16 10:28:06 [conn112695] got request after shutdown()
Wed Feb 16 10:28:06 [conn113140] got request after shutdown()
Wed Feb 16 10:28:06 [conn113881] shutdown: waiting for fs preallocator...
Wed Feb 16 10:28:06 [conn113881] shutdown: closing all files...
Wed Feb 16 10:28:06 [conn112337] got request after shutdown()
Wed Feb 16 10:28:06 [initandlisten] now exiting
Wed Feb 16 10:28:06 dbexit: ; exiting immediately
Wed Feb 16 10:28:06 [conn111690] got request after shutdown()
Wed Feb 16 10:28:06 [conn113265] got request after shutdown()
Wed Feb 16 10:28:06 [conn113157] got request after shutdown()
Wed Feb 16 10:28:06 [conn112451] got request after shutdown()
Wed Feb 16 10:28:06 [conn112255] got request after shutdown()
Wed Feb 16 10:28:06 [conn112088] got request after shutdown()
Wed Feb 16 10:28:07 [conn113785] got request after shutdown()
Wed Feb 16 10:28:07 [conn113202] got request after shutdown()
Wed Feb 16 10:28:07 [conn113518] got request after shutdown()
Wed Feb 16 10:28:07 [conn113267] got request after shutdown()
Wed Feb 16 10:28:07 [conn113413] got request after shutdown()
Wed Feb 16 10:28:07 [conn112565] got request after shutdown()
Wed Feb 16 10:28:07 [conn113884] assertion 11600 interrupted at shutdown ns:local.oplog.rs query:{ ts:

{ $gte: new Date(5574201246161567754) }

}
Wed Feb 16 10:28:07 [conn113885] assertion 11600 interrupted at shutdown ns:local.oplog.rs query:{ ts:

{ $gte: new Date(5574201246161567754) }

}
Wed Feb 16 10:28:07 [conn112456] got request after shutdown()
Wed Feb 16 10:28:07 [conn113319] got request after shutdown()
Wed Feb 16 10:28:07 [conn112713] got request after shutdown()
Wed Feb 16 10:28:07 [conn113555] got request after shutdown()
Wed Feb 16 10:28:07 [conn113364] got request after shutdown()
Wed Feb 16 10:28:07 [conn113248] got request after shutdown()
Wed Feb 16 10:28:07 [conn113884] query local.oplog.rs exception 10230ms
Wed Feb 16 10:28:07 [conn113885] query local.oplog.rs exception 10107ms
Wed Feb 16 10:28:07 [conn111054] got request after shutdown()
Wed Feb 16 10:28:07 [conn113598] got request after shutdown()
Wed Feb 16 10:28:07 [conn112527] got request after shutdown()
Wed Feb 16 10:28:07 [conn112670] got request after shutdown()
Wed Feb 16 10:28:07 Got signal: 11 (Segmentation fault).

Wed Feb 16 10:28:07 Backtrace:
0x899789 0x3f7aa302d0 0x5fa0a70
/home/david/mongodb/latest/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x899789]
/lib64/libc.so.6 [0x3f7aa302d0]
[0x5fa0a70]