-
Type:
Bug
-
Resolution: Cannot Reproduce
-
Priority:
Minor - P4
-
None
-
Affects Version/s: 2.6.3
-
Component/s: Storage
-
Labels:None
-
ALL
Received the following stack trace during a load/update performance test:
2014-08-06T00:54:48.945-0400 [conn452] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server2429", ts: new Date(1325581200000) }, u: { $inc: { cpu_sum: 16.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:106 142ms
2014-08-06T00:54:48.945-0400 [conn458] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server90", ts: new Date(1325581200000) }, u: { $inc: { cpu_sum: 5.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:55 141ms
2014-08-06T00:54:48.945-0400 [conn455] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server6123", ts: new Date(1325577600000) }, u: { $inc: { cpu_sum: 31.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:55 141ms
2014-08-06T00:54:48.945-0400 [conn460] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server6654", ts: new Date(1325581200000) }, u: { $inc: { cpu_sum: 9.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:55 141ms
2014-08-06T00:54:48.945-0400 [conn459] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server8741", ts: new Date(1325581200000) }, u: { $inc: { cpu_sum: 8.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:55 141ms
2014-08-06T00:54:49.613-0400 [conn460] Fatal Assertion 16146
2014-08-06T00:54:49.614-0400 [conn453] Fatal Assertion 16146
2014-08-06T00:54:49.614-0400 [conn452] Fatal Assertion 16146
2014-08-06T00:54:49.614-0400 [conn456] Fatal Assertion 16146
2014-08-06T00:54:49.614-0400 [conn458] Fatal Assertion 16146
2014-08-06T00:54:49.615-0400 [conn71] update test.samples_hour query: { server: "server990", ts: new Date(1356836400000) } update: { $inc: { cpu_sum: 45.0, cpu_count: 1.0 } } nscanned:0 nscannedObjects:0 nMatched:1 nModified:0 fastmodinsert:1 upsert:1 keyUpdates:0 numYields:0 locks(micros) w:250 39002271ms
2014-08-06T00:54:49.615-0400 [conn71] command test.$cmd command: update { update: "samples_hour", updates: [ { q: { server: "server990", ts: new Date(1356836400000) }, u: { $inc: { cpu_sum: 45.0, cpu_count: 1.0 } }, multi: false, upsert: true } ], ordered: true } keyUpdates:0 numYields:0 reslen:106 39002271ms
2014-08-06T00:54:49.615-0400 [conn71] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [127.0.0.1:50743]
2014-08-06T00:54:49.647-0400 [conn460] 0x11037f2ea 0x11032a3b5 0x1103177ec 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39 0x1103b7fe1 0x7fff92eef657 0x7fff92eef5d4
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011032a3b5 _ZN5mongo10logContextEPKc + 453
2 mongod 0x00000001103177ec _ZN5mongo13fassertFailedEi + 332
3 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
4 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
5 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
6 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
7 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
8 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
9 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
10 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
11 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
12 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
13 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
14 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
15 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
16 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065
17 mongod 0x00000001103b7fe1 _ZN5boost12_GLOBAL__N_112thread_proxyEPv + 177
18 libsystem_pthread.dylib 0x00007fff92eef657 _pthread_body + 131
19 libsystem_pthread.dylib 0x00007fff92eef5d4 _pthread_body + 0
2014-08-06T00:54:49.647-0400 [conn456] 0x11037f2ea 0x11032a3b5 0x1103177ec 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39 0x1103b7fe1 0x7fff92eef657 0x7fff92eef5d4
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011032a3b5 _ZN5mongo10logContextEPKc + 453
2 mongod 0x00000001103177ec _ZN5mongo13fassertFailedEi + 332
3 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
4 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
5 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
6 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
7 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
8 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
9 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
10 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
11 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
12 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
13 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
14 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
15 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
16 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065
17 mongod 0x00000001103b7fe1 _ZN5boost12_GLOBAL__N_112thread_proxyEPv + 177
18 libsystem_pthread.dylib 0x00007fff92eef657 _pthread_body + 131
19 libsystem_pthread.dylib 0x00007fff92eef5d4 _pthread_body + 0
2014-08-06T00:54:49.647-0400 [conn453] 0x11037f2ea 0x11032a3b5 0x1103177ec 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39 0x1103b7fe1 0x7fff92eef657 0x7fff92eef5d4
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011032a3b5 _ZN5mongo10logContextEPKc + 453
2 mongod 0x00000001103177ec _ZN5mongo13fassertFailedEi + 332
3 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
4 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
5 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
6 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
7 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
8 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
9 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
10 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
11 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
12 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
13 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
14 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
15 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
16 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065
17 mongod 0x00000001103b7fe1 _ZN5boost12_GLOBAL__N_112thread_proxyEPv + 177
18 libsystem_pthread.dylib 0x00007fff92eef657 _pthread_body + 131
19 libsystem_pthread.dylib 0x00007fff92eef5d4 _pthread_body + 0
2014-08-06T00:54:49.647-0400 [conn458] 0x11037f2ea 0x11032a3b5 0x1103177ec 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39 0x1103b7fe1 0x7fff92eef657 0x7fff92eef5d4
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011032a3b5 _ZN5mongo10logContextEPKc + 453
2 mongod 0x00000001103177ec _ZN5mongo13fassertFailedEi + 332
3 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
4 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
5 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
6 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
7 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
8 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
9 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
10 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
11 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
12 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
13 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
14 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
15 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
16 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065
17 mongod 0x00000001103b7fe1 _ZN5boost12_GLOBAL__N_112thread_proxyEPv + 177
18 libsystem_pthread.dylib 0x00007fff92eef657 _pthread_body + 131
19 libsystem_pthread.dylib 0x00007fff92eef5d4 _pthread_body + 0
2014-08-06T00:54:49.647-0400 [conn452] 0x11037f2ea 0x11032a3b5 0x1103177ec 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39 0x1103b7fe1 0x7fff92eef657 0x7fff92eef5d4
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011032a3b5 _ZN5mongo10logContextEPKc + 453
2 mongod 0x00000001103177ec _ZN5mongo13fassertFailedEi + 332
3 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
4 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
5 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
6 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
7 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
8 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
9 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
10 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
11 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
12 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
13 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
14 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
15 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
16 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065
17 mongod 0x00000001103b7fe1 _ZN5boost12_GLOBAL__N_112thread_proxyEPv + 177
18 libsystem_pthread.dylib 0x00007fff92eef657 _pthread_body + 131
19 libsystem_pthread.dylib 0x00007fff92eef5d4 _pthread_body + 0
2014-08-06T00:54:49.648-0400 [conn460]
***aborting after fassert() failure
2014-08-06T00:54:49.648-0400 [conn456]
*() failure
after fasser*() failure
00:54:49.6400:54:49.649-0400 [conn453]
***aborting after fassert() failure
2014-08-06T00:54:49.649-0400 [conn458]
***aborting after fassert() failure
2014-08-06T00:54:49.649-0400 [conn452]
***aborting after fassert() failure
2014-08-06T00:54:49.671-0400 [conn460] SEVERE: Got signal: 6 (Abort trap: 6).
Backtrace:0x11037f2ea 0x11037ed43 0x7fff8cd84f1a 0x7fff8f05194f 0x7fff8d97a523 0x1103178d7 0x10fedb564 0x10fd91240 0x10fd93320 0x10fd847cb 0x10fd837aa 0x10fd81cc7 0x10fd8a2dc 0x10fd99b59 0x10fd9b32b 0x10fd9c80c 0x110002d6a 0x10fe9b28b 0x10fba6783 0x110338b39
0 mongod 0x000000011037f2ea _ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE + 58
1 mongod 0x000000011037ed43 _ZN5mongo12_GLOBAL__N_110abruptQuitEi + 227
2 libsystem_platform.dylib 0x00007fff8cd84f1a _sigtramp + 26
3 libsystem_malloc.dylib 0x00007fff8f05194f malloc_zone_malloc + 71
4 libsystem_c.dylib 0x00007fff8d97a523 abort + 129
5 mongod 0x00000001103178d7 _ZN5mongo13fassertFailedEi + 567
6 mongod 0x000000010fedb564 _ZN5mongo8LockStat23recordAcquireTimeMicrosEcx + 68
7 mongod 0x000000010fd91240 _ZN5mongo4Lock7DBWrite6lockDBERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 736
8 mongod 0x000000010fd93320 _ZN5mongo4Lock7DBWriteC2ERKNS_10StringDataE + 96
9 mongod 0x000000010fd847cb _ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE + 1019
10 mongod 0x000000010fd837aa _ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPNSt3__16vectorIPNS_19BatchedUpsertDetailENS4_9allocatorIS7_EEEEPNS5_IPNS_16WriteErrorDetailENS8_ISD_EEEE + 154
11 mongod 0x000000010fd81cc7 _ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE + 2055
12 mongod 0x000000010fd8a2dc _ZN5mongo8WriteCmd3runERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS_7BSONObjEiRS7_RNS_14BSONObjBuilderEb + 620
13 mongod 0x000000010fd99b59 _ZN5mongo12_execCommandEPNS_7CommandERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderEb + 41
14 mongod 0x000000010fd9b32b _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4459
15 mongod 0x000000010fd9c80c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1420
16 mongod 0x0000000110002d6a _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1258
17 mongod 0x000000010fe9b28b _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2619
18 mongod 0x000000010fba6783 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 179
19 mongod 0x0000000110338b39 _ZN5mongo17PortMessageServer17handleIncomingMsgEPv + 1065