Mon Jul 18 15:21:47 [initandlisten] MongoDB starting : pid=23144 port=27017 dbpath=/data/db 64-bit Mon Jul 18 15:21:47 [initandlisten] db version v1.8.2, pdfile version 4.5 Mon Jul 18 15:21:47 [initandlisten] git version: 433bbaa14aaba6860da15bd4de8edf600f56501b Mon Jul 18 15:21:47 [initandlisten] build sys info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41 Mon Jul 18 15:21:47 [initandlisten] journal dir=/data/db/journal Mon Jul 18 15:21:47 [initandlisten] recover begin Mon Jul 18 15:21:47 [initandlisten] recover lsn: 0 Mon Jul 18 15:21:47 [initandlisten] recover /data/db/journal/j._0 Mon Jul 18 15:21:47 [initandlisten] recover create file /data/db/graph.39 2047MB Mon Jul 18 15:22:17 [initandlisten] recover cleaning up Mon Jul 18 15:22:17 [initandlisten] removeJournalFiles Mon Jul 18 15:22:17 [initandlisten] recover done Mon Jul 18 15:22:18 [initandlisten] preallocateIsFaster=true 5.04 Mon Jul 18 15:22:19 [initandlisten] preallocateIsFaster=true 4.18 Mon Jul 18 15:22:20 [initandlisten] preallocateIsFaster=true 8.88 Mon Jul 18 15:22:20 [initandlisten] preallocating a journal file /data/db/journal/prealloc.0 Mon Jul 18 15:22:46 [initandlisten] preallocating a journal file /data/db/journal/prealloc.1 Mon Jul 18 15:23:11 [initandlisten] preallocating a journal file /data/db/journal/prealloc.2 Mon Jul 18 15:23:35 [dur] lsn set 0 Mon Jul 18 15:23:35 [initandlisten] waiting for connections on port 27017 Mon Jul 18 15:23:35 [websvr] web admin interface listening on port 28017 Mon Jul 18 15:23:35 [initandlisten] connection accepted from 127.0.0.1:39396 #1 Mon Jul 18 15:23:35 [initandlisten] connection accepted from 10.86.70.192:57576 #2 Mon Jul 18 15:23:35 [startReplSets] replSet STARTUP2 Mon Jul 18 15:23:35 [rs Manager] replSet can't see a majority, will not try to elect self Mon Jul 18 15:23:35 [ReplSetHealthPollTask] replSet info 10.78.233.4:27017 is up Mon Jul 18 15:23:35 [ReplSetHealthPollTask] replSet member 10.78.233.4:27017 PRIMARY Mon Jul 18 15:23:35 [initandlisten] connection accepted from 10.78.233.4:45438 #3 Mon Jul 18 15:23:35 [initandlisten] connection accepted from 127.0.0.1:39401 #4 Mon Jul 18 15:23:36 [initandlisten] connection accepted from 10.210.247.170:59829 #5 Mon Jul 18 15:23:37 [ReplSetHealthPollTask] replSet info 10.210.247.170:27019 is up Mon Jul 18 15:23:37 [ReplSetHealthPollTask] replSet member 10.210.247.170:27019 ARBITER Mon Jul 18 15:23:37 [replica set sync] replSet still syncing, not yet to minValid optime4e244ef3:1f Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.87.67.111:36782 #6 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.126.177.54:56044 #7 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.223.43.79:60654 #8 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.126.229.202:54770 #9 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.126.210.164:44511 #10 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.222.237.89:49611 #11 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.87.161.25:45667 #12 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.126.217.114:46513 #13 Mon Jul 18 15:23:43 [initandlisten] connection accepted from 10.252.177.241:40429 #14 Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.581MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.589MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.597MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.606MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.614MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.622MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.63MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.638MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.647MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.655MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.663MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.671MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.679MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.687MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.696MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.704MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.712MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.72MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.728MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.737MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.745MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.753MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.761MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.769MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.778MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.786MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.794MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.802MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.81MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.819MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.827MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.835MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.843MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.851MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.86MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.868MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.876MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.884MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.892MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.9MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.909MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.917MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.925MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.933MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.941MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.95MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.958MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.966MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.974MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.982MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.991MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 314.999MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.007MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.015MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.023MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.032MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.04MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.048MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.056MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.064MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.073MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.081MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.089MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.097MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.105MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.113MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.122MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.13MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.138MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.146MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.154MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.163MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.171MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.179MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.187MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.195MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.204MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.212MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.22MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.228MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.236MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.245MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.253MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.261MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.269MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.277MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.286MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.294MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.302MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.31MB Mon Jul 18 15:23:55 [replica set sync] graph.identities Assertion failure !"replSet warning DR102 too much data written uncommitted" db/dur_commitjob.cpp 211 0x54f0ee 0x55f1dd 0x638497 0x62b12a 0x68c2a5 0x7719ae 0x772d80 0x77e465 0x669e0c 0x66cd30 0x715c81 0x70599e 0x70874d 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x227) [0x638497] mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0x62b12a] mongod(_ZNK5mongo11BtreeBucket7unindexENS_7DiskLocERNS_12IndexDetailsERKNS_7BSONObjES1_+0x5b5) [0x68c2a5] mongod() [0x7719ae] mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbb+0x140) [0x772d80] mongod(_ZN5mongo11DataFileMgr12updateRecordEPKcPNS_16NamespaceDetailsEPNS_25NamespaceDetailsTransientEPNS_6RecordERKNS_7DiskLocES2_iRNS_7OpDebugEb+0x465) [0x77e465] mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE+0xccc) [0x669e0c] mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE+0x130) [0x66cd30] mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0x5c1) [0x715c81] mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1ce) [0x70599e] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b1d) [0x70874d] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.318MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.326MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.335MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.343MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.351MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.359MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.367MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.376MB Mon Jul 18 15:23:55 [replica set sync] replSet warning DR102 too much data written uncommitted 315.384MB Mon Jul 18 15:24:41 [replica set sync] Assertion failure a <= 256*1024*1024 util/alignedbuilder.cpp 51 0x54f0ee 0x55f1dd 0x6284f4 0x63648a 0x636ce4 0x636e99 0x63706b 0x62de8f 0x632176 0x708762 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo14AlignedBuilder14growReallocateEj+0x44) [0x6284f4] mongod(_ZN5mongo3dur21prepBasicWrite_inlockERNS_14AlignedBuilderEPKNS0_11WriteIntentERNS_12RelativePathE+0x2aa) [0x63648a] mongod(_ZN5mongo3dur15prepBasicWritesERNS_14AlignedBuilderE+0x74) [0x636ce4] mongod(_ZN5mongo3dur14_PREPLOGBUFFEREv+0x79) [0x636e99] mongod(_ZN5mongo3dur13PREPLOGBUFFEREv+0x9b) [0x63706b] mongod() [0x62de8f] mongod(_ZN5mongo9writelockD1Ev+0xc6) [0x632176] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b32) [0x708762] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:24:42 [replica set sync] Assertion failure a <= 256*1024*1024 util/alignedbuilder.cpp 51 0x54f0ee 0x55f1dd 0x6284f4 0x63648a 0x636ce4 0x636e99 0x63706b 0x62de8f 0x632176 0x708762 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo9wassertedEPKcS1_j+0x10d) [0x55f1dd] mongod(_ZN5mongo14AlignedBuilder14growReallocateEj+0x44) [0x6284f4] mongod(_ZN5mongo3dur21prepBasicWrite_inlockERNS_14AlignedBuilderEPKNS0_11WriteIntentERNS_12RelativePathE+0x2aa) [0x63648a] mongod(_ZN5mongo3dur15prepBasicWritesERNS_14AlignedBuilderE+0x74) [0x636ce4] mongod(_ZN5mongo3dur14_PREPLOGBUFFEREv+0x79) [0x636e99] mongod(_ZN5mongo3dur13PREPLOGBUFFEREv+0x9b) [0x63706b] mongod() [0x62de8f] mongod(_ZN5mongo9writelockD1Ev+0xc6) [0x632176] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b32) [0x708762] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:24:42 [replica set sync] Assertion failure a <= 512*1024*1024 util/alignedbuilder.cpp 52 0x54f0ee 0x56073d 0x628518 0x63648a 0x636ce4 0x636e99 0x63706b 0x62de8f 0x632176 0x708762 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x54f0ee] mongod(_ZN5mongo8assertedEPKcS1_j+0xfd) [0x56073d] mongod(_ZN5mongo14AlignedBuilder14growReallocateEj+0x68) [0x628518] mongod(_ZN5mongo3dur21prepBasicWrite_inlockERNS_14AlignedBuilderEPKNS0_11WriteIntentERNS_12RelativePathE+0x2aa) [0x63648a] mongod(_ZN5mongo3dur15prepBasicWritesERNS_14AlignedBuilderE+0x74) [0x636ce4] mongod(_ZN5mongo3dur14_PREPLOGBUFFEREv+0x79) [0x636e99] mongod(_ZN5mongo3dur13PREPLOGBUFFEREv+0x9b) [0x63706b] mongod() [0x62de8f] mongod(_ZN5mongo9writelockD1Ev+0xc6) [0x632176] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b32) [0x708762] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:24:42 [replica set sync] dbexception in groupCommit causing immediate shutdown: 0 assertion util/alignedbuilder.cpp:52 Mon Jul 18 15:24:42 Got signal: 6 (Aborted). Mon Jul 18 15:24:42 Backtrace: 0x8a8039 0x7fa16164cd80 0x7fa16164cd05 0x7fa161650ab6 0x62e3e4 0x632176 0x708762 0x708b18 0x708b9c 0x70b9b2 0x8bb630 0x7fa162154d8c 0x7fa1616ff04d mongod(_ZN5mongo10abruptQuitEi+0x399) [0x8a8039] /lib/x86_64-linux-gnu/libc.so.6(+0x33d80) [0x7fa16164cd80] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fa16164cd05] /lib/x86_64-linux-gnu/libc.so.6(abort+0x186) [0x7fa161650ab6] mongod() [0x62e3e4] mongod(_ZN5mongo9writelockD1Ev+0xc6) [0x632176] mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0x1b32) [0x708762] mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc8) [0x708b18] mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x3c) [0x708b9c] mongod(_ZN5mongo15startSyncThreadEv+0x252) [0x70b9b2] mongod(thread_proxy+0x80) [0x8bb630] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa162154d8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1616ff04d] Mon Jul 18 15:24:42 ERROR: Client::shutdown not called: replica set sync