I have a mongo instance used primarily for backup, running LVM on loop filesystems (losetup /dev/loop0 /lvmdata/data0). The LVM kernel module crashes occasionally, and once I bring the mongo instance up, it joins as a secondary (as expected), only it doesn't start catching up.

After an hour or so, it begins to catch up. I noticed this in the logs after it was in sync. I am not sure of the timing of events, as I looked at the logs after it caught up.

Fri Nov 9 20:47:34 [rsSync] warning: DR102 too much data written uncommitted 314.577MB
Fri Nov 9 20:47:36 [rsSync] local.oplog.rs warning assertion failure _intents.size() < 2000000 src/mongo/db/dur_commitjob.h 101
0xade6e1 0x802c5a 0x78c4a0 0x78c4ff 0x78c7d2 0x78c8ed 0x78c95b 0xa07c1a 0x739675 0xb6708a 0x64b5eb 0x65345e 0x6538f8 0x65394a 0x653d58 0x7c3659 0x7f22b7718e9a 0
x7f22b6a2ddbd
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xade6e1]
/usr/bin/mongod(_ZN5mongo9wassertedEPKcS1_j+0x11a) [0x802c5a]
/usr/bin/mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x280) [0x78c4a0]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents8_unspoolEv+0x4f) [0x78c4ff]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents7unspoolEv+0x52) [0x78c7d2]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents4pushERKNS0_11WriteIntentE+0x6d) [0x78c8ed]
/usr/bin/mongod(_ZN5mongo3dur11DurableImpl18declareWriteIntentEPvj+0x6b) [0x78c95b]
/usr/bin/mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0xa07c1a]
/usr/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x195) [0x739675]
/usr/bin/mongod(_ZN5mongo11_logOpObjRSERKNS_7BSONObjE+0x27a) [0xb6708a]
/usr/bin/mongod(_ZN5mongo7replset8SyncTail15applyOpsToOplogEPSt5dequeINS_7BSONObjESaIS3_EE+0x4b) [0x64b5eb]
/usr/bin/mongod(_ZN5mongo7replset8SyncTail16oplogApplicationEv+0x48e) [0x65345e]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xb8) [0x6538f8]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x2a) [0x65394a]
/usr/bin/mongod(_ZN5mongo15startSyncThreadEv+0xa8) [0x653d58]
/usr/bin/mongod() [0x7c3659]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7f22b7718e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f22b6a2ddbd]
Fri Nov 9 20:47:41 [rsSync] local.oplog.rs warning assertion failure _intents.size() < 2000000 src/mongo/db/dur_commitjob.h 101
0xade6e1 0x802c5a 0x78c4a0 0x78c4ff 0x78c7d2 0x78c8ed 0x78c95b 0xa07c1a 0xb670ba 0x64b5eb 0x65345e 0x6538f8 0x65394a 0x653d58 0x7c3659 0x7f22b7718e9a 0x7f22b6a2
ddbd
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xade6e1]
/usr/bin/mongod(_ZN5mongo9wassertedEPKcS1_j+0x11a) [0x802c5a]
/usr/bin/mongod(_ZN5mongo3dur9CommitJob4noteEPvi+0x280) [0x78c4a0]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents8_unspoolEv+0x4f) [0x78c4ff]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents7unspoolEv+0x52) [0x78c7d2]
/usr/bin/mongod(_ZN5mongo3dur18ThreadLocalIntents4pushERKNS0_11WriteIntentE+0x6d) [0x78c8ed]
/usr/bin/mongod(_ZN5mongo3dur11DurableImpl18declareWriteIntentEPvj+0x6b) [0x78c95b]
/usr/bin/mongod(_ZN5mongo3dur11DurableImpl10writingPtrEPvj+0xa) [0xa07c1a]
/usr/bin/mongod(_ZN5mongo11_logOpObjRSERKNS_7BSONObjE+0x2aa) [0xb670ba]
/usr/bin/mongod(_ZN5mongo7replset8SyncTail15applyOpsToOplogEPSt5dequeINS_7BSONObjESaIS3_EE+0x4b) [0x64b5eb]
/usr/bin/mongod(_ZN5mongo7replset8SyncTail16oplogApplicationEv+0x48e) [0x65345e]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xb8) [0x6538f8]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x2a) [0x65394a]
/usr/bin/mongod(_ZN5mongo15startSyncThreadEv+0xa8) [0x653d58]
/usr/bin/mongod() [0x7c3659]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7f22b7718e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f22b6a2ddbd]