Run 10 mongod shards, 1 config server,1 mongos with journal option enabled. Insertion and deletion on mongodb by mongodb c driver.System restart mongods system.
One mongod shard was not restarted.
logs are below
Mon Feb 6 22:46:13 [conn2] insert 00060212.Database 64ms
Mon Feb 6 22:46:13 [conn2] runQuery called admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:13 [conn2] run command admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:13 [conn2] command admin.$cmd command:

{ getlasterror: 1 }

ntoreturn:1 reslen:67 0ms
Mon Feb 6 22:46:13 [journal] lsn set 358132
Mon Feb 6 22:46:15 [conn2] insert 00060212.Database 321ms
Mon Feb 6 22:46:15 [conn2] runQuery called admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:15 [conn2] run command admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:15 [conn2] command admin.$cmd command:

{ getlasterror: 1 }

ntoreturn:1 reslen:67 0ms
Mon Feb 6 22:46:15 [conn2] runQuery called admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:15 [conn2] run command admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:15 [conn2] command admin.$cmd command:

{ getlasterror: 1 }

ntoreturn:1 reslen:67 0ms
Mon Feb 6 22:46:16 [conn2] insert 00060212.Database 69ms
Mon Feb 6 22:46:16 [conn2] runQuery called admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:16 [conn2] run command admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:16 [conn2] command admin.$cmd command:

{ getlasterror: 1 }

ntoreturn:1 reslen:67 0ms
Mon Feb 6 22:46:17 [conn2] insert 00060212.Database 70ms
Mon Feb 6 22:46:17 [conn2] runQuery called admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:17 [conn2] run command admin.$cmd

{ getlasterror: 1 }

Mon Feb 6 22:46:17 [conn2] command admin.$cmd command:

{ getlasterror: 1 }

ntoreturn:1 reslen:67 0ms

• • • • • SERVER RESTARTED *****

Mon Feb 6 22:54:59 isInRangeTest passed
Mon Feb 6 22:54:59 shardKeyTest passed
Mon Feb 6 22:54:59 shardObjTest passed
Mon Feb 6 22:54:59 versionCmpTest passed
Mon Feb 6 22:54:59 versionArrayTest passed
Mon Feb 6 22:54:59 [initandlisten] MongoDB starting : pid=1718 port=15000 dbpath=/sata1/shard2 64-bit host=debian
Mon Feb 6 22:54:59 [initandlisten] db version v2.0.2, pdfile version 4.5
Mon Feb 6 22:54:59 [initandlisten] git version: 514b122d308928517f5841888ceaa4246a7f18e3
Mon Feb 6 22:54:59 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Mon Feb 6 22:54:59 [initandlisten] options:

{ bind_ip: "-all", dbpath: "/sata1/shard2", journalCommitInterval: 2, logappend: true, logpath: "/usr/local/ct/depend/mongo/logs/mongod_15000.log", port: 15000, quiet: true, shardsvr: true, vvv: true }

Mon Feb 6 22:54:59 BackgroundJob starting: DataFileSync
Mon Feb 6 22:55:02 [initandlisten] flushing directory /sata1/shard2
Mon Feb 6 22:55:14 [initandlisten] journal dir=/sata1/shard2/journal
Mon Feb 6 22:55:14 [initandlisten] recover begin
Mon Feb 6 22:55:14 [initandlisten] recover lsn: 358132
Mon Feb 6 22:55:14 [initandlisten] recover /sata1/shard2/journal/j._0
Mon Feb 6 22:55:15 [initandlisten] recover skipping application of section seq:238770 < lsn:358132
Mon Feb 6 22:55:15 [initandlisten] recover skipping application of section seq:298452 < lsn:358132
Mon Feb 6 22:55:18 [initandlisten] couldn't uncompress journal section
Mon Feb 6 22:55:24 [initandlisten] Assertion: 15874:couldn't uncompress journal section
0x584722 0x76d5f5 0x76dac9 0x76e621 0x76e8f5 0x76f6ed 0x76f80b 0x75df7f 0xa91509 0xa91e7d 0xa9bbaa 0x7ff1c95b9c4d 0x500f09
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584722]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur11RecoveryJob14processSectionEPKNS0_11JSectHeaderEPKvjPKNS0_11JSectFooterE+0x595) [0x76d5f5]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur11RecoveryJob17processFileBufferEPKvj+0xf9) [0x76dac9]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur11RecoveryJob11processFileEN5boost10filesystem10basic_pathISsNS3_11path_traitsEEE+0xd1) [0x76e621]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur11RecoveryJob2goERSt6vectorIN5boost10filesystem10basic_pathISsNS4_11path_traitsEEESaIS7_EE+0xc5) [0x76e8f5]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur8_recoverEv+0x18d) [0x76f6ed]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur7recoverEv+0x3b) [0x76f80b]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo3dur7startupEv+0x4f) [0x75df7f]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo14_initAndListenEi+0x3c9) [0xa91509]
/usr/local/ct/depend/mongo/bin/mongod(_ZN5mongo13initAndListenEi+0x1d) [0xa91e7d]
/usr/local/ct/depend/mongo/bin/mongod(main+0x9aaa) [0xa9bbaa]
/lib/libc.so.6(__libc_start_main+0xfd) [0x7ff1c95b9c4d]
/usr/local/ct/depend/mongo/bin/mongod(__gxx_personality_v0+0x3e9) [0x500f09]
Mon Feb 6 22:55:25 [initandlisten] exception during recovery
Mon Feb 6 22:55:25 [initandlisten] exception in initAndListen: 15874 couldn't uncompress journal section, terminating
Mon Feb 6 22:55:25 dbexit:
Mon Feb 6 22:55:25 [initandlisten] shutdown: going to close listening sockets...
Mon Feb 6 22:55:25 [initandlisten] shutdown: going to flush diaglog...
Mon Feb 6 22:55:25 [initandlisten] shutdown: going to close sockets...
Mon Feb 6 22:55:25 [initandlisten] shutdown: waiting for fs preallocator...
Mon Feb 6 22:55:25 [initandlisten] shutdown: lock for final commit...
Mon Feb 6 22:55:25 [initandlisten] shutdown: final commit...
Mon Feb 6 22:55:27 [initandlisten] shutdown: closing all files...
Mon Feb 6 22:55:27 [initandlisten] closeAllFiles() finished
Mon Feb 6 22:55:27 [initandlisten] shutdown: removing fs lock...
Mon Feb 6 22:55:27 dbexit: really exiting now
~