Major - P3 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] MongoDB starting : pid=31896 port=27017 dbpath=/var/lib/mongodb 64-bit host=vps306768 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] db version v3.2.9 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips 1 Mar 2016 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] allocator: tcmalloc 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] modules: none 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] build environment: 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] distmod: ubuntu1604 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] distarch: x86_64 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] target_arch: x86_64 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: "/var/lib/mongodb", engine: "wiredTiger" } } 2017-01-12T06:21:26.966+0100 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint. 2017-01-12T06:21:26.966+0100 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=3G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (0) [1484198486:986082][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 53248: block header checksum of 2750094563 doesn't match expected checksum of 1661035712 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (0) [1484198486:986144][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (-31804) [1484198486:986158][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic 2017-01-12T06:21:26.986+0100 I - [initandlisten] Fatal Assertion 28558 2017-01-12T06:21:26.986+0100 I - [initandlisten] ***aborting after fassert() failure 2017-01-12T06:21:27.005+0100 F - [initandlisten] Got signal: 6 (Aborted). 0x1521902 0x15208a9 0x1521112 0x7f004873d3d0 0x7f0048398418 0x7f004839a01a 0x14a3713 0x12480f2 0x1c371b7 0x1c375ae 0x1c379b4 0x1b6421f 0x1b6474b 0x1b613e8 0x1b65827 0x1b83008 0x1bb77d0 0x1c35fcb 0x1c36560 0x1c366dc 0x1bc6bb8 0x1c33208 0x1bfc56f 0x1bfc62c 0x1bb3f2a 0x122cbae 0x1225f85 0x1121898 0x9c0070 0x975802 0x7f0048383830 0x9ba699 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"1121902","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"11208A9"},{"b":"400000","o":"1121112"},{"b":"7F004872C000","o":"113D0"},{"b":"7F0048363000","o":"35418","s":"gsignal"},{"b":"7F0048363000","o":"3701A","s":"abort"},{"b":"400000","o":"10A3713","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"E480F2"},{"b":"400000","o":"18371B7","s":"__wt_eventv"},{"b":"400000","o":"18375AE","s":"__wt_err"},{"b":"400000","o":"18379B4","s":"__wt_panic"},{"b":"400000","o":"176421F","s":"__wt_block_extlist_read"},{"b":"400000","o":"176474B","s":"__wt_block_extlist_read_avail"},{"b":"400000","o":"17613E8","s":"__wt_block_checkpoint_load"},{"b":"400000","o":"1765827"},{"b":"400000","o":"1783008","s":"__wt_btree_open"},{"b":"400000","o":"17B77D0","s":"__wt_conn_btree_open"},{"b":"400000","o":"1835FCB","s":"__wt_session_get_btree"},{"b":"400000","o":"1836560","s":"__wt_session_get_btree"},{"b":"400000","o":"18366DC","s":"__wt_session_get_btree_ckpt"},{"b":"400000","o":"17C6BB8","s":"__wt_curfile_open"},{"b":"400000","o":"1833208"},{"b":"400000","o":"17FC56F","s":"__wt_metadata_cursor_open"},{"b":"400000","o":"17FC62C","s":"__wt_metadata_cursor"},{"b":"400000","o":"17B3F2A","s":"wiredtiger_open"},{"b":"400000","o":"E2CBAE","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb"},{"b":"400000","o":"E25F85"},{"b":"400000","o":"D21898","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"400000","o":"5C0070","s":"_ZN5mongo13initAndListenEi"},{"b":"400000","o":"575802","s":"main"},{"b":"7F0048363000","o":"20830","s":"__libc_start_main"},{"b":"400000","o":"5BA699","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.2.9", "gitVersion" : "22ec9e93b40c85fc7cae7d56e7d6a02fd811088c", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-22-generic", "version" : "#40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "1F47133C759EFF79F90A6EBF640DC97B744D70C1" }, { "b" : "7FFD695E0000", "elfType" : 3, "buildId" : "0BFAB3B46E2221A329F60F59D05CBFBED1FECA7C" }, { "b" : "7F00496CF000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "822754695E4B31AE82937258BDFF3D52EFA0BA36" }, { "b" : "7F0049274000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "B75A96C59BE1B5B54FBF1A91ED722BEC9406288E" }, { "b" : "7F004906C000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "A779DBCB3A477DC0C8D09B60FAC7335D396C19DF" }, { "b" : "7F0048E68000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "A2ADF3615338D49C702C41EB83A99AB743D2B574" }, { "b" : "7F0048B5F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "5C4078C04888A418F3DB0868702ECFDB35B3AD8B" }, { "b" : "7F0048949000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F004872C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "B77847CC9CACBCA3B5753D0D25A32E5795AFE75B" }, { "b" : "7F0048363000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "369DE0E1D833CAA693AF17F17C83BA937F0A4DAD" }, { "b" : "7F0049938000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "EDFA6D46E00CA97F349FDD3333D88493D442932C" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1521902] mongod(+0x11208A9) [0x15208a9] mongod(+0x1121112) [0x1521112] libpthread.so.0(+0x113D0) [0x7f004873d3d0] libc.so.6(gsignal+0x38) [0x7f0048398418] libc.so.6(abort+0x16A) [0x7f004839a01a] mongod(_ZN5mongo13fassertFailedEi+0x93) [0x14a3713] mongod(+0xE480F2) [0x12480f2] mongod(__wt_eventv+0x4F7) [0x1c371b7] mongod(__wt_err+0x8E) [0x1c375ae] mongod(__wt_panic+0x24) [0x1c379b4] mongod(__wt_block_extlist_read+0x6F) [0x1b6421f] mongod(__wt_block_extlist_read_avail+0x2B) [0x1b6474b] mongod(__wt_block_checkpoint_load+0x1F8) [0x1b613e8] mongod(+0x1765827) [0x1b65827] mongod(__wt_btree_open+0xBA8) [0x1b83008] mongod(__wt_conn_btree_open+0x140) [0x1bb77d0] mongod(__wt_session_get_btree+0xEB) [0x1c35fcb] mongod(__wt_session_get_btree+0x680) [0x1c36560] mongod(__wt_session_get_btree_ckpt+0x12C) [0x1c366dc] mongod(__wt_curfile_open+0x218) [0x1bc6bb8] mongod(+0x1833208) [0x1c33208] mongod(__wt_metadata_cursor_open+0x5F) [0x1bfc56f] mongod(__wt_metadata_cursor+0x3C) [0x1bfc62c] mongod(wiredtiger_open+0x158A) [0x1bb3f2a] mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb+0xBCE) [0x122cbae] mongod(+0xE25F85) [0x1225f85] mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x3E8) [0x1121898] mongod(_ZN5mongo13initAndListenEi+0x480) [0x9c0070] mongod(main+0x712) [0x975802] libc.so.6(__libc_start_main+0xF0) [0x7f0048383830] mongod(_start+0x29) [0x9ba699] ----- END BACKTRACE ----- mongorepair.sh: line 5: 31896 Aborted mongod --dbpath /var/lib/mongodb --repair --storageEngine wiredTiger Mongo Repair Complete Starting Mongo Server
I tried to revover data using
http://www.alexbevi.com/blog/2016/02/10/recovering-a-wiredtiger-collection-from-a-corrupt-mongodb-installation/
But this is also giving me error while recovery
Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
root@vps306768:~/wiredtiger-2.9.1# ./wt -v -h ../backup/mongodb -C "extensions=[./ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R salvage collection-8--2945606112955784657.wt [1484199812:302917][32152:0x7f757c003700], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 53248: block header checksum of 2750094563 doesn't match expected checksum of 1661035712 [1484199812:303046][32152:0x7f757c003700], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value [1484199812:303094][32152:0x7f757c003700], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic lt-wt: WT_PANIC: WiredTiger library panic
Attached : WiredTiger.wt and WiredTiger.turtle