[SERVER-33225] WiredTiger.wt: encountered an illegal file format or internal value Created: 09/Feb/18  Updated: 21/Mar/18  Resolved: 09/Feb/18

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mohammad majidi Assignee: Kelsey Schubert
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.turtle     File WiredTiger.wt     File repair_attempt.tar.gz    
Operating System: ALL
Participants:

 Description   

I am using mongodb 3.6, storage engine - wiredTiger
After a bad server/mongod shutdown, I am unable to start Mongod.
my service status was like :

mongod.service - High-performance, schema-free document-oriented database
   Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2018-02-09 14:18:29 +0330; 2s ago
     Docs: https://docs.mongodb.org/manual
  Process: 2123 ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf (code=exited, status=14)
 Main PID: 2123 (code=exited, status=14)
 
Feb 09 14:18:28 example.com systemd[1]: Started High-performance, schema-free document-oriented database.
Feb 09 14:18:29 example.com mongod[2123]: 2018-02-09T14:18:29.013+0330 W CONTROL  [main] No SSL certificate validation can be 
Feb 09 14:18:29 example.com systemd[1]: mongod.service: Main process exited, code=exited, status=14/n/a
Feb 09 14:18:29 example.com systemd[1]: mongod.service: Unit entered failed state.
Feb 09 14:18:29 example.com systemd[1]: mongod.service: Failed with result 'exit-code'

So I tried to repair mongodb with this command :

sudo mongod --repair --dbpath /var/lib/mongodb

This is what happend :

018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] MongoDB starting : pid=2236 port=27017 dbpath=/var/lib/mongodb 64-bit host=example.com
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] db version v3.2.18
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] git version: 4c1bae566c0c00f996a2feb16febf84936ecaf6f
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] allocator: tcmalloc
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] modules: none
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] build environment:
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten]     distmod: ubuntu1604
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten]     distarch: x86_64
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten]     target_arch: x86_64
2018-02-09T14:24:08.348+0330 I CONTROL  [initandlisten] options: { repair: true, storage: { dbPath: "/var/lib/mongodb" } }
2018-02-09T14:24:08.380+0330 I -        [initandlisten] Detected data files in /var/lib/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2018-02-09T14:24:08.380+0330 I STORAGE  [initandlisten] Detected WT journal files.  Running recovery from last checkpoint.
2018-02-09T14:24:08.380+0330 I STORAGE  [initandlisten] journal to nojournal transition config: create,cache_size=6G,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),verbose=(recovery_progress),
2018-02-09T14:24:08.400+0330 E STORAGE  [initandlisten] WiredTiger (0) [1518173648:400739][2236:0x7f4d446fcc80], file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value
2018-02-09T14:24:08.400+0330 E STORAGE  [initandlisten] WiredTiger (-31804) [1518173648:400812][2236:0x7f4d446fcc80], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic
2018-02-09T14:24:08.400+0330 I -        [initandlisten] Fatal Assertion 28558
2018-02-09T14:24:08.400+0330 I -        [initandlisten] 
 
***aborting after fassert() failure
 
 
2018-02-09T14:24:08.428+0330 F -        [initandlisten] Got signal: 6 (Aborted).
 
 0x1559492 0x1558439 0x1558ca2 0x7f4d43301390 0x7f4d42f5b428 0x7f4d42f5d02a 0x14d5563 0x1275b5a 0x97de59 0x97e03f 0x97e205 0x1c3988c 0x1c381da 0x1bef994 0x1c75add 0x1c76009 0x1c762ac 0x1bfdae1 0x1c72978 0x1c379af 0x1c37a6c 0x1bec12c 0x125a57a 0x1253065 0x114e28e 0x9cb181 0x97efda 0x7f4d42f46830 0x9c6479
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"1159492","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"1158439"},{"b":"400000","o":"1158CA2"},{"b":"7F4D432F0000","o":"11390"},{"b":"7F4D42F26000","o":"35428","s":"gsignal"},{"b":"7F4D42F26000","o":"3702A","s":"abort"},{"b":"400000","o":"10D5563","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"E75B5A"},{"b":"400000","o":"57DE59","s":"__wt_eventv"},{"b":"400000","o":"57E03F","s":"__wt_err"},{"b":"400000","o":"57E205","s":"__wt_panic"},{"b":"400000","o":"183988C","s":"__wt_turtle_read"},{"b":"400000","o":"18381DA","s":"__wt_metadata_search"},{"b":"400000","o":"17EF994","s":"__wt_conn_btree_open"},{"b":"400000","o":"1875ADD","s":"__wt_session_get_btree"},{"b":"400000","o":"1876009","s":"__wt_session_get_btree"},{"b":"400000","o":"18762AC","s":"__wt_session_get_btree_ckpt"},{"b":"400000","o":"17FDAE1","s":"__wt_curfile_open"},{"b":"400000","o":"1872978"},{"b":"400000","o":"18379AF","s":"__wt_metadata_cursor_open"},{"b":"400000","o":"1837A6C","s":"__wt_metadata_cursor"},{"b":"400000","o":"17EC12C","s":"wiredtiger_open"},{"b":"400000","o":"E5A57A","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb"},{"b":"400000","o":"E53065"},{"b":"400000","o":"D4E28E","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"400000","o":"5CB181"},{"b":"400000","o":"57EFDA","s":"main"},{"b":"7F4D42F26000","o":"20830","s":"__libc_start_main"},{"b":"400000","o":"5C6479","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.2.18", "gitVersion" : "4c1bae566c0c00f996a2feb16febf84936ecaf6f", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-104-generic", "version" : "#127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "3C61A0DBFA07F1A1069E23BC3A1ADF2E695605CE" }, { "b" : "7FFCD70B7000", "elfType" : 3, "buildId" : "BF4B5E36B7E2464DCF29A888C247A27A3FF2BC5C" }, { "b" : "7F4D4427C000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "DCF10134B91ED2139E3E8C72564668F5CDBA8522" }, { "b" : "7F4D43E38000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "1649272BE0CA9FA22F082DC86372B6C9959779B0" }, { "b" : "7F4D43C30000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "F951C1E0765FCAE48F82CAFE35D1ADD36D6C9AF9" }, { "b" : "7F4D43A2C000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "0FC788F0861846257B5F1773FBD438E95DFC1032" }, { "b" : "7F4D43723000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "FF7A33D389E756CA381A8189291A968EA5E1F4F8" }, { "b" : "7F4D4350D000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F4D432F0000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "27F189EF8DB8C3734C6A678E6EF3CB0B206D58B2" }, { "b" : "7F4D42F26000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "088A6E00A1814622219F346B41E775B8DD46C518" }, { "b" : "7F4D444E5000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9157F205547F0EB588E2AB1F2F120B74253A43EA" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1559492]
 mongod(+0x1158439) [0x1558439]
 mongod(+0x1158CA2) [0x1558ca2]
 libpthread.so.0(+0x11390) [0x7f4d43301390]
 libc.so.6(gsignal+0x38) [0x7f4d42f5b428]
 libc.so.6(abort+0x16A) [0x7f4d42f5d02a]
 mongod(_ZN5mongo13fassertFailedEi+0x93) [0x14d5563]
 mongod(+0xE75B5A) [0x1275b5a]
 mongod(__wt_eventv+0x3BA) [0x97de59]
 mongod(__wt_err+0x8B) [0x97e03f]
 mongod(__wt_panic+0x24) [0x97e205]
 mongod(__wt_turtle_read+0x24C) [0x1c3988c]
 mongod(__wt_metadata_search+0x8A) [0x1c381da]
 mongod(__wt_conn_btree_open+0x64) [0x1bef994]
 mongod(__wt_session_get_btree+0xDD) [0x1c75add]
 mongod(__wt_session_get_btree+0x609) [0x1c76009]
 mongod(__wt_session_get_btree_ckpt+0x12C) [0x1c762ac]
 mongod(__wt_curfile_open+0x151) [0x1bfdae1]
 mongod(+0x1872978) [0x1c72978]
 mongod(__wt_metadata_cursor_open+0x5F) [0x1c379af]
 mongod(__wt_metadata_cursor+0x3C) [0x1c37a6c]
 mongod(wiredtiger_open+0x18DC) [0x1bec12c]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb+0x86A) [0x125a57a]
 mongod(+0xE53065) [0x1253065]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x3EE) [0x114e28e]
 mongod(+0x5CB181) [0x9cb181]
 mongod(main+0x73A) [0x97efda]
 libc.so.6(__libc_start_main+0xF0) [0x7f4d42f46830]
 mongod(_start+0x29) [0x9c6479]
-----  END BACKTRACE  -----
Aborted

I know this is like a duplicate to other issues but after searching for a whole day I realized that WiredTiger.* files shoud be repaired so I've attached this files to this ticket, so maybe you could fix this files .
Thank you



 Comments   
Comment by Kelsey Schubert [ 09/Feb/18 ]

Hi mr96robot

Unfortunately, this error indicates that there was corruption on the disk, most often cause by a faulty storage layer. In this situation, our best recommendation would be to resync the affected node or restore from a backup if possible.

To prevent this type of problem in the future please take note of the following guidelines to help mitigate any issues related to unreliable storage layers or server failures.

Kind regards,
Kelsey

Comment by Mohammad majidi [ 09/Feb/18 ]

Dear @Kelsey.T.Schubert Thank you for responding.
I extracted your attached file into /var/lib/mongodb, which is my dbpath, I restarted the server And in the log file I found the same error, I tried --repair option but again I got the same error.

MongoDB starting : pid=979 port=27017 dbpath=/var/lib/mongodb 64-bit host=mashinbank.com
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] db version v3.6.2
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] git version: 489d177dbd0f0420a8ca04d39fd78d0a2c539420
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] allocator: tcmalloc
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] modules: none
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] build environment:
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten]     distmod: ubuntu1604
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten]     distarch: x86_64
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten]     target_arch: x86_64
2018-02-10T00:10:11.309+0330 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "0.0.0.0", port: 27017, ssl: { PEMKeyFile: "/etc/ssl/mongo.pem", mode: "requireSSL" } }, processManagement: { timeZoneInfo: "/usr/share/zoneinfo" }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2018-02-10T00:10:11.310+0330 I -        [initandlisten] Detected data files in /var/lib/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2018-02-10T00:10:11.310+0330 I STORAGE  [initandlisten] 
2018-02-10T00:10:11.310+0330 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-02-10T00:10:11.310+0330 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-02-10T00:10:11.310+0330 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=5495M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2018-02-10T00:10:11.328+0330 E STORAGE  [initandlisten] WiredTiger error (0) [1518208811:328695][979:0x7f04f3af59c0], file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value: (__wt_turtle_read, 291)
2018-02-10T00:10:11.328+0330 E STORAGE  [initandlisten] WiredTiger error (-31804) [1518208811:328742][979:0x7f04f3af59c0], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic
2018-02-10T00:10:11.328+0330 F -        [initandlisten] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361
2018-02-10T00:10:11.328+0330 F -        [initandlisten] 
 
***aborting after fassert() failure
 
 
2018-02-10T00:10:11.357+0330 F -        [initandlisten] Got signal: 6 (Aborted).
 
 0x55c4014ee4f1 0x55c4014ed709 0x55c4014edbed 0x7f04f24df390 0x7f04f2139428 0x7f04f213b02a 0x55c3ffc84a8b 0x55c3ffd50e9e 0x55c3ffc21c07 0x55c3ffc21e2c 0x55c3ffc22155 0x55c3ffd7c809 0x55c3ffd7ae89 0x55c3ffd60aca 0x55c3ffdad6db 0x55c3ffdadbfd 0x55c3ffdadebc 0x55c3ffe164d2 0x55c3ffda4468 0x55c3ffd7a55e 0x55c3ffd7a63b 0x55c3ffd5f833 0x55c3ffd35255 0x55c3ffd19958 0x55c3ffee9c60 0x55c3ffc1da10 0x55c3ffcf89db 0x55c3ffc86759 0x7f04f2124830 0x55c3ffce8419
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"55C3FF2FB000","o":"21F34F1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55C3FF2FB000","o":"21F2709"},{"b":"55C3FF2FB000","o":"21F2BED"},{"b":"7F04F24CE000","o":"11390"},{"b":"7F04F2104000","o":"35428","s":"gsignal"},{"b":"7F04F2104000","o":"3702A","s":"abort"},{"b":"55C3FF2FB000","o":"989A8B","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"55C3FF2FB000","o":"A55E9E"},{"b":"55C3FF2FB000","o":"926C07","s":"__wt_eventv"},{"b":"55C3FF2FB000","o":"926E2C","s":"__wt_err"},{"b":"55C3FF2FB000","o":"927155","s":"__wt_panic"},{"b":"55C3FF2FB000","o":"A81809","s":"__wt_turtle_read"},{"b":"55C3FF2FB000","o":"A7FE89","s":"__wt_metadata_search"},{"b":"55C3FF2FB000","o":"A65ACA","s":"__wt_conn_dhandle_open"},{"b":"55C3FF2FB000","o":"AB26DB","s":"__wt_session_get_dhandle"},{"b":"55C3FF2FB000","o":"AB2BFD","s":"__wt_session_get_dhandle"},{"b":"55C3FF2FB000","o":"AB2EBC","s":"__wt_session_get_btree_ckpt"},{"b":"55C3FF2FB000","o":"B1B4D2","s":"__wt_curfile_open"},{"b":"55C3FF2FB000","o":"AA9468"},{"b":"55C3FF2FB000","o":"A7F55E","s":"__wt_metadata_cursor_open"},{"b":"55C3FF2FB000","o":"A7F63B","s":"__wt_metadata_cursor"},{"b":"55C3FF2FB000","o":"A64833","s":"wiredtiger_open"},{"b":"55C3FF2FB000","o":"A3A255","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"55C3FF2FB000","o":"A1E958"},{"b":"55C3FF2FB000","o":"BEEC60","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"55C3FF2FB000","o":"922A10"},{"b":"55C3FF2FB000","o":"9FD9DB","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},{"b":"55C3FF2FB000","o":"98B759","s":"main"},{"b":"7F04F2104000","o":"20830","s":"__libc_start_main"},{"b":"55C3FF2FB000","o":"9ED419","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.6.2", "gitVersion" : "489d177dbd0f0420a8ca04d39fd78d0a2c539420", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-104-generic", "version" : "#127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "55C3FF2FB000", "elfType" : 3, "buildId" : "90F4CC751C09ABD90756CE2480F0217355B846B5" }, { "b" : "7FFD5FFA9000", "elfType" : 3, "buildId" : "BF4B5E36B7E2464DCF29A888C247A27A3FF2BC5C" }, { "b" : "7F04F36C3000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "E7196EFCC044747C45DA6705CBE6E3EB2CE5DAE2" }, { "b" : "7F04F345A000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "DCF10134B91ED2139E3E8C72564668F5CDBA8522" }, { "b" : "7F04F3016000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "1649272BE0CA9FA22F082DC86372B6C9959779B0" }, { "b" : "7F04F2E0E000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "F951C1E0765FCAE48F82CAFE35D1ADD36D6C9AF9" }, { "b" : "7F04F2C0A000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "0FC788F0861846257B5F1773FBD438E95DFC1032" }, { "b" : "7F04F2901000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "FF7A33D389E756CA381A8189291A968EA5E1F4F8" }, { "b" : "7F04F26EB000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F04F24CE000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "27F189EF8DB8C3734C6A678E6EF3CB0B206D58B2" }, { "b" : "7F04F2104000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "088A6E00A1814622219F346B41E775B8DD46C518" }, { "b" : "7F04F38DE000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9157F205547F0EB588E2AB1F2F120B74253A43EA" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55c4014ee4f1]
 mongod(+0x21F2709) [0x55c4014ed709]
 mongod(+0x21F2BED) [0x55c4014edbed]
 libpthread.so.0(+0x11390) [0x7f04f24df390]
 libc.so.6(gsignal+0x38) [0x7f04f2139428]
 libc.so.6(abort+0x16A) [0x7f04f213b02a]
 mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55c3ffc84a8b]
 mongod(+0xA55E9E) [0x55c3ffd50e9e]
 mongod(__wt_eventv+0x3D7) [0x55c3ffc21c07]
 mongod(__wt_err+0x9D) [0x55c3ffc21e2c]
 mongod(__wt_panic+0x33) [0x55c3ffc22155]
 mongod(__wt_turtle_read+0x269) [0x55c3ffd7c809]
 mongod(__wt_metadata_search+0xA9) [0x55c3ffd7ae89]
 mongod(__wt_conn_dhandle_open+0x8A) [0x55c3ffd60aca]
 mongod(__wt_session_get_dhandle+0xFB) [0x55c3ffdad6db]
 mongod(__wt_session_get_dhandle+0x61D) [0x55c3ffdadbfd]
 mongod(__wt_session_get_btree_ckpt+0x14C) [0x55c3ffdadebc]
 mongod(__wt_curfile_open+0x52) [0x55c3ffe164d2]
 mongod(+0xAA9468) [0x55c3ffda4468]
 mongod(__wt_metadata_cursor_open+0x6E) [0x55c3ffd7a55e]
 mongod(__wt_metadata_cursor+0x4B) [0x55c3ffd7a63b]
 mongod(wiredtiger_open+0x19C3) [0x55c3ffd5f833]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x845) [0x55c3ffd35255]
 mongod(+0xA1E958) [0x55c3ffd19958]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x650) [0x55c3ffee9c60]
 mongod(+0x922A10) [0x55c3ffc1da10]
 mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0x86B) [0x55c3ffcf89db]
 mongod(main+0x9) [0x55c3ffc86759]
 libc.so.6(__libc_start_main+0xF0) [0x7f04f2124830]
 mongod(_start+0x29) [0x55c3ffce8419]
-----  END BACKTRACE  -----

Comment by Kelsey Schubert [ 09/Feb/18 ]

Hello mr96robot,

Thank you for your report. I've attached a repair attempt of the files you provided. Please extract these files and replace them in your $dbpath and let us know if it resolves the issue. If you are still seeing errors after replacing these files, please provide the complete logs from the affected node(s) so that we can further investigate.

Thanks,
Kelsey

Comment by Mohammad majidi [ 09/Feb/18 ]

please please please help me
I'm going crazy I don't know what to do anymore, I searched the whole web and I didn't find any solution I just saw some other tickets like my issue and realized you can repair these files. please help me

Comment by Mohammad majidi [ 09/Feb/18 ]

Any advices ?

Generated at Thu Feb 08 04:32:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.