[SERVER-49412] file:WiredTiger.wt read checksum error, mongodb won't start Created: 09/Jul/20 Updated: 14/Jul/20 Resolved: 14/Jul/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Critical - P2 |
| Reporter: | Michael Peterson | Assignee: | Dmitry Agranat |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Participants: |
| Description |
|
Hi, unfortuneatly I am having issues getting a mongodb started back up after a system failure. Thank you, logs below: 2020-07-09T21:16:03.399+0000 I CONTROL [initandlisten] MongoDB starting : pid=29 port=27017 dbpath=/var/lib/mongodb/data 64-bit host=mongodb-1-wnfnn , replication: { oplogSizeMB: 64 }, security: { authorization: "disabled" }, storage: { dbPath: "/var/lib/mongodb/data", wiredTiger: { engineConfig: { cacheSizeGB: 1.0 } } }, systemLog: { quiet: true } } ***aborting after fassert() failure exception: connect failed 0x56354b05735a 0x56354b056699 0x56354b056b66 0x7f47de49e6d0 0x7f47de0f8277 0x7f47de0f9968 0x56354a2b745b 0x56354ad5cc26 0x56354a2c077a 0x56354a2c0991 0x56354a2c0bef 0x56354b84887f 0x56354b848ddb 0x56354b845435 0x56354b849fa7 0x56354b863a35 0x56354b89d3f3 0x56354b928747 0x56354b928c96 0x56354b928f5c 0x56354b8abb31 0x56354b91e9f8 0x56354b8e82be 0x56354b8e83ab 0x56354b899777 0x56354ad42031 0x56354ad39fc6 0x56354ac20517 0x56354a29d984 0x56354a2c1e44 0x7f47de0e4445 0x56354a31f615 ,{"b":"56354A0B4000","o":"FA2699"},{"b":"56354A0B4000","o":"FA2B66"},{"b":"7F47DE48F000","o":"F6D0"},{"b":"7F47DE0C2000","o":"36277","s":"gsignal"},{"b":"7F47DE0C2000","o":"37968","s":"abort"},{"b":"56354A0B4000","o":"20345B"},{"b":"56354A0B4000","o":"CA8C26"},{"b":"56354A0B4000","o":"20C77A"},{"b":"56354A0B4000","o":"20C991","s":"_wt_err"},{"b":"56354A0B4000","o":"20CBEF","s":"wt_panic"},{"b":"56354A0B4000","o":"179487F"},{"b":"56354A0B4000","o":"1794DDB"},{"b":"56354A0B4000","o":"1791435"},{"b":"56354A0B4000","o":"1795FA7"},{"b":"56354A0B4000","o":"17AFA35"},{"b":"56354A0B4000","o":"17E93F3"},{"b":"56354A0B4000","o":"1874747"},{"b":"56354A0B4000","o":"1874C96"},{"b":"56354A0B4000","o":"1874F5C"},{"b":"56354A0B4000","o":"17F7B31"},{"b":"56354A0B4000","o":"186A9F8"},{"b":"56354A0B4000","o":"18342BE"},{"b":"56354A0B4000","o":"18343AB"},{"b":"56354A0B4000","o":"17E5777","s":"wiredtiger_open"},{"b":"56354A0B4000","o":"C8E031"},{"b":"56354A0B4000","o":"C85FC6"},{"b":"56354A0B4000","o":"B6C517"},{"b":"56354A0B4000","o":"1E9984"},{"b":"56354A0B4000","o":"20DE44"},{"b":"7F47DE0C2000","o":"22445","s":"_libc_start_main"},{"b":"56354A0B4000","o":"26B615"}],"processInfo":{ "mongodbVersion" : "3.4.9", "gitVersion" : "876ebee8c7dd0e2d992f36a848ff4dc50ee6603e", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-1127.13.1.el7.x86_64", "version" : "#1 SMP Tue Jun 23 15:46:38 UTC 2020", "machine" : "x86_64" }, "somap" : [ { "b" : "56354A0B4000", "elfType" : 3, "buildId" : "28974745B6B8764D64F97F6BF19D04AC5EDD5B65" }, { "b" : "7FFE6449A000", "elfType" : 3, "buildId" : "B1962B2DF9C13102C04B311EBA759764C1B8E9D8" }, { "b" : "7F47E1749000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EA8E45DC8E395CC5E26890470112D97A1F1E0B65" }, { "b" : "7F47E14C0000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libyaml-cpp.so.rh-mongodb34-0.5", "elfType" : 3, "buildId" : "FE3E65529D3D5623ECF32A331FFBA7ACD02D40A6" }, { "b" : "7F47E12BA000", "path" : "/lib64/libsnappy.so.1", "elfType" : 3, "buildId" : "3CEB901120465B031FA92FA449A9AD981CEC1659" }, { "b" : "7F47E1048000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libtcmalloc.so.rh-mongodb34-4", "elfType" : 3, "buildId" : "9B8AE99A244389054FED1DC74C0F7B79CD6920CE" }, { "b" : "7F47E0DE6000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }, { "b" : "7F47E0BDD000", "path" : "/lib64/libpcrecpp.so.0", "elfType" : 3, "buildId" : "8BBAA8A5638DCB0C4B523FCA4B54613AE543BDF1" }, { "b" : "7F47E096A000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_program_options.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "51F7B5995F6FD852D84CE889AED4DF5807C588FD" }, { "b" : "7F47E0753000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_filesystem.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "BC537D140DFB358B75B19A985EDE485EB9D122EE" }, { "b" : "7F47E0530000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_thread.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "66EB1D274A8B36597AE3AEDB3A86730FD37F5317" }, { "b" : "7F47E032C000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_system.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "5BCE91B1A7B7AAA44098CF23DA1B795F62F2B1D4" }, { "b" : "7F47E0114000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_iostreams.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "5FF6A64EF26564056ECEB54D3BD7923CEDB27112" }, { "b" : "7F47DFF0C000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_chrono.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "413AD9DB5A13FFC6E5252AD320DC2C84D17753DF" }, { "b" : "7F47DFBFC000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libboost_regex.so.rh-mongodb34-1.60.0", "elfType" : 3, "buildId" : "C581BE7B9B2C0A59BC05D34F6922A66C5D89D475" }, { "b" : "7F47DF9A9000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libstemmer.so.rh-mongodb34-0", "elfType" : 3, "buildId" : "EAAFF5B4F644622B2E88F23A70098C539A7DA9E8" }, { "b" : "7F47DF737000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "6468A4E28A19CDD885A3CBC30E009589CA4C2E92" }, { "b" : "7F47DF2D6000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "67CEB4EDD36BFE0EB31CD92DA2694ACA5377A599" }, { "b" : "7F47DF0CE000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "D33989EC31EFE745EB0D3B68A92D19E77D7DDFDA" }, { "b" : "7F47DEECA000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "5CDB5A56336E7E2BD14FFA189411E44A834AFCD8" }, { "b" : "7F47DEBC3000", "path" : "/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "9589AE0FDA6AEB1183EBA1C62A328F933E7817FD" }, { "b" : "7F47DE8C1000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "F4CAE74047F9AA2D5A71FDEC67C4285D75753EBA" }, { "b" : "7F47DE6AB000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "531AA1391EA4E1489D5EF11AA5DC2FFD9E2BDFEE" }, { "b" : "7F47DE48F000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "F4C04BCE85D2D269D0A2AF4972FC69805B50345B" }, { "b" : "7F47DE0C2000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "CB4B7554D1ADBEF2F001142DD6F0A5139FC9AA69" }, { "b" : "7F47DDEA8000", "path" : "/opt/rh/rh-mongodb34/root/usr/lib64/libunwind.so.rh-mongodb34-8", "elfType" : 3, "buildId" : "CE5996C64EA0376E71D6F37FCEC1FB95F860972F" }, { "b" : "7F47E195F000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D266B1F6650927E18108323BCCA8F7B68E68EB92" }, { "b" : "7F47DDC98000", "path" : "/lib64/libbz2.so.1", "elfType" : 3, "buildId" : "0C85C0386F0CF41EA39969CF7F58A558D1AD3235" }, { "b" : "7F47DC6C4000", "path" : "/lib64/libicudata.so.50", "elfType" : 3, "buildId" : "04813117A9FB30F2DB02B8C546C4FC38BF4890A0" }, { "b" : "7F47DC2C5000", "path" : "/lib64/libicui18n.so.50", "elfType" : 3, "buildId" : "7CAA9714B33AA18B16FAFDC88FA43185238C6C86" }, { "b" : "7F47DBF4C000", "path" : "/lib64/libicuuc.so.50", "elfType" : 3, "buildId" : "E9EA46B17EEE2AE15094650C00B891C094CB802D" }, { "b" : "7F47DBCFF000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "16FE0DC6CEFC5F444BC876516D02EFE9CC2D432F" }, { "b" : "7F47DBA17000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "D1CD1B94855A85FBC735C745DB39BC096F7D8CC3" }, { "b" : "7F47DB813000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "2C7EF64EF0C5AF8BCFA8F9E628E5605A7D8C52D3" }, { "b" : "7F47DB5E0000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "A20F715C514B3EA873F4CC77D585A50CB670E266" }, { "b" : "7F47DB3D2000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "9F82B06CE44724A8ACE827A2C95A9A76619EA314" }, { "b" : "7F47DB1CE000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F47DAFB5000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "2BDC2B6FF0B2C204CCE34D139A9EADA0272EB070" }, { "b" : "7F47DAD8E000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "1B0C0DC4F9054C5922082921C1447739F27AC8F4" } ] }} |
| Comments |
| Comment by Michael Peterson [ 13/Jul/20 ] |
|
The fourth backup I pulled i was able to run a successful repair on. This can be cancelled (i dont see how to do it myself?) Thank you for all of your work you do for the community! |