[SERVER-48259] MongoDB 4.0.17 crashing in Ubuntu 16.04 Created: 17/May/20  Updated: 26/May/20  Resolved: 21/May/20

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 4.0.17
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Harshad Bhagwat Assignee: Carl Champain (Inactive)
Resolution: Done Votes: 0
Labels: repair, repairDatabase
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File diag.tar.gz     File err.rar    
Operating System: ALL
Participants:

 Description   

I have installed Mongo container for my product to log data.

Here is the version of Mongo for your reference.

MongoDB shell version v4.0.17
git version: 0bc918c73390f6e4d6349660e4cd233f5900b69a
OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
allocator: tcmalloc
modules: none
build environment:
distmod: ubuntu1604
distarch: x86_64
target_arch: x86_64

 root@9e31c7f2d640:/# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 38G 3.1G 33G 9% /
tmpfs 64M 0 64M 0% /dev
tmpfs 2.6G 0 2.6G 0% /sys/fs/cgroup
shm 64M 0 64M 0% /dev/shm
/dev/sda1 38G 3.1G 33G 9% /data/db
tmpfs 2.6G 0 2.6G 0% /proc/asound
tmpfs 2.6G 0 2.6G 0% /proc/acpi
tmpfs 2.6G 0 2.6G 0% /proc/scsi
tmpfs 2.6G 0 2.6G 0% /sys/firmware

After logging data for a couple of days I am getting data corruption issues. This is resulting in data loss for me.

 

 
2020-05-17T06:44:22.673+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
 2020-05-17T06:44:22.681+0000 I CONTROL [initandlisten] MongoDB starting : pid=6 port=27017 dbpath=/data/db 64-bit host=9e31c7f2d640
 2020-05-17T06:44:22.681+0000 I CONTROL [initandlisten] db version v4.0.18
 2020-05-17T06:44:22.681+0000 I CONTROL [initandlisten] git version: 6883bdfb8b8cff32176b1fd176df04da9165fd67
 2020-05-17T06:44:22.682+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
 2020-05-17T06:44:22.682+0000 I CONTROL [initandlisten] allocator: tcmalloc
 2020-05-17T06:44:22.682+0000 I CONTROL [initandlisten] modules: none
 2020-05-17T06:44:22.682+0000 I CONTROL [initandlisten] build environment:
 2020-05-17T06:44:22.682+0000 I CONTROL [initandlisten] distmod: ubuntu1604
 2020-05-17T06:44:22.683+0000 I CONTROL [initandlisten] distarch: x86_64
 2020-05-17T06:44:22.683+0000 I CONTROL [initandlisten] target_arch: x86_64
 2020-05-17T06:44:22.683+0000 I CONTROL [initandlisten] options: {}
 2020-05-17T06:44:22.684+0000 W STORAGE [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
 2020-05-17T06:44:22.684+0000 I STORAGE [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
 2020-05-17T06:44:22.684+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
 2020-05-17T06:44:22.685+0000 I STORAGE [initandlisten] 
 2020-05-17T06:44:22.685+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
 2020-05-17T06:44:22.685+0000 I STORAGE [initandlisten] ** See [http://dochub.mongodb.org/core/prodnotes-filesystem]
 2020-05-17T06:44:22.685+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=2148M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
 2020-05-17T06:44:24.298+0000 E STORAGE [initandlisten] WiredTiger error (-31803) [1589697864:298835][6:0x7f8a9bbaaa80], [file:WiredTiger.wt|file:///WiredTiger.wt], connection: __wt_turtle_read, 341: WiredTiger.turtle: fatal turtle file read error: WT_NOTFOUND: item not found Raw: [1589697864:298835][6:0x7f8a9bbaaa80], [file:WiredTiger.wt|file:///WiredTiger.wt], connection: __wt_turtle_read, 341: WiredTiger.turtle: fatal turtle file read error: WT_NOTFOUND: item not found
 2020-05-17T06:44:24.299+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1589697864:299514][6:0x7f8a9bbaaa80], [file:WiredTiger.wt|file:///WiredTiger.wt], connection: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1589697864:299514][6:0x7f8a9bbaaa80], [file:WiredTiger.wt|file:///WiredTiger.wt], connection: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic
 2020-05-17T06:44:24.299+0000 F - [initandlisten] Fatal Assertion 50853 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 420
 2020-05-17T06:44:24.299+0000 F - [initandlisten]
 
***aborting after fassert() failure
 
2020-05-17T06:44:24.369+0000 F - [initandlisten] Got signal: 6 (Aborted).
 0x55a2f0532031 0x55a2f0531249 0x55a2f053172d 0x7f8a9a32e390 0x7f8a99f88428 0x7f8a99f8a02a 0x55a2eeae8761 0x55a2eebec4d6 0x55a2eec56a99 0x55a2eea6fac0 0x55a2eea6feda 0x55a2eec22bb0 0x55a2eec20bdc 0x55a2eec00580 0x55a2eec5532d 0x55a2eec5593f 0x55a2eec55bcc 0x55a2eecd4e16 0x55a2eec50ca8 0x55a2eec200ce 0x55a2eec201ab 0x55a2eebfee83 0x55a2eebc03fa 0x55a2eebc6594 0x55a2eeba63d8 0x55a2ef2c7452 0x55a2eeb56986 0x55a2eeb5ab45 0x55a2eeaea2f9 0x7f8a99f73830 0x55a2eeb55299
 ----- BEGIN BACKTRACE -----
 
{"backtrace":[\\{"b":"55A2EE0C0000","o":"2472031","s":"_ZN5mongo15printStackTraceERSo"}
 
,\{"b":"55A2EE0C0000","o":"2471249"},\{"b":"55A2EE0C0000","o":"247172D"},\{"b":"7F8A9A31D000","o":"11390"},\{"b":"7F8A99F53000","o":"35428","s":"gsignal"},\{"b":"7F8A99F53000","o":"3702A","s":"abort"},{"b":"55A2EE0C0000","o":"A28761","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},\{"b":"55A2EE0C0000","o":"B2C4D6"},\{"b":"55A2EE0C0000","o":"B96A99"},{"b":"55A2EE0C0000","o":"9AFAC0","s":"__wt_err_func"},{"b":"55A2EE0C0000","o":"9AFEDA","s":"__wt_panic"},{"b":"55A2EE0C0000","o":"B62BB0","s":"__wt_turtle_read"},{"b":"55A2EE0C0000","o":"B60BDC","s":"__wt_metadata_search"},{"b":"55A2EE0C0000","o":"B40580","s":"__wt_conn_dhandle_open"},{"b":"55A2EE0C0000","o":"B9532D","s":"__wt_session_get_dhandle"},{"b":"55A2EE0C0000","o":"B9593F","s":"__wt_session_get_dhandle"},{"b":"55A2EE0C0000","o":"B95BCC","s":"__wt_session_get_btree_ckpt"},{"b":"55A2EE0C0000","o":"C14E16","s":"__wt_curfile_open"},\{"b":"55A2EE0C0000","o":"B90CA8"},{"b":"55A2EE0C0000","o":"B600CE","s":"__wt_metadata_cursor_open"},{"b":"55A2EE0C0000","o":"B601AB","s":"__wt_metadata_cursor"},\{"b":"55A2EE0C0000","o":"B3EE83","s":"wiredtiger_open"},{"b":"55A2EE0C0000","o":"B003FA","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_"},{"b":"55A2EE0C0000","o":"B06594","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb"},\{"b":"55A2EE0C0000","o":"AE63D8"},\{"b":"55A2EE0C0000","o":"1207452","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE"},\{"b":"55A2EE0C0000","o":"A96986"},{"b":"55A2EE0C0000","o":"A9AB45","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},\{"b":"55A2EE0C0000","o":"A2A2F9","s":"main"},\{"b":"7F8A99F53000","o":"20830","s":"__libc_start_main"},\{"b":"55A2EE0C0000","o":"A95299","s":"_start"}],"processInfo":{ "mongodbVersion" : "4.0.18", "gitVersion" : "6883bdfb8b8cff32176b1fd176df04da9165fd67", "compiledModules" : [], "uname" :
 
{ "sysname" : "Linux", "release" : "4.14.154-boot2docker", "version" : "#1 SMP Thu Nov 14 19:19:08 UTC 2019", "machine" : "x86_64" }
 
, "somap" : [ \{ "b" : "55A2EE0C0000", "elfType" : 3, "buildId" : "9BAA54C4A92F5506E39F152D9D3F90BD0DA0526F" }, \{ "b" : "7FFDF51B0000", "elfType" : 3, "buildId" : "ACA410FCB9C0B0653F00BCE6E8284FD17E9006CF" }, \{ "b" : "7F8A9B72E000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "03F16B86D21AA657A636EB6D7977D285E5B111E4" }, \{ "b" : "7F8A9B513000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "50A923F8DAFECBCD969C8573116A38C18D0E24D5" }, \{ "b" : "7F8A9B0CE000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "15FFEB43278726B025F020862BF51302822A40EC" }, \{ "b" : "7F8A9AE65000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "FF69EA60EBE05F2DD689D2B26FC85A73E5FBC3A0" }, \{ "b" : "7F8A9AC61000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "37BFC3D8F7E3B022DAC7943B1A5FACD40CEBF0AD" }, \{ "b" : "7F8A9AA59000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "69143E8B39040C964D3958490535322675F15DD3" }, \{ "b" : "7F8A9A750000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "BAD67A84E56E73D031AE507261DA066B35949D34" }, \{ "b" : "7F8A9A53A000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, \{ "b" : "7F8A9A31D000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "B17C21299099640A6D863E423D99265824E7BB16" }, \{ "b" : "7F8A99F53000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "1CA54A6E0D76188105B12E49FE6B8019BF08803A" }, \{ "b" : "7F8A9B99D000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "C0ADBAD6F9A33944F2B3567C078EC472A1DAE98E" }, \{ "b" : "7F8A99D20000", "path" : "/usr/lib/x86_64-linux-gnu/libidn.so.11", "elfType" : 3, "buildId" : "E09D3783AD1D0BBCD3204FA01E4EF6D756E18F57" }, \{ "b" : "7F8A99B04000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "8D1CC1204D6B6D33BD1D2C5A2A0516A2234322CF" }, \{ "b" : "7F8A998BA000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "41971A4A3CCDC54A447F41DF4BD96C948C546E0E" }, \{ "b" : "7F8A996AB000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "E150CC35D09A20D93B4D936D72AA7637DABC978D" }, \{ "b" : "7F8A9945A000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "3D51E97C8DDD5F9EA1674954F8FA8BB2712E64B5" }, \{ "b" : "7F8A99240000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "E2357F88F4E53F0DE76266532DDDCB7FEB6F87A7" }, \{ "b" : "7F8A98F10000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "85FA1D993E14AC403E56E2D45CAE00ACFBD5766C" }, \{ "b" : "7F8A98CDD000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "B11678F560199547DCF726384EA39153EE0DFABF" }, \{ "b" : "7F8A98AA7000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "D6B36C5A463EE0FA84FDD6D5FD3F7726EDB90D54" }, \{ "b" : "7F8A98827000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "7B3533D5998D20EE1A1BE3F87789B69041E7F620" }, \{ "b" : "7F8A98555000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "0EEF7058B0737B68BDF89E5DC604D0AC389C8BB1" }, \{ "b" : "7F8A98326000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "FFBA483A43D9EF73925AC116811890C037523DA1" }, \{ "b" : "7F8A98122000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "E6F8CA010177433A1298DAB8912203E5A5EBB9BA" }, \{ "b" : "7F8A97F17000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "B789D8D4B4FC333405AB34387D9237F954060EA4" }, \{ "b" : "7F8A97CFC000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "4FE2C629045AB0AF26EC5D7F8A3D64B9F8CB21B2" }, \{ "b" : "7F8A97ABB000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "1FE877BE52A424D0636AFD4D35BB330E41D6E0F3" }, \{ "b" : "7F8A97857000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "A0E2D03FF5CF65937F4425D4EFD4D655243809EB" }, \{ "b" : "7F8A97644000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "E07E186694852D8F69459C6AB28A53F8DA3CE3B6" }, \{ "b" : "7F8A97440000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "3364D4BF2113C4E8D17EF533867ECC99A53413D6" }, \{ "b" : "7F8A97237000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "73A8EADBC85860662B24850E71D4AFBE22C33359" }, \{ "b" : "7F8A96FAD000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "59E742306A4EA2872E061ECCE92F35FADDA75357" }, \{ "b" : "7F8A96D0B000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "E5C159E415406AE79D21056D752BA949C408B5B1" }, \{ "b" : "7F8A96AD8000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "7D15576E1F096614D360784E4A01A1F5FAF908C9" }, \{ "b" : "7F8A968C2000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "481DB33C28D88E43DA6BED65E1A7599407D4D818" }, \{ "b" : "7F8A966BA000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "9D9C958F1F4894AFEF6AECD90D1C430EA29AC34F" }, \{ "b" : "7F8A96491000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType" : 3, "buildId" : "57E25072866B2D30CF02EBE7AE623B84F96FA700" }, \{ "b" : "7F8A96282000", "path" : "/usr/lib/x86_64-linux-gnu/libheimbase.so.1", "elfType" : 3, "buildId" : "F6F1B4E9F89B716C4A0BA5819BDFFAF4A13EFB91" }, \{ "b" : "7F8A96037000", "path" : "/usr/lib/x86_64-linux-gnu/libhx509.so.5", "elfType" : 3, "buildId" : "C60082E3BB78D0D42868D9B359B89BF66CE5A1A7" }, \{ "b" : "7F8A95D62000", "path" : "/usr/lib/x86_64-linux-gnu/libsqlite3.so.0", "elfType" : 3, "buildId" : "F0615D615CC5E394B0A73D12AA1382AA872DD1F0" }, \{ "b" : "7F8A95B2A000", "path" : "/lib/x86_64-linux-gnu/libcrypt.so.1", "elfType" : 3, "buildId" : "FD61CA7A6D603E94E5EFD5C88D8810AE104FCF40" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55a2f0532031]
 mongod(+0x2471249) [0x55a2f0531249]
 mongod(+0x247172D) [0x55a2f053172d]
 libpthread.so.0(+0x11390) [0x7f8a9a32e390]
 libc.so.6(gsignal+0x38) [0x7f8a99f88428]
 libc.so.6(abort+0x16A) [0x7f8a99f8a02a]
 mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55a2eeae8761]
 mongod(+0xB2C4D6) [0x55a2eebec4d6]
 mongod(+0xB96A99) [0x55a2eec56a99]
 mongod(__wt_err_func+0x90) [0x55a2eea6fac0]
 mongod(__wt_panic+0x39) [0x55a2eea6feda]
 mongod(__wt_turtle_read+0x2B0) [0x55a2eec22bb0]
 mongod(__wt_metadata_search+0x1CC) [0x55a2eec20bdc]
 mongod(__wt_conn_dhandle_open+0xA0) [0x55a2eec00580]
 mongod(__wt_session_get_dhandle+0xED) [0x55a2eec5532d]
 mongod(__wt_session_get_dhandle+0x6FF) [0x55a2eec5593f]
 mongod(__wt_session_get_btree_ckpt+0x14C) [0x55a2eec55bcc]
 mongod(__wt_curfile_open+0x56) [0x55a2eecd4e16]
 mongod(+0xB90CA8) [0x55a2eec50ca8]
 mongod(__wt_metadata_cursor_open+0x6E) [0x55a2eec200ce]
 mongod(__wt_metadata_cursor+0x4B) [0x55a2eec201ab]
 mongod(wiredtiger_open+0x1A13) [0x55a2eebfee83]
 mongod(_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_+0xAA) [0x55a2eebc03fa]
 mongod(_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb+0x674) [0x55a2eebc6594]
 mongod(+0xAE63D8) [0x55a2eeba63d8]
 mongod(_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE+0x552) [0x55a2ef2c7452]
 mongod(+0xA96986) [0x55a2eeb56986]
 mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0xF95) [0x55a2eeb5ab45]
 mongod(main+0x9) [0x55a2eeaea2f9]
 libc.so.6(__libc_start_main+0xF0) [0x7f8a99f73830]
 mongod(_start+0x29) [0x55a2eeb55299]
 ----- END BACKTRACE -----

 

I would like to know the reason this might happen. Is there any thing which I am missing in configuration ?

Attached the complete error log & diagnostic data for reference.

Thanks

-Harshad 

 



 Comments   
Comment by Carl Champain (Inactive) [ 20/May/20 ]

Hi greathary@gmail.com ,

 

The errors seem to indicate that the failure is most likely caused by the customer’s disk.

 

could we get an evidence that the issue is due to physical corruption and not anything related to Mongo?

The ability to determine a source of corruption depends greatly on:

  1. The logs for the affected node, including before, leading up to, and after the first sign of corruption.
  2. A description of the underlying storage mechanism in use, including details like:
    1. What file system and/or volume management system is in use?
    2. Is data storage locally attached or network-attached?
    3. Are disks RAIDed and if so how?
    4. Are disks SSDs or HDDs?
  3. A description of your backup method, if any.
  4. A description of your disks have been recently checked for integrity?
  5. A history of the deployment, including:
    1. a timeline of version changes
    2. a timeline of hardware upgrade/downgrade cycles or configuration changes
    3. a timeline of disaster recovery or backup restoration activities
    4. a timeline of any manipulations of the underlying database files, including copies or moves, and information about whether mongod was running during each manipulation.

 

Could we verify it from outside whether hardware malfunction is causing this issue

A good disk-checker command is fsck.

 

if it re occurs I could again loose data

To avoid a problem like this in the future, it is our strong recommendation to:

 

That said, the SERVER project is for bugs and feature suggestions for the MongoDB server. As this ticket does not appear to be a bug, I will now close it. If you need further assistance troubleshooting, I encourage you to ask our community by posting on the MongoDB Community Forums or on Stack Overflow with the mongodb tag.

Thank you,
Carl

Comment by Harshad Bhagwat [ 20/May/20 ]

Hi Carl,

As this is my customer machine , could we get an evidence that the issue is due to physical corruption and not anything related to Mongo ? Could we verify it from outside whether hardware malfunction is causing this issue.

I did a mongo repair and it did start my things up but my worry is if it re occurs I could again loose data.

Thanks

-Harshad 

Comment by Carl Champain (Inactive) [ 19/May/20 ]

Hi greathary@gmail.com,

Unfortunately this error message leads us to suspect some form of physical corruption. Please make a complete copy of the database's $dbpath directory to safeguard so that you can work off of the current $dbpath.

The ideal resolution is to perform a clean resync from an unaffected node.

You can also try mongod --repair using the latest version of MongoDB.

In the event that a --repair operation is unsuccessful, then please also provide:

  • The logs of the repair operation.
  • The logs of any attempt to start mongod after the repair operation completed.

Thank you,
Carl

Generated at Thu Feb 08 05:16:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.