[SERVER-35104] Cannot start or --repair after WiredTiger.wt corruption Created: 20/May/18  Updated: 23/Jul/18  Resolved: 16/Jun/18

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.4.9
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: zeroencry Assignee: Kelsey Schubert
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.turtle     File WiredTiger.wt    
Operating System: ALL
Steps To Reproduce:

an unlucky unclear shutdown

Participants:

 Description   

Hi all,

currently, i can't repair my mongodb.

when i repair, i got this issue.

2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten] allocator: tcmalloc
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten] modules: none
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten] build environment:
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten]     distmod: rhel70
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten]     distarch: x86_64
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten]     target_arch: x86_64
 2018-05-20T21:29:38.350+0700 I CONTROL  [initandlisten] options: \{ repair: true, storage: { dbPath: "mongodb/db/" } }
 2018-05-20T21:29:38.373+0700 I -        [initandlisten] Detected data files in mongodb/db/ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
 2018-05-20T21:29:38.374+0700 I STORAGE  [initandlisten] 
 2018-05-20T21:29:38.374+0700 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
 2018-05-20T21:29:38.374+0700 I STORAGE  [initandlisten] **          See [http://dochub.mongodb.org/core/prodnotes-filesystem]
 2018-05-20T21:29:38.374+0700 I STORAGE  [initandlisten] Detected WT journal files.  Running recovery from last checkpoint.
 2018-05-20T21:29:38.374+0700 I STORAGE  [initandlisten] journal to nojournal transition config: create,cache_size=15574M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
 2018-05-20T21:29:38.914+0700 E STORAGE  [initandlisten] WiredTiger error (0) [1526826578:914401][5776:0x2ba1e3481f80], [file:index-42--55760471527035086.wt|file:///index-42--55760471527035086.wt], WT_CURSOR.insert: read checksum error for 12288B block at offset 404283392: block header checksum of 2660055333 doesn't match expected checksum of 2524959011
 2018-05-20T21:29:38.914+0700 E STORAGE  [initandlisten] WiredTiger error (0) [1526826578:914470][5776:0x2ba1e3481f80], [file:index-42--55760471527035086.wt|file:///index-42--55760471527035086.wt], WT_CURSOR.insert: index-42--55760471527035086.wt: encountered an illegal file format or internal value
 2018-05-20T21:29:38.914+0700 E STORAGE  [initandlisten] WiredTiger error (-31804) [1526826578:914486][5776:0x2ba1e3481f80], [file:index-42--55760471527035086.wt|file:///index-42--55760471527035086.wt], WT_CURSOR.insert: the process must exit and restart: WT_PANIC: WiredTiger library panic
 2018-05-20T21:29:38.914+0700 I -        [initandlisten] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361
 2018-05-20T21:29:38.914+0700 I -        [initandlisten] 



 Comments   
Comment by Kelsey Schubert [ 16/Jun/18 ]

Hi zeroencry,

Unfortunately this error most likely indicates widespread filesystem level corruption. In particular, we see that at least one index has become corrupted on disk. In this situation, our best recommendation would be to resync the affected node or restore from a backup if possible.

To prevent this type of problem in the future please take note of the following guidelines to help mitigate any issues related to unreliable storage layers or server failures.

Thank you,
Kelsey

Comment by zeroencry [ 02/Jun/18 ]

Hi @Kelsey T Schubert

I continue to get this error.

I attached  the WiredTiger.wt and WiredTiger.turtle files.

Thanks

Comment by zeroencry [ 22/May/18 ]

Hi Ramon,
I restored my mongodb.
Can you say to me a way which run a repair for the WiredTiger.wt and WiredTiger.turtle files ?
Thank you.

 

Comment by Ramon Fernandez Marina [ 21/May/18 ]

zeroencry, if you can please upload the WiredTiger.wt and WiredTiger.turtle files we can run a repair attempt on those.

Thanks,
Ramón.

Generated at Thu Feb 08 04:38:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.