[SERVER-38912] Repair fails WT_CURSOR.next: read checksum error Created: 09/Jan/19 Updated: 10/Jan/19 Resolved: 10/Jan/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 4.0.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Rémi AUGUSTE | Assignee: | Kelsey Schubert |
| Resolution: | Done | Votes: | 0 |
| Labels: | wt-repair-success | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Steps To Reproduce: | Maybe having a mongodb running on a NFS shared folder, kill it, and run two instance trying to repair. |
| Participants: |
| Description |
|
I have a micro-service cluster with a NFS share, normally only one instance of mongodb (docker's mongo) is running. After a crash of the cluster I cannot bring the DB back up, the repair fails violently: E STORAGE [initandlisten] WiredTiger error (0) [1547049030:493990][10:0x7f75f72aea00], file:WiredTiger.wt, WT_CURSOR.next: read checksum error for 28672B block at offset 16384: block header checksum of 806183382 doesn't match expected checksum of 619640434 Raw: [1547049030:493990][10:0x7f75f72aea00], file:WiredTiger.wt, WT_CURSOR.next: read checksum error for 28672B block at offset 16384: block header checksum of 806183382 doesn't match expected checksum of 619640434
and displays multiple chunks of data. I'd appreciate any help in restoring the DB, it wasn't backed up and wasn't part of a replicate neither... I attached the faulty WiredTiger.wt, the mongod --repair output and the WiredTiger.turtle (as it seems to be useful for repairing). |
| Comments |
| Comment by Kelsey Schubert [ 10/Jan/19 ] |
|
Thanks for the follow up, foloex. I'm glad you were able to take advantage of the improvements we made in To prevent this type of problem in the future please take note of the following guidelines to help mitigate any issues related to unreliable storage layers or server failures.
Thank you, |
| Comment by Rémi AUGUSTE [ 09/Jan/19 ] |
|
I managed to repair the DB using the latest mongodb (4.0.5) and it was able to run back with docker's version (4.0.1). It's running fine now |