[SERVER-40067] WiredTiger.wt corrupted Created: 11/Mar/19 Updated: 20/Mar/19 Resolved: 20/Mar/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 3.2.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Michael | Assignee: | Eric Sedor |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows Server 2012 |
||
| Attachments: |
|
| Participants: |
| Description |
|
Hi Support, It looks like wiredtiger.wt file has been corrupted. As per below error. It is possible to repair this file? 2019-03-07T14:14:52.704+0000 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2 }, systemLog: { destination: "file", logAppend: true, path: "C:\Program Files\test\APM_Mongo\logs\mongo.log" } } ***aborting after fassert() failure
|
| Comments |
| Comment by Eric Sedor [ 20/Mar/19 ] |
|
mclass, we've attempted a repair and attached the results as repairedFrom03132019.tgz Importantly, please store a copy of these repaired files with the copy of the dbpath from that time before starting mongod, as a backup. You may need to return to this backup as you explore how to protect your system against further issue. Again, please consider MongoDB 4.0. |
| Comment by Michael [ 20/Mar/19 ] |
|
Hi Eric,
Thank you, I have attached zip "WTFiles3202019.zip" containing WT from 13th and also from the 4.0 repair.
|
| Comment by Eric Sedor [ 20/Mar/19 ] |
|
Please provide the .wt and .turtle file as of your Mar 14 2019 11:44:10 AM GMT comment. We can try our internal repair script one more time from that point. Going forward, we strongly recommend you look into your disk setup (and our other recommendations above) given the rapid speed with which corruption occurred again. And we strongly recommend you upgrade to MongoDB 4.0, which has improved repair capabilities. Again, 3.2 reached end of life on September 2018. Unfortunately, we cannot continue to provide repair attempts on unsupported versions. |
| Comment by Michael [ 20/Mar/19 ] |
|
Hi Eric, I see you have closed the ticket however may I ask. I have installed Mongo 4 as you suggested and run the repair against mongo 3.2 data directory which has completed. However when I run the mongo 3.2 service it logs the following and the service stops.
874+0000 I CONTROL [initandlisten] MongoDB starting : pid=48504 port=27017 dbpath=D:\xxxx\Data 64-bit host=WINDOWS-QQUTJK2 }, systemLog: { destination: "file", logAppend: true, path: "C:\Program Files\xxxx\APM_Mongo\logs\mongo.log" } }
I have also attached the repaired wiredtiger.wt and wiredtiger.turtle in zip "DBrepair.WT.zip". |
| Comment by Eric Sedor [ 14/Mar/19 ] |
|
mclass, it looks like there are several important errors, warnings, and restarts in the logs prior to the checksum error, including an indication that the disk ran out of space. We do try to help with corruption in earlier versions where possible. But MongoDB 3.2 reached end of life on September 2018, and the SERVER project is for bugs and feature requests for active versions of MongoDB. At this time we'd like to ask that you look into using MongoDB 4.0's --repair option and consider our recommendations above. To clarify about backing up the path, we were not suggesting anything fancy: just make a copy of the dbpath so that you don't run --repair on the only copy of the files. For further assistance troubleshooting, please post on the mongodb-user group or on Stack Overflow with the mongodb tag. |
| Comment by Michael [ 14/Mar/19 ] |
|
Hi Eric, I replaced the files you provided however after a short time the mongo service stops. I can see in the logs which I attached that the same error message appears towards the end of the log file starting @ 2019-03-14T10:35:55.122+0000
|
| Comment by Eric Sedor [ 13/Mar/19 ] |
|
Understood. Since this has happened again please take note of the following guidelines to help mitigate any issues related to unreliable storage layers or server failures.
I've attached a repair attempt of the files you provided (dated Mar 13 2019 01:56:52 PM GMT-0400). Please extract these files and replace them in your $dbpath |
| Comment by Michael [ 13/Mar/19 ] |
|
Hi Eric,
Thank you for the feedback, I don't know much about Mongo and backing up the path. I had this same problem last year which was resolved under ticket https://jira.mongodb.org/browse/SERVER-34534?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel
I have attached the log and turtle file. |
| Comment by Eric Sedor [ 13/Mar/19 ] |
|
Hello, as a first step, can you please:
If this is not successful, let us know and please also provide
Thank you |
| Comment by Keith Bostic (Inactive) [ 11/Mar/19 ] |
|
Program Management Note
|