[SERVER-22816] Corrupt metadata after unexpected shutdown -> unable to start or repair Created: 23/Feb/16  Updated: 10/Mar/16  Resolved: 10/Mar/16

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Alexandru Dobre Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.turtle     File WiredTiger.wt     File _mdb_catalog.wt     File repair_attempt.tgz     File sizeStorer.wt    
Issue Links:
Duplicate
duplicates SERVER-19815 Improved mongod --repair option for W... Closed
Operating System: ALL
Participants:

 Description   

Using mongodb 3.0.1 with wiredTiger
Following an unexpected shutdown database is unable to start or even repair. Console output indicates wiredTiger is unable to open its metadata files. They are likely to have been corrupted during the server crash. Is there any way to repair them?



 Comments   
Comment by Kelsey Schubert [ 10/Mar/16 ]

Hi a.dobre,

Unfortunately, repairDatabase cannot continue when it hits an empty collection file. There is an improvement planned, SERVER-19815, which would make repairDatabase more robust when executed against WiredTiger data files. Please feel free to watch SERVER-19815 for updates and vote for it as well.

If you are able, I would recommend resyncing from an unaffected node or restoring from a backup.

For MongoDB-related support discussion please post on the mongodb-users group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-users group.

Kind regards,
Thomas

Comment by Alexandru Dobre [ 23/Feb/16 ]

Hello and thank you for the file fixes.
I was able to start the repair process using the fixed files but it failed when reaching the following file "collection-145-2725977566440307375.wt" which is probably corrupt as well (cannot upload since file is empty). Is there any way to just skip recovery of this file and continue with the repair process?

Thanks,
Alexandru

Comment by Ramon Fernandez Marina [ 23/Feb/16 ]

a.dobre, can you please extract the repair_attempt.tgz files in your dbpath and start mongod again?

Thanks,
Ramón.

Comment by Ramon Fernandez Marina [ 23/Feb/16 ]

a.dobre, there are some known issues with early 3.0 releases where unclean shutdowns may leave some of the WiredTiger files in an unclean state and prevent startup.

I'll try to repair these files and upload the results. In the meantime I strongly urge you to upgrade to the latest MongoDB version (3.0.9 in the v3.0 branch, 3.2.3 in the v3.2 branch).

Generated at Thu Feb 08 04:01:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.