[SERVER-19225] Wiredtiger corrupt after shutdown Created: 29/Jun/15  Updated: 03/Jul/15  Resolved: 30/Jun/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Denis Tsai Assignee: Ramon Fernandez Marina
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: HTML File WiredTiger     File WiredTiger.basecfg     File WiredTiger.lock     File WiredTiger.turtle     File WiredTiger.turtle.new     File WiredTiger.wt     File WiredTiger.wt.new     File _mdb_catalog.wt     Text File repair.log     File sizeStorer.wt    
Issue Links:
Duplicate
duplicates SERVER-18316 Database with WT engine fails to reco... Closed
Operating System: ALL
Steps To Reproduce:

improper shutdown while mongod is active

Participants:

 Description   

I'm ran into the same issue as this: https://jira.mongodb.org/browse/SERVER-18448

and I'm wondering if there is now a known method to recover the data or the database now that this is fixed in newer versions. I've tried using "mongod --repair --storageEngine wiredTiger" with no success. The output is saved to attached file repair.log.

I've also attached the same files as issue 18448. Thanks for your help!



 Comments   
Comment by Ramon Fernandez Marina [ 30/Jun/15 ]

Thanks for getting back to us dt665m, glad to hear that the new files work. I would advise you to run repair validate to make sure there are no other issues.

If there were lock files owned by root, attempting to start mongod as a different user should just fail, not cause any problems. The cause of the type of corruption you observed is an unclean shutdown (of either mongod or your system) exposing a bug in WiredTiger (see SERVER-18316 for more details), and I'd recommend you upgrade to 3.0.4 as soon as possible.

I'm going to close this ticket as a duplicate of SERVER-18316, please feel free to open a new ticket if you find a bug in MongoDB.

Regards,
Ramón.

Comment by Denis Tsai [ 30/Jun/15 ]

Hi Ramon!

Thanks a lot for taking the time to fix this. After moving the new files over and setting the relevant owner back to mongodb, everything seems to be back to normal. Thankfully this is a test server and not on our production server. Can you give me some hints on how to prevent this from happening? We do some weird stuff on the test server (for example I noticed that a few of the .lock files and the .sock files were owned by root), which means someone tried to run the mongod service as root instead of using the startup script. Could this have anything to do with it?

Again thank you so much, the recovered data is still valuable to us.

Comment by Ramon Fernandez Marina [ 30/Jun/15 ]

dt665m, I'm uploading the WiredTiger.wt and WiredTiger.turtle files after a repair attempt with a .new suffix. Can you please put them in your dbpath (without the .new suffix) and try to start mongod?

Please let us know how this goes.

Thanks,
Ramón.

Comment by Ramon Fernandez Marina [ 30/Jun/15 ]

Sorry you run into this bug dt665m, and thanks for opening a ticket and uploading the files. We'll take a look and see what the damage was after the unclean shutdown and whether it can be repaired. Stay tuned for updates.

Generated at Thu Feb 08 03:50:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.