[SERVER-30635] WiredTiger read checksum error Created: 14/Aug/17  Updated: 06/Sep/17  Resolved: 15/Aug/17

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Bing Zou Assignee: Mark Agarunov
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File SERVER-30635-repair.tar.gz     File WiredTiger.turtle     File WiredTiger.wt    
Operating System: ALL
Participants:

 Description   

mongo crashed again. Please help me fix mongo files. Thanks a lot.



 Comments   
Comment by Bing Zou [ 16/Aug/17 ]

there is no backup as it's my local. I guess I will have to start from scratch.

Thanks for the help.

Comment by Mark Agarunov [ 15/Aug/17 ]

Hello xigua,

Unfortunately, this error indicates that there was additional corruption on the disk. In this situation, my best recommendation would be to resync the affected node or restore from a backup if possible.

Thanks,
Mark

Comment by Bing Zou [ 15/Aug/17 ]

somehow doesn't help this time. got the following error while starting up

2017-08-15T03:56:37.725+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=2690fba6587a
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] db version v3.4.1
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] git version: 5e103c4f5583e2566a45d740225dc250baacfbd7
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] allocator: tcmalloc
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] modules: none
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] build environment:
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] distmod: debian81
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] distarch: x86_64
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] target_arch: x86_64
2017-08-15T03:56:37.726+0000 I CONTROL [initandlisten] options: { storage:

{ engine: "wiredTiger" }

}
2017-08-15T03:56:37.728+0000 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
2017-08-15T03:56:37.737+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
2017-08-15T03:56:37.738+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=487M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-08-15T03:56:38.958+0000 I - [initandlisten] Fatal Assertion 34433 at src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp 793
2017-08-15T03:56:38.958+0000 I - [initandlisten]

***aborting after fassert() failure

This is my local mongo inside docker, the /data/db volume is mapped to local disk. My mac crashed a few time the day before which seemed cause the docker mongo failure.
Please help.

It's not production but I do have many databases on my local for my day to day work. My mac is late 2013 MBP with SSD and OS is Sierra.
Thanks,
Bing

Comment by Mark Agarunov [ 14/Aug/17 ]

Hello xigua,

Thank you for the report. I've attached a repair attempt of the files you've provided. Would you please extract these files and replace them in your $dbpath and let us know if it resolves the issue? If you are still seeing errors after replacing these files, please provide the complete logs from mongod so that we can further investigate. Additionally, if this issue persists, please provide the following information:

  1. What kind of underlying storage mechanism are you using? Are the storage devices attached locally or over the network? Are the disks SSDs or HDDs? What kind of RAID and/or volume management system are you using?
  2. Would you please check the integrity of your disks?
  3. Has the database always been running this version of MongoDB? If not please describe the upgrade/downgrade cycles the database has been through.
  4. Have you manipulated (copied or moved) the underlying database files? If so, was mongod running?
  5. Have you ever restored this instance from backups?
  6. What method do you use to create backups?
  7. When was the underlying filesystem last checked and is it currently marked clean?

Thanks,
Mark

Generated at Thu Feb 08 04:24:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.