[SERVER-41018] file:WiredTiger.wt, connection: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error Created: 06/May/19  Updated: 29/Jul/19  Resolved: 29/Jul/19

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.4.9
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Larry Assignee: Danny Hatcher (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.turtle     File WiredTiger.wt     File repair.tar    
Operating System: ALL
Participants:

 Description   

We run MongoDB 3.4.9 on Windows Server. Since some disk problems,  WiredTiger data files are corrupted. I copied data files to another server and tried to repair them using mongod --repair, then i got errors below.

Now i need some help to repair the files. Thank you!

 

2019-05-06T15:05:52.151+0800 I CONTROL [initandlisten] MongoDB starting : pid=20012 port=27017 dbpath=/data/db 64-bit host=DockerRegistry
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] db version v3.4.9
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] git version: 876ebee8c7dd0e2d992f36a848ff4dc50ee6603e
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] allocator: tcmalloc
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] modules: none
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] build environment:
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] distmod: ubuntu1604
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] distarch: x86_64
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] target_arch: x86_64
2019-05-06T15:05:52.152+0800 I CONTROL [initandlisten] options: { repair: true, storage:

{ dbPath: "/data/db" }

}
2019-05-06T15:05:52.152+0800 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
2019-05-06T15:05:52.200+0800 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2019-05-06T15:05:52.200+0800 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
2019-05-06T15:05:52.200+0800 I STORAGE [initandlisten]
2019-05-06T15:05:52.200+0800 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-05-06T15:05:52.200+0800 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-05-06T15:05:52.200+0800 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint.
2019-05-06T15:05:52.200+0800 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=3478M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2019-05-06T15:05:52.223+0800 E STORAGE [initandlisten] WiredTiger error (-31802) [1557126352:223732][20012:0x7f2c7ea681c0], file:WiredTiger.wt, connection: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error
2019-05-06T15:05:52.223+0800 E STORAGE [initandlisten] WiredTiger error (0) [1557126352:223805][20012:0x7f2c7ea681c0], file:WiredTiger.wt, connection: WiredTiger has failed to open its metadata
2019-05-06T15:05:52.223+0800 E STORAGE [initandlisten] WiredTiger error (0) [1557126352:223817][20012:0x7f2c7ea681c0], file:WiredTiger.wt, connection: This may be due to the database files being encrypted, being from an older version or due to corruption on disk
2019-05-06T15:05:52.223+0800 E STORAGE [initandlisten] WiredTiger error (0) [1557126352:223827][20012:0x7f2c7ea681c0], file:WiredTiger.wt, connection: You should confirm that you have opened the database with the correct options including all encryption and compression options
2019-05-06T15:05:52.224+0800 I - [initandlisten] Assertion: 28718:-31802: WT_ERROR: non-specific WiredTiger error src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 253
2019-05-06T15:05:52.224+0800 I STORAGE [initandlisten] exception in initAndListen: 28718 -31802: WT_ERROR: non-specific WiredTiger error, terminating
2019-05-06T15:05:52.224+0800 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2019-05-06T15:05:52.224+0800 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
2019-05-06T15:05:52.224+0800 I NETWORK [initandlisten] shutdown: going to flush diaglog...
2019-05-06T15:05:52.224+0800 I CONTROL [initandlisten] now exiting
2019-05-06T15:05:52.224+0800 I CONTROL [initandlisten] shutting down with code:100



 Comments   
Comment by Danny Hatcher (Inactive) [ 06/May/19 ]

Hello Larry,

Our ability to determine the source of this corruption depends greatly on your ability to provide:

  1. The logs for the affected node, including before, leading up to, and after the first sign of corruption.
  2. A description of the underlying storage mechanism in use, including details like:
    1. What file system and/or volume management system is in use?
    2. Is data storage locally attached or network-attached?
    3. Are disks RAIDed and if so how?
    4. Are disks SSDs or HDDs?
  3. A description of your backup method, if any.
  4. A description of your disks have been recently checked for integrity?
  5. A history of the deployment, including:
    1. a timeline of version changes
    2. a timeline of hardware upgrade/downgrade cycles or configuration changes
    3. a timeline of disaster recovery or backup restoration activities
    4. a timeline of any manipulations of the underlying database files, including copies or moves, and information about whether mongod was running during each manipulation.

I have attached a repair attempt as repair.tar. Please extract the files therein and replace the existing files in your $dbpath.

Thank you,

Danny

Generated at Thu Feb 08 04:56:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.