[SERVER-37737] WT_ERROR non-specific WiredTiger error Created: 24/Oct/18  Updated: 25/Oct/18  Resolved: 25/Oct/18

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.2.21
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Rafael Oceguera Assignee: Danny Hatcher (Inactive)
Resolution: Done Votes: 0
Labels: wtc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

After an unclean shutdown, Mongo fails to start. I cannot provide WiredTiger.turtle or WiredTiger.wt. Is there any way to recover? Error logs below

rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=5675ebccdadc
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] db version v3.2.21
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] git version: 1ab1010737145ba3761318508ff65ba74dfe8155
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] allocator: tcmalloc
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] modules: none
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] build environment:
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] distmod: debian81
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] distarch: x86_64
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] target_arch: x86_64
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.016+0000 I CONTROL [initandlisten] options: { replication:

{ oplogSizeMB: 128, replSet: "rs0" }

, storage: { mmapv1:

{ smallFiles: true }

} }
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.043+0000 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.045+0000 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.050+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.050+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=17G,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),verbose=(recovery_progress),
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.115+0000 E STORAGE [initandlisten] WiredTiger (-31802) [1540406844:115295][1:0x7f5d9c3efcc0], file:WiredTiger.wt, connection: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.115+0000 E STORAGE [initandlisten] WiredTiger (0) [1540406844:115356][1:0x7f5d9c3efcc0], file:WiredTiger.wt, connection: WiredTiger has failed to open its metadata
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.115+0000 E STORAGE [initandlisten] WiredTiger (0) [1540406844:115373][1:0x7f5d9c3efcc0], file:WiredTiger.wt, connection: This may be due to the database files being encrypted, being from an older version or due to corruption on disk
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.115+0000 E STORAGE [initandlisten] WiredTiger (0) [1540406844:115386][1:0x7f5d9c3efcc0], file:WiredTiger.wt, connection: You should confirm that you have opened the database with the correct options including all encryption and compression options
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.125+0000 I - [initandlisten] Assertion: 28595:-31802: WT_ERROR: non-specific WiredTiger error
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.126+0000 I STORAGE [initandlisten] exception in initAndListen: 28595 -31802: WT_ERROR: non-specific WiredTiger error, terminating
rocketchat_mongo.1.tx30eimlae23@host.domain.com | 2018-10-24T18:47:24.126+0000 I CONTROL [initandlisten] dbexit: rc: 100



 Comments   
Comment by Danny Hatcher (Inactive) [ 25/Oct/18 ]

Hello Rafael,

Unfortunately, this error indicates that there was corruption on the disk, most often caused by a faulty storage layer. In this situation, our best recommendation would be to resync the affected node or restore from a backup if possible.

To prevent this type of problem in the future please take note of the following guidelines to help mitigate any issues related to unreliable storage layers or server failures.

Thank you,

Danny

Generated at Thu Feb 08 04:46:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.