Details
-
Question
-
Resolution: Done
-
Major - P3
-
None
-
3.2.11
-
None
-
None
Description
after an VM clear shutdown we cannot start back mongodb with this error in log:
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (-31802) [1543869878:580798][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: Recovery failed: WT_ERROR: non-specific WiredTiger error |
2018-12-03T20:44:37.130+0000 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint. |
2018-12-03T20:44:37.130+0000 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=6G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (-31802) [1543869878:580654][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: collection-2--2481999773037646866.wt does not appear to be a WiredTiger file: WT_ERROR: non-specific WiredTiger error |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (-31802) [1543869878:580739][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: operation apply failed during recovery: operation type 4 at LSN 59643/86453504: WT_ERROR: non-specific WiredTiger error |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (0) [1543869878:580755][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: WiredTiger is unable to read the recovery log. |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (0) [1543869878:580769][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: This may be due to the log files being encrypted, being from an older version or due to corruption on disk |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (0) [1543869878:580780][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: You should confirm that you have opened the database with the correct options including all encryption and compression options |
2018-12-03T20:44:38.580+0000 E STORAGE [initandlisten] WiredTiger (-31802) [1543869878:580798][2363:0x7f49e439acc0], file:collection-2--2481999773037646866.wt, txn-recover: Recovery failed: WT_ERROR: non-specific WiredTiger error |
2018-12-03T20:44:38.643+0000 I - [initandlisten] Assertion: 28718:-31802: WT_ERROR: non-specific WiredTiger error |
2018-12-03T20:44:38.643+0000 I STORAGE [initandlisten] exception in initAndListen: 28718 -31802: WT_ERROR: non-specific WiredTiger error, terminating |
2018-12-03T20:44:38.643+0000 I CONTROL [initandlisten] dbexit: rc: 100 |
|
The outher ways to get data from this collection file and move them to outher mongo instance facing the same error:
We try:
1) mongod --repair (same error in logs)
2) wt tool specially build for our wiretiger version (http://source.wiredtiger.com/2.9.2/command_line.html#util_salvage)
./wt -v -h /storage1/restore_checkArchive -C "extensions=[./ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R verify collection-2--2481999773037646866 |
[1544005066:138304][28344:0x7fe82dc5c300], file:collection-2--2481999773037646866.wt, WT_SESSION.verify: collection-2--2481999773037646866.wt does not appear to be a WiredTiger file: WT_ERROR: non-specific WiredTiger error
./wt -v -h /storage1/restore_checkArchive -C "extensions=[./ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R salvage collection-2--2481999773037646866.wt |
[1544005420:773648][29867:0x7f8a8dbe0300], file:collection-2--2481999773037646866.wt, WT_SESSION.salvage: collection-2--2481999773037646866.wt does not appear to be a WiredTiger file: WT_ERROR: non-specific WiredTiger error
wt: session.salvage: file:collection-2--2481999773037646866.wt: WT_ERROR: non-specific WiredTiger error
With the help of wt utility (salvage -> dump -> load) we recover some outher collection files to use in outher mongo instance but not this one
Is there some outher way to fix this or just extract data from it?
What additional data (files or so) you need to clarify the problem (The size of problem collection filesis close to 200Gb.)?