-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.18
-
Component/s: None
-
None
-
ALL
-
-
None
-
3
-
None
-
None
-
None
-
None
-
None
-
None
We are using MongoDB v3.6.18 as docker containers to store the logs from the applications. There are 3 instances of MongoDB running at any point of time.
Today, the database got corrupted due to improper shutdown. When I tried to repair this, the repair tool gave an error saying that the operation is unsuccessful. The logs for the repair operation are follows:
[root@dcvsl125 mongodb]# docker run -it -v /docker/services/mongodb/db01:/data/db mongo:3.6.18 mongod --repair
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=c4feb442a438
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] db version v3.6.18
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] git version: 2005f25eed7ed88fa698d9b800fe536bb0410ba4
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] allocator: tcmalloc
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] modules: none
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] build environment:
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] distmod: ubuntu1604
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] distarch: x86_64
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] target_arch: x86_64
2021-12-07T01:49:59.777+0000 I CONTROL [initandlisten] options: { net:
, repair: true }
2021-12-07T01:49:59.778+0000 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
2021-12-07T01:49:59.778+0000 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2021-12-07T01:49:59.779+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
2021-12-07T01:49:59.779+0000 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint.
2021-12-07T01:49:59.779+0000 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=63873M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2021-12-07T01:50:00.410+0000 E STORAGE [initandlisten] WiredTiger error (-31802) [1638841800:410928][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 604: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error Raw: [1638841800:410928][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 604: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error
2021-12-07T01:50:00.411+0000 E STORAGE [initandlisten] WiredTiger error (0) [1638841800:411036][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 611: WiredTiger has failed to open its metadata Raw: [1638841800:411036][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 611: WiredTiger has failed to open its metadata
2021-12-07T01:50:00.411+0000 E STORAGE [initandlisten] WiredTiger error (0) [1638841800:411063][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 614: This may be due to the database files being encrypted, being from an older version or due to corruption on disk Raw: [1638841800:411063][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 614: This may be due to the database files being encrypted, being from an older version or due to corruption on disk
2021-12-07T01:50:00.411+0000 E STORAGE [initandlisten] WiredTiger error (0) [1638841800:411082][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 617: You should confirm that you have opened the database with the correct options including all encryption and compression options Raw: [1638841800:411082][1:0x7f27e7e59a40], file:WiredTiger.wt, connection: __wt_btree_tree_open, 617: You should confirm that you have opened the database with the correct options including all encryption and compression options
2021-12-07T01:50:00.414+0000 F STORAGE [initandlisten] WiredTiger metadata corruption detected
2021-12-07T01:50:00.414+0000 F STORAGE [initandlisten] This version of MongoDB is unable to repair this kind of corruption, but version 4.0.3+ may be able to repair it. See http://dochub.mongodb.org/core/repair for more information.
2021-12-07T01:50:00.414+0000 F - [initandlisten] Fatal Assertion 50944 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 71
2021-12-07T01:50:00.414+0000 F - [initandlisten]
***aborting after fassert() failure
[root@dcvsl125 mongodb]#
Looking around for the issue, I came across the following Jira ticket (https://jira.mongodb.org/browse/SERVER-40088) which had the similar issue reported.
Attaching the files from this database.
Can you please help me in recovering the data for this database?
Please let me know if you need more information.
- duplicates
-
SERVER-61936 WiredTiger metadata corruption detected - unable to repair
-
- Closed
-