|
Hi lyzenghq@gmail.com,
Unfortunately we aren't able to repair this corruption.
To avoid a problem like this in the future, it is our strong recommendation to:
Best,
Edwin
|
|
Tried v3.6.23, still this error
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] MongoDB starting : pid=92728 port=27017 dbpath=/home/user/mongodb/io01/var/lib/mongodb/ 64-bit host=mu01
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] db version v3.6.23
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] git version: d352e6a4764659e0d0350ce77279de3c1f243e5c
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] allocator: tcmalloc
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] modules: none
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] build environment:
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] distmod: rhel70
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] distarch: x86_64
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] target_arch: x86_64
|
2021-05-13T09:24:17.034+0800 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: "/home/user/mongodb/io01/var/lib/mongodb/", engine: "wiredTiger" } }
|
2021-05-13T09:24:17.037+0800 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint.
|
2021-05-13T09:24:17.037+0800 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=128194M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(
|
fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
|
2021-05-13T09:24:17.645+0800 E STORAGE [initandlisten] WiredTiger error (13) [1620869057:645700][92728:0x2b856e528d00], file:WiredTiger.wt, connection: __ckpt_version_chk, 596: file:WiredTiger.wt is an unsupported WiredTiger source file
|
version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied Raw: [1620869057:645700][92728:0x2b856e528d00], file:WiredTiger.wt, connection: __ckpt_version_chk, 596: file:WiredTiger.wt is an unsupported Wir
|
edTiger source file version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied
|
2021-05-13T09:24:17.647+0800 E - [initandlisten] Assertion: 28718:13: Permission denied src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 463
|
2021-05-13T09:24:17.647+0800 I STORAGE [initandlisten] exception in initAndListen: Location28718: 13: Permission denied, terminating
|
2021-05-13T09:24:17.647+0800 I NETWORK [initandlisten] shutdown: going to close listening sockets...
|
2021-05-13T09:24:17.647+0800 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
|
2021-05-13T09:24:17.647+0800 I CONTROL [initandlisten] now exiting
|
2021-05-13T09:24:17.647+0800 I CONTROL [initandlisten] shutting down with code:100
|
|
|
Just a hunch lyzenghq@gmail.com, but can you try with a later version of 3.6? I see you're running on 3.6.3 and that error message looks like something I'd expect when trying to open datafiles that were output by a later version of 3.6.
|
|
I used v3.6 to run the repair and still have this error
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] MongoDB starting : pid=20385 port=27017 dbpath=/var/lib/mongo/ 64-bit host=io01
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] db version v3.6.3
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] git version: nogitversion
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] allocator: tcmalloc
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] modules: none
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] build environment:
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] distarch: x86_64
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] target_arch: x86_64
|
2021-05-13T08:45:43.122+0800 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: "/var/lib/mongo/", engine: "wiredTiger" } }
|
2021-05-13T08:45:43.125+0800 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint.
|
2021-05-13T08:45:43.125+0800 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=128282M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archi
|
ve=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
|
2021-05-13T08:45:43.715+0800 E STORAGE [initandlisten] WiredTiger error (13) [1620866743:715603][20385:0x7f4edae11b80], file:WiredTiger.wt, connection: file:WiredTiger.wt is an unsupported WiredTiger source file version 2.1; this WiredTi
|
ger build only supports versions from 1.1 to 1.1: Permission denied
|
2021-05-13T08:45:43.716+0800 E - [initandlisten] Assertion: 28718:13: Permission denied src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 389
|
2021-05-13T08:45:43.716+0800 I STORAGE [initandlisten] exception in initAndListen: Location28718: 13: Permission denied, terminating
|
2021-05-13T08:45:43.716+0800 I NETWORK [initandlisten] shutdown: going to close listening sockets...
|
2021-05-13T08:45:43.716+0800 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
|
2021-05-13T08:45:43.717+0800 I CONTROL [initandlisten] now exiting
|
2021-05-13T08:45:43.717+0800 I CONTROL [initandlisten] shutting down with code:100
|
|
|
Hi lyzenghq@gmail.com,
Can you attempt to run using MongoDB v3.6?
Best,
Edwin
|
|
And,
[root@io01 ~]# sudo -u mongodb mongod --version
|
db version v4.4.5
|
Build Info: {
|
"version": "4.4.5",
|
"gitVersion": "ff5cb77101b052fa02da43b8538093486cf9b3f7",
|
"openSSLVersion": "OpenSSL 1.0.1e-fips 11 Feb 2013",
|
"modules": [],
|
"allocator": "tcmalloc",
|
"environment": {
|
"distmod": "rhel70",
|
"distarch": "x86_64",
|
"target_arch": "x86_64"
|
}
|
}
|
|
|
The permissions of the files in $dbpath all seem to be correct, as detailed below.
[root@io01 ~]# ls -ld /var/lib/mongo/
|
drwxr-x--- 5 mongodb mongodb 32 May 12 08:42 /var/lib/mongo/
|
[root@io01 ~]#
|
[root@io01 ~]# ls -l /var/lib/mongo/
|
total 720464057
|
-rw------- 1 mongodb mongodb 16384 Apr 14 15:42 collection-0-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 328083353600 Apr 28 03:36 collection-0--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 36864 Apr 14 15:43 collection-2-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 170693124096 Apr 28 03:36 collection-3--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 36864 Apr 28 03:35 collection-4-3978976212545669616.wt
|
drwx------ 2 mongodb mongodb 23 Apr 28 03:36 diagnostic.data
|
-rw------- 1 mongodb mongodb 11099844608 Apr 28 03:36 index-10--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 16384 Apr 14 15:42 index-1-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 53919219712 Apr 28 03:36 index-1--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 33569361920 Apr 28 03:36 index-2--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 36864 Apr 14 15:43 index-3-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 43571269632 Apr 28 03:36 index-4--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 36864 Apr 26 11:15 index-5-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 26918404096 Apr 28 03:36 index-5--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 36864 Apr 28 03:35 index-6-3978976212545669616.wt
|
-rw------- 1 mongodb mongodb 26919858176 Apr 28 03:36 index-6--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 15400296448 Apr 28 03:36 index-7--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 16370061312 Apr 28 03:36 index-8--4446081023038618115.wt
|
-rw------- 1 mongodb mongodb 11209912320 Apr 28 03:36 index-9--4446081023038618115.wt
|
drwx------ 2 mongodb mongodb 22 May 7 10:16 journal
|
-rw------- 1 mongodb mongodb 36864 Apr 20 10:59 _mdb_catalog.wt
|
-rw------- 1 mongodb mongodb 0 May 6 14:20 mongod.lock
|
-rw------- 1 mongodb mongodb 73 May 12 08:42 _repair_incomplete
|
-rw------- 1 mongodb mongodb 36864 Apr 28 03:36 sizeStorer.wt
|
-rw------- 1 mongodb mongodb 114 Apr 14 15:17 storage.bson
|
drwx------ 2 mongodb mongodb 0 Apr 16 09:28 _tmp
|
-rw------- 1 mongodb mongodb 48 Apr 14 15:17 WiredTiger
|
-rw------- 1 mongodb mongodb 4096 Apr 14 15:42 WiredTigerLAS.wt
|
-rw------- 1 mongodb mongodb 21 Apr 14 15:17 WiredTiger.lock
|
-rw------- 1 mongodb mongodb 1452 May 7 23:49 WiredTiger.turtle
|
-rw------- 1 mongodb mongodb 94208 May 7 23:49 WiredTiger.wt
|
-rw------- 1 mongodb mongodb 94208 May 12 08:42 WiredTiger.wt.orig
|
Does the above error message mean that the version of the WiredTiger.wt file is not supported?
In addition, the repair command I used is:
sudo -u mongodb mongod --repair --dbpath /var/lib/mongo/ --storageEngine wiredTiger
|
|
|
There appears to be an issue with the permissions of the files in the repair attempt I provided based on these log lines:
{"t":{"$date":"2021-05-08T11:13:58.974+08:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":13,"message":"[1620443638:974462][59086:0x7f77817f8bc0], file:WiredTiger.wt, connection: __ckpt_version_chk, 1112: file:WiredTiger.wt is an unsupported WiredTiger source file version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied"}}
|
{"t":{"$date":"2021-05-08T11:13:58.985+08:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":13,"message":"[1620443638:985351][59086:0x7f77817f8bc0], file:WiredTiger.wt, connection: __ckpt_version_chk, 1112: file:WiredTiger.wt is an unsupported WiredTiger source file version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied"}}
|
{"t":{"$date":"2021-05-08T11:13:58.996+08:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":13,"message":"[1620443638:996098][59086:0x7f77817f8bc0], file:WiredTiger.wt, connection: __ckpt_version_chk, 1112: file:WiredTiger.wt is an unsupported WiredTiger source file version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied"}}
|
{"t":{"$date":"2021-05-08T11:13:58.997+08:00"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
|
{"t":{"$date":"2021-05-08T11:13:58.997+08:00"},"s":"W", "c":"STORAGE", "id":22349, "ctx":"initandlisten","msg":"Attempting to salvage WiredTiger metadata"}
|
{"t":{"$date":"2021-05-08T11:13:59.119+08:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":13,"message":"[1620443639:119295][59086:0x7f77817f8bc0], file:WiredTiger.wt, WT_SESSION.salvage: __ckpt_version_chk, 1112: file:WiredTiger.wt is an unsupported WiredTiger source file version 2.1; this WiredTiger build only supports versions from 1.1 to 1.1: Permission denied"}}
|
{"t":{"$date":"2021-05-08T11:13:59.120+08:00"},"s":"F", "c":"STORAGE", "id":50947, "ctx":"initandlisten","msg":"Failed to salvage WiredTiger metadata","attr":{"details":"13: Permission denied"}}
|
Can you please verify that the permissions of the repair files are correctly set?
Best,
Edwin
|
|
After replacing the files in tar, the number of errors is reduced but still exists. The repair output is in the attachment repair-210508-111358.log, and the service log is in mongod-210508-111358.log
|
|
Hi lyzenghq@gmail.com,
I've attached a repair attempt of the files you provided as repair.tar.gz . Please extract these files, replace them in your $dbpath, and let us know if it resolves the issue.
Best,
Edwin
|
|
Use mongodb 4.4 to repair, still failed. The repair output is in the attachment repair-210507-101605.log
|
|
Hi lyzenghq@gmail.com,
MongoDB 3.6 reached end of life in April of 2021. But we can provide limited guidance on this issue.
First, make a complete copy of the database's $dbpath directory to safeguard so that you can work off of the current $dbpath. Then, try mongod --repair using the latest version of MongoDB.
Best,
Edwin
|
Generated at Thu Feb 08 05:39:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.