[SERVER-47102] WiredTiger.wt: encountered an illegal file format or internal value Created: 25/Mar/20  Updated: 30/Mar/20  Resolved: 26/Mar/20

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 4.0.0
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: zhen wang Assignee: Carl Champain (Inactive)
Resolution: Done Votes: 0
Labels: envh, repair, rpo, rps, trcf, wtc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux


Attachments: File repair_attempt.tar.gz     Zip Archive wiredTigerFiles.zip    
Issue Links:
Duplicate
is duplicated by SERVER-47101 WiredTiger.wt: encountered an illegal... Closed
Participants:

 Description   

2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] MongoDB starting : pid=16951 port=27017 dbpath=/app/mongodb/data/db 64-bit host=yfweb
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] db version v4.0.0
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] git version: 3b07af3d4f471ae89e8186d33bbb1d5259597d51
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] allocator: tcmalloc
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] modules: none
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] build environment:
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] distmod: rhel70
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] distarch: x86_64
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] target_arch: x86_64
2020-03-25T09:53:41.515+0000 I CONTROL [initandlisten] options: { config: "mongodb.conf", net:

{ bindIp: "0.0.0.0", port: 27017 }

, processManagement: { fork: true }, repair: true, storage: { dbPath: "/app/mongodb/data/db" }, systemLog: { destination: "file", path: "/app/mongodb/logs/mongodb.log" } }
2020-03-25T09:53:41.515+0000 I STORAGE [initandlisten] Detected data files in /app/mongodb/data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-03-25T09:53:41.515+0000 I STORAGE [initandlisten]
2020-03-25T09:53:41.515+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-03-25T09:53:41.515+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-03-25T09:53:41.515+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3480M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2020-03-25T09:53:42.327+0000 E STORAGE [initandlisten] WiredTiger error (0) [1585130022:327046][16951:0x7f0a3ff2da60], file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value: (_wt_turtle_read, 293) Raw: [1585130022:327046][16951:0x7f0a3ff2da60], file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value: (_wt_turtle_read, 293)
2020-03-25T09:53:42.327+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1585130022:327105][16951:0x7f0a3ff2da60], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1585130022:327105][16951:0x7f0a3ff2da60], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic
2020-03-25T09:53:42.327+0000 F - [initandlisten] Fatal Assertion 50853 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 403
2020-03-25T09:53:42.327+0000 F - [initandlisten]

***aborting after fassert() failure

  • 2020-03-25T09:53:42.350+0000 F - [initandlisten] Got signal: 6 (Aborted).
    0x55d1f738e331 0x55d1f738d549 0x55d1f738da2d 0x7f0a3e7b35e0 0x7f0a3e44a3be 0x7f0a3e44b805 0x55d1f5a04c89 0x55d1f5af66a6 0x55d1f5b645b1 0x55d1f598faa1 0x55d1f598fdca 0x55d1f5b2a163 0x55d1f5b28664 0x55d1f5b094aa 0x55d1f5b62e2d 0x55d1f5b6340a 0x55d1f5b6367c 0x55d1f5bd11f6 0x55d1f5b5e9f8 0x55d1f5b27bae 0x55d1f5b27c8b 0x55d1f5b080e2 0x55d1f5ad541c 0x55d1f5ab9374 0x55d1f60e46eb 0x55d1f59889c5 0x55d1f5a6e604 0x55d1f5a06819 0x7f0a3e4370c0 0x55d1f5a6d0af

          • BEGIN BACKTRACE ----- {"backtrace":[\{"b":"55D1F500C000","o":"2382331","s":"_ZN5mongo15printStackTraceERSo"}

            ,{"b":"55D1F500C000","o":"2381549"},{"b":"55D1F500C000","o":"2381A2D"},{"b":"7F0A3E7A4000","o":"F5E0"},{"b":"7F0A3E418000","o":"323BE","s":"gsignal"},{"b":"7F0A3E418000","o":"33805","s":"abort"},{"b":"55D1F500C000","o":"9F8C89","s":"ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"55D1F500C000","o":"AEA6A6"},{"b":"55D1F500C000","o":"B585B1"},{"b":"55D1F500C000","o":"983AA1","s":"wt_err"},{"b":"55D1F500C000","o":"983DCA","s":"wt_panic"},{"b":"55D1F500C000","o":"B1E163","s":"wt_turtle_read"},{"b":"55D1F500C000","o":"B1C664","s":"wt_metadata_search"},{"b":"55D1F500C000","o":"AFD4AA","s":"wt_conn_dhandle_open"},{"b":"55D1F500C000","o":"B56E2D","s":"wt_session_get_dhandle"},{"b":"55D1F500C000","o":"B5740A","s":"wt_session_get_dhandle"},{"b":"55D1F500C000","o":"B5767C","s":"wt_session_get_btree_ckpt"},{"b":"55D1F500C000","o":"BC51F6","s":"wt_curfile_open"},{"b":"55D1F500C000","o":"B529F8"},{"b":"55D1F500C000","o":"B1BBAE","s":"wt_metadata_cursor_open"},{"b":"55D1F500C000","o":"B1BC8B","s":"wt_metadata_cursor"},{"b":"55D1F500C000","o":"AFC0E2","s":"wiredtiger_open"},{"b":"55D1F500C000","o":"AC941C","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"55D1F500C000","o":"AAD374"},{"b":"55D1F500C000","o":"10D86EB","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE"},{"b":"55D1F500C000","o":"97C9C5"},{"b":"55D1F500C000","o":"A62604","s":"_ZN5mongo11mongoDbMainEiPPcS1"},{"b":"55D1F500C000","o":"9FA819","s":"main"},{"b":"7F0A3E418000","o":"1F0C0","s":"__libc_start_main"},{"b":"55D1F500C000","o":"A610AF"}],"processInfo":{ "mongodbVersion" : "4.0.0", "gitVersion" : "3b07af3d4f471ae89e8186d33bbb1d5259597d51", "compiledModules" : [], "uname" :

            { "sysname" : "Linux", "release" : "4.4.180-1.el6.elrepo.x86_64", "version" : "#1 SMP Thu May 16 16:37:14 EDT 2019", "machine" : "x86_64" }

            , "somap" : [ { "b" : "55D1F500C000", "elfType" : 3, "buildId" : "596DECA05DB9B54F6E765AEC1A8AADE4665E8946" }, { "b" : "7FFEAA597000", "elfType" : 3, "buildId" : "471A8DF32A9F1908B1974DA8DAFB158FCBBDFC76" }, { "b" : "7ED4766E1000", "path" : "/usr/lib64/libcurl.so.4", "elfType" : 3, "buildId" : "A38B9CE8AEAF277CBD8BC1298B1731E2C9A66192" }, { "b" : "7F0A3F8CA000", "path" : "/opt/glibc-2.14/lib/libresolv.so.2", "elfType" : 3, "buildId" : "FA7431250D27BA688CA5B69076901C13077000F8" }, { "b" : "7ED46D8E5000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "CBDA444A7109874C5350AE9CEEF3F82F749B347F" }, { "b" : "7ED467A79000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "BECFB85A8BC084042D5BF2BA9E66325CE798B659" }, { "b" : "7F0A3F075000", "path" : "/opt/glibc-2.14/lib/libdl.so.2", "elfType" : 3, "buildId" : "D69A532E8B08784D52163137CA68F79000EEFAD8" }, { "b" : "7F0A3EE5A000", "path" : "/opt/glibc-2.14/lib/librt.so.1", "elfType" : 3, "buildId" : "01EB41696F517055116AB32AD62AE124C740AE42" }, { "b" : "7F0A3EBD7000", "path" : "/opt/glibc-2.14/lib/libm.so.6", "elfType" : 3, "buildId" : "5E3C92CA474CED089DD266B56412919B2A50F4AC" }, { "b" : "7ED4715C1000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "EDC925E58FE28DCA536993EB13179C739F1E6566" }, { "b" : "7F0A3E7A4000", "path" : "/opt/glibc-2.14/lib/libpthread.so.0", "elfType" : 3, "buildId" : "201FCBE70E81EB9B5B28CDBA5BB89346C72E8C19" }, { "b" : "7F0A3E418000", "path" : "/opt/glibc-2.14/lib/libc.so.6", "elfType" : 3, "buildId" : "17E07E976790E5BCC6CCB06F1123D140B7F2A06C" }, { "b" : "7ED479536000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "97AF4B77212F74CFF72B6C013E6AA2D74A97EF60" }, { "b" : "7ED4755E6000", "path" : "/lib64/libidn.so.11", "elfType" : 3, "buildId" : "417A6ED67D904CD799091B56BDFADA0D594BB094" }, { "b" : "7ED463F95000", "path" : "/lib64/libldap-2.4.so.2", "elfType" : 3, "buildId" : "1E1F213BB990982FA49BCDD2CBBFB2DB680019AC" }, { "b" : "7ED467551000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "9A737F8BF10FC99C37CC404D3FC188F6E11FEDD9" }, { "b" : "7ED467A6A000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "8D3D6E28DF6EB3752642A7031AAC17D39EA4265D" }, { "b" : "7ED467C3E000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "CC89B4C8CDCCD32BA610BC72784DC3B7E9BD9E19" }, { "b" : "7ED469E3A000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "7EC54D6E88BB7D2C1284117C2A483496A01EAAF4" }, { "b" : "7ED475824000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "5FA8E5038EC04A774AF72A9BB62DC86E1049C4D6" }, { "b" : "7ED465DD1000", "path" : "/usr/lib64/libssl3.so", "elfType" : 3, "buildId" : "7B31DCDBC6EA53E83B165E972066324EB11F3674" }, { "b" : "7ED4653A4000", "path" : "/usr/lib64/libsmime3.so", "elfType" : 3, "buildId" : "B0496DD0482E729BC33F5B19CFA52787389477A0" }, { "b" : "7ED467C5C000", "path" : "/usr/lib64/libnss3.so", "elfType" : 3, "buildId" : "B786AFA781039748E951FE1045E932BCF31900F0" }, { "b" : "7ED468A2D000", "path" : "/usr/lib64/libnssutil3.so", "elfType" : 3, "buildId" : "06AD65B988B6F761D5B334BF6AEE9CFED2D98AFF" }, { "b" : "7ED468029000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "CF853ECBE1D643A626D65701261A12ABE0828BF0" }, { "b" : "7ED468224000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "810CDF602A65C7AF9CC7EA989BAFA4702B7738BF" }, { "b" : "7ED4687E5000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "36DBF3933E1ABF4DD774F7B456B5F406B632AA2A" }, { "b" : "7ED4729BD000", "path" : "/usr/lib64/libssh2.so.1", "elfType" : 3, "buildId" : "8E1B3E6D72631F40825E105431067352A8565B94" }, { "b" : "7ED4627AE000", "path" : "/lib64/liblber-2.4.so.2", "elfType" : 3, "buildId" : "B250057251FC715C5CCA27489506D7BBB1C9137C" }, { "b" : "7ED463194000", "path" : "/usr/lib64/libsasl2.so.2", "elfType" : 3, "buildId" : "90F78E5E1AF40EE481268BEAF15F9A2E3E8EFBD2" }, { "b" : "7ED465789000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "E0C522C589F775C324330BE09CE67DC83950A213" }, { "b" : "7ED466D86000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "AF374BAFB7F5B139A0B431D3F06D82014AFF3251" }, { "b" : "7F0A3B74E000", "path" : "/opt/glibc-2.14/lib/libcrypt.so.1", "elfType" : 3, "buildId" : "DEDC46423407162FCAC0F4C24751237E07326854" }, { "b" : "7ED472D2F000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B4576BE308DDCF7BC31F7304E4734C3D846D0236" } ] }}
            mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55d1f738e331]
            mongod(+0x2381549) [0x55d1f738d549]
            mongod(+0x2381A2D) [0x55d1f738da2d]
            libpthread.so.0(+0xF5E0) [0x7f0a3e7b35e0]
            libc.so.6(gsignal+0x3E) [0x7f0a3e44a3be]
            libc.so.6(abort+0x175) [0x7f0a3e44b805]
            mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55d1f5a04c89]
            mongod(+0xAEA6A6) [0x55d1f5af66a6]
            mongod(+0xB585B1) [0x55d1f5b645b1]
            mongod(__wt_err+0x9D) [0x55d1f598faa1]
            mongod(__wt_panic+0x33) [0x55d1f598fdca]
            mongod(__wt_turtle_read+0x343) [0x55d1f5b2a163]
            mongod(__wt_metadata_search+0x1C4) [0x55d1f5b28664]
            mongod(__wt_conn_dhandle_open+0x8A) [0x55d1f5b094aa]
            mongod(__wt_session_get_dhandle+0xED) [0x55d1f5b62e2d]
            mongod(__wt_session_get_dhandle+0x6CA) [0x55d1f5b6340a]
            mongod(__wt_session_get_btree_ckpt+0x14C) [0x55d1f5b6367c]
            mongod(__wt_curfile_open+0x56) [0x55d1f5bd11f6]
            mongod(+0xB529F8) [0x55d1f5b5e9f8]
            mongod(__wt_metadata_cursor_open+0x6E) [0x55d1f5b27bae]
            mongod(__wt_metadata_cursor+0x4B) [0x55d1f5b27c8b]
            mongod(wiredtiger_open+0x1A12) [0x55d1f5b080e2]
            mongod(ZN5mongo18WiredTigerKVEngineC1ERKNSt7_cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x8FC) [0x55d1f5ad541c]
            mongod(+0xAAD374) [0x55d1f5ab9374]
            mongod(_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE+0xBEB) [0x55d1f60e46eb]
            mongod(+0x97C9C5) [0x55d1f59889c5]
            mongod(ZN5mongo11mongoDbMainEiPPcS1+0x874) [0x55d1f5a6e604]
            mongod(main+0x9) [0x55d1f5a06819]
            libc.so.6(__libc_start_main+0x100) [0x7f0a3e4370c0]
            mongod(+0xA610AF) [0x55d1f5a6d0af]

          • END BACKTRACE -----


 Comments   
Comment by zhen wang [ 30/Mar/20 ]

Hi carl.champain,

      Thank you for your suggestions, I will use a replica set  to start later.
      Kind regards,  
      Zhen

Comment by Carl Champain (Inactive) [ 26/Mar/20 ]

1278827329@qq.com,

Unfortunately we aren't able to repair this corruption.

To avoid a problem like in the future, it is our strong recommendation to:

I am going to close this ticket. 

Kind regards,
Carl
 

 

Comment by zhen wang [ 26/Mar/20 ]

Hi carl.champain,

      I tried the files you sent me, but the collection- * and index- * in the db folder in mongodb are deleted.  So the repair failed.

2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] MongoDB starting : pid=21246 port=27017 dbpath=/app/mongodb/data/db 64-bit host=yfweb
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] db version v4.0.0
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] git version: 3b07af3d4f471ae89e8186d33bbb1d5259597d51
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] allocator: tcmalloc
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] modules: none
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] build environment:
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] distmod: rhel70
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] distarch: x86_64
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] target_arch: x86_64
2020-03-26T10:48:40.663+0000 I CONTROL [initandlisten] options: { config: "mongodb.conf", net:

{ bindIp: "0.0.0.0", port: 27017 }

, processManagement: { fork: true }, repair: true, storage: { dbPath: "/app/mongodb/data/db" }, systemLog: { destination: "file", path: "/app/mongodb/logs/mongodb.log" } }
2020-03-26T10:48:40.663+0000 I STORAGE [initandlisten] Detected data files in /app/mongodb/data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-03-26T10:48:40.663+0000 I STORAGE [initandlisten]
2020-03-26T10:48:40.663+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-03-26T10:48:40.663+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-03-26T10:48:40.663+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3480M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2020-03-26T10:48:41.686+0000 I STORAGE [initandlisten] WiredTiger message [1585219721:686232][21246:0x7f9ab8c3ba60], txn-recover: Main recovery loop: starting at 1095/256
2020-03-26T10:48:41.686+0000 I STORAGE [initandlisten] WiredTiger message [1585219721:686613][21246:0x7f9ab8c3ba60], txn-recover: Recovering log 1095 through 1096
2020-03-26T10:48:41.768+0000 I STORAGE [initandlisten] WiredTiger message [1585219721:768163][21246:0x7f9ab8c3ba60], txn-recover: Recovering log 1096 through 1096
2020-03-26T10:48:41.826+0000 I STORAGE [initandlisten] WiredTiger message [1585219721:826860][21246:0x7f9ab8c3ba60], txn-recover: Set global recovery timestamp: 5d7894a400000001
2020-03-26T10:48:41.835+0000 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(1568183460, 1)
2020-03-26T10:48:41.835+0000 I STORAGE [initandlisten] Triggering the first stable checkpoint. Initial Data: Timestamp(1568183460, 1) PrevStable: Timestamp(0, 0) CurrStable: Timestamp(1568183460, 1)
2020-03-26T10:48:41.836+0000 I STORAGE [initandlisten] Repairing size cache
2020-03-26T10:48:41.853+0000 E STORAGE [initandlisten] WiredTiger error (-31802) [1585219721:853165][21246:0x7f9ab8c3ba60], file:sizeStorer.wt, WT_SESSION.verify: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error Raw: [1585219721:853165][21246:0x7f9ab8c3ba60], file:sizeStorer.wt, WT_SESSION.verify: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2020-03-26T10:48:41.853+0000 E STORAGE [initandlisten] WiredTiger error (0) [1585219721:853242][21246:0x7f9ab8c3ba60], file:sizeStorer.wt, WT_SESSION.verify: file ranges never verified: 1 Raw: [1585219721:853242][21246:0x7f9ab8c3ba60], file:sizeStorer.wt, WT_SESSION.verify: file ranges never verified: 1
2020-03-26T10:48:41.853+0000 I STORAGE [initandlisten] Verify failed on uri table:sizeStorer. Running a salvage operation.
2020-03-26T10:48:41.897+0000 I STORAGE [initandlisten] Repairing catalog metadata
2020-03-26T10:48:41.904+0000 E STORAGE [initandlisten] WiredTiger error (-31802) [1585219721:904309][21246:0x7f9ab8c3ba60], file:_mdb_catalog.wt, WT_SESSION.verify: /app/mongodb/data/db/_mdb_catalog.wt: handle-read: pread: failed to read 4096 bytes at offset 36864: WT_ERROR: non-specific WiredTiger error Raw: [1585219721:904309][21246:0x7f9ab8c3ba60], file:_mdb_catalog.wt, WT_SESSION.verify: /app/mongodb/data/db/_mdb_catalog.wt: handle-read: pread: failed to read 4096 bytes at offset 36864: WT_ERROR: non-specific WiredTiger error
2020-03-26T10:48:41.904+0000 I STORAGE [initandlisten] Verify failed on uri table:_mdb_catalog. Running a salvage operation.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 31818 processes, 65536 files. Number of processes should be at least 32768 : 0.5 times number of files.
2020-03-26T10:48:41.916+0000 I CONTROL [initandlisten]
2020-03-26T10:48:41.972+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0--3138757354455307334
2020-03-26T10:48:41.973+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0-1280430728660390383
2020-03-26T10:48:41.983+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0-6375416066776234219
2020-03-26T10:48:41.984+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0-6380925353307504982
2020-03-26T10:48:41.985+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0-6798498555873876114
2020-03-26T10:48:42.001+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-0-8553844172800445369
2020-03-26T10:48:42.001+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-10-2827412739017360918
2020-03-26T10:48:42.002+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-10-8553844172800445369
2020-03-26T10:48:42.003+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-11-6380925353307504982
2020-03-26T10:48:42.003+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-12--2805923774461609203
2020-03-26T10:48:42.024+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-12--5163129163982591919
2020-03-26T10:48:42.058+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-12-8553844172800445369
2020-03-26T10:48:42.059+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-13-6380925353307504982
2020-03-26T10:48:42.060+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-14--2805923774461609203
2020-03-26T10:48:42.061+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-16--2805923774461609203
2020-03-26T10:48:42.096+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-18--6472108667114390324
2020-03-26T10:48:42.115+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-2--3138757354455307334
2020-03-26T10:48:42.116+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-2-6380925353307504982
2020-03-26T10:48:42.137+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-2-6798498555873876114
2020-03-26T10:48:42.138+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-2-8553844172800445369
2020-03-26T10:48:42.138+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-20--6472108667114390324
2020-03-26T10:48:42.148+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-22--2805923774461609203
2020-03-26T10:48:42.149+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-22--5163129163982591919
2020-03-26T10:48:42.279+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-22--6472108667114390324
2020-03-26T10:48:42.283+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-24--2805923774461609203
2020-03-26T10:48:42.284+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-24--536915609928663585
2020-03-26T10:48:42.321+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-26--5163129163982591919
2020-03-26T10:48:42.326+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-26--6472108667114390324
2020-03-26T10:48:42.327+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-30--536915609928663585
2020-03-26T10:48:42.555+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-32--6472108667114390324
2020-03-26T10:48:42.725+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-4-6380925353307504982
2020-03-26T10:48:42.725+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-4-6798498555873876114
2020-03-26T10:48:42.726+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-4-8553844172800445369
2020-03-26T10:48:42.727+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-44--6472108667114390324
2020-03-26T10:48:42.826+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-6--6165242803416714748
2020-03-26T10:48:42.826+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-6-6380925353307504982
2020-03-26T10:48:42.902+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-6-8553844172800445369
2020-03-26T10:48:42.903+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-7-6380925353307504982
2020-03-26T10:48:42.904+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-8-2827412739017360918
2020-03-26T10:48:43.012+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-8-8553844172800445369
2020-03-26T10:48:43.014+0000 I STORAGE [initandlisten] Dropping unknown ident: collection-9-6380925353307504982
2020-03-26T10:48:43.014+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1--3138757354455307334
2020-03-26T10:48:43.085+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1-1280430728660390383
2020-03-26T10:48:43.086+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1-6375416066776234219
2020-03-26T10:48:43.086+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1-6380925353307504982
2020-03-26T10:48:43.087+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1-6798498555873876114
2020-03-26T10:48:43.087+0000 I STORAGE [initandlisten] Dropping unknown ident: index-1-8553844172800445369
2020-03-26T10:48:43.088+0000 I STORAGE [initandlisten] Dropping unknown ident: index-10-6380925353307504982
2020-03-26T10:48:43.088+0000 I STORAGE [initandlisten] Dropping unknown ident: index-11-2827412739017360918
2020-03-26T10:48:43.089+0000 I STORAGE [initandlisten] Dropping unknown ident: index-11-8553844172800445369
2020-03-26T10:48:43.090+0000 I STORAGE [initandlisten] Dropping unknown ident: index-12-6380925353307504982
2020-03-26T10:48:43.090+0000 I STORAGE [initandlisten] Dropping unknown ident: index-13--2805923774461609203
2020-03-26T10:48:43.091+0000 I STORAGE [initandlisten] Dropping unknown ident: index-13--5163129163982591919
2020-03-26T10:48:43.092+0000 I STORAGE [initandlisten] Dropping unknown ident: index-13-8553844172800445369
2020-03-26T10:48:43.092+0000 I STORAGE [initandlisten] Dropping unknown ident: index-14-6380925353307504982
2020-03-26T10:48:43.093+0000 I STORAGE [initandlisten] Dropping unknown ident: index-15--2805923774461609203
2020-03-26T10:48:43.094+0000 I STORAGE [initandlisten] Dropping unknown ident: index-17--2805923774461609203
2020-03-26T10:48:43.095+0000 I STORAGE [initandlisten] Dropping unknown ident: index-19--6472108667114390324
2020-03-26T10:48:43.098+0000 I STORAGE [initandlisten] Dropping unknown ident: index-2-1280430728660390383
2020-03-26T10:48:43.098+0000 I STORAGE [initandlisten] Dropping unknown ident: index-21--6472108667114390324
2020-03-26T10:48:43.098+0000 I STORAGE [initandlisten] Dropping unknown ident: index-23--2805923774461609203
2020-03-26T10:48:43.099+0000 I STORAGE [initandlisten] Dropping unknown ident: index-23--5163129163982591919
2020-03-26T10:48:43.175+0000 I STORAGE [initandlisten] Dropping unknown ident: index-23--6472108667114390324
2020-03-26T10:48:43.178+0000 I STORAGE [initandlisten] Dropping unknown ident: index-25--2805923774461609203
2020-03-26T10:48:43.179+0000 I STORAGE [initandlisten] Dropping unknown ident: index-25--536915609928663585
2020-03-26T10:48:43.179+0000 I STORAGE [initandlisten] Dropping unknown ident: index-27--5163129163982591919
2020-03-26T10:48:43.202+0000 I STORAGE [initandlisten] Dropping unknown ident: index-27--6472108667114390324
2020-03-26T10:48:43.203+0000 I STORAGE [initandlisten] Dropping unknown ident: index-3--3138757354455307334
2020-03-26T10:48:43.203+0000 I STORAGE [initandlisten] Dropping unknown ident: index-3-6380925353307504982
2020-03-26T10:48:43.204+0000 I STORAGE [initandlisten] Dropping unknown ident: index-3-6798498555873876114
2020-03-26T10:48:43.204+0000 I STORAGE [initandlisten] Dropping unknown ident: index-3-8553844172800445369
2020-03-26T10:48:43.205+0000 I STORAGE [initandlisten] Dropping unknown ident: index-31--536915609928663585
2020-03-26T10:48:43.257+0000 I STORAGE [initandlisten] Dropping unknown ident: index-33--6472108667114390324
2020-03-26T10:48:43.261+0000 I STORAGE [initandlisten] Dropping unknown ident: index-45--6472108667114390324
2020-03-26T10:48:43.264+0000 I STORAGE [initandlisten] Dropping unknown ident: index-5-6380925353307504982
2020-03-26T10:48:43.264+0000 I STORAGE [initandlisten] Dropping unknown ident: index-5-6798498555873876114
2020-03-26T10:48:43.267+0000 I STORAGE [initandlisten] Dropping unknown ident: index-5-8553844172800445369
2020-03-26T10:48:43.268+0000 I STORAGE [initandlisten] Dropping unknown ident: index-6-6798498555873876114
2020-03-26T10:48:43.268+0000 I STORAGE [initandlisten] Dropping unknown ident: index-7--6165242803416714748
2020-03-26T10:48:43.269+0000 I STORAGE [initandlisten] Dropping unknown ident: index-7-8553844172800445369
2020-03-26T10:48:43.269+0000 I STORAGE [initandlisten] Dropping unknown ident: index-8-6380925353307504982
2020-03-26T10:48:43.270+0000 I STORAGE [initandlisten] Dropping unknown ident: index-9-2827412739017360918
2020-03-26T10:48:43.290+0000 I STORAGE [initandlisten] Dropping unknown ident: index-9-8553844172800445369
2020-03-26T10:48:43.290+0000 F - [initandlisten] Fatal assertion 40593 UnrecoverableRollbackError: Expected collection does not exist. Collection: admin.system.version Ident: collection-0-7862889373208688964 at src/mongo/db/repair_database_and_check_version.cpp 245
2020-03-26T10:48:43.290+0000 F - [initandlisten]

***aborting after fassert() failure

2020-03-26T10:48:43.300+0000 F - [initandlisten] Got signal: 6 (Aborted).
0x561a012e0331 0x561a012df549 0x561a012dfa2d 0x7f9ab74c15e0 0x7f9ab71583be 0x7f9ab7159805 0x5619ff956f61 0x5619ffd5d566 0x5619ffd5fa9b 0x5619ff8db699 0x5619ff9c0604 0x5619ff958819 0x7f9ab71450c0 0x5619ff9bf0af

Comment by Carl Champain (Inactive) [ 25/Mar/20 ]

Hi 1278827329@qq.com,

I've attached a repair attempt of the files you provided as repair_attempt.tar.gz . Please extract these files, replace them in your $dbpath, and let us know if it resolves the issue.

Thanks,
Carl
 

Generated at Thu Feb 08 05:13:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.