[SERVER-37844] 删除数据目录dbpath中某个数据目录后,mongodb启不起来 Created: 31/Oct/18  Updated: 01/Nov/18  Resolved: 31/Oct/18

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0 Required
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: william.wu Assignee: Danny Hatcher (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.turtle     File WiredTiger.wt     File master.conf    
Participants:

 Description   

不小心删除(rm)了dbpath中的local目录后,启动mongod时报错:

2018-10-30T18:06:58.996+0800 I CONTROL ***** SERVER RESTARTED *****
2018-10-30T18:06:59.065+0800 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=31G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journ
al,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2018-10-30T18:06:59.115+0800 E STORAGE [initandlisten] WiredTiger (2) [1540894019:115325][43966:0x7fe42fdc2c80], file:local/collection/0--451989828212995074.wt, session.open_cursor: /data/master-mongod/d
b/local/collection/0--451989828212995074.wt: No such file or directory
2018-10-30T18:06:59.115+0800 E STORAGE [initandlisten] no cursor for uri: table:local/collection/0--451989828212995074
2018-10-30T18:06:59.115+0800 I - [initandlisten] Invariant failure c src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp 1020
2018-10-30T18:06:59.129+0800 I CONTROL [initandlisten] 
 0xf5c569 0xefb431 0xedfee2 0xd754f1 0xd7560c 0xd75f43 0xd76251 0xd70382 0xcf2896 0xcf5731 0xd6f2d6 0xa7dd5d 0x808872 0x7d6bc9 0x7fe42e37baf5 0x806689
----- BEGIN BACKTRACE -----
{"backtrace":[\{"b":"400000","o":"B5C569"},\{"b":"400000","o":"AFB431"},\{"b":"400000","o":"ADFEE2"},\{"b":"400000","o":"9754F1"},\{"b":"400000","o":"97560C"},\{"b":"400000","o":"975F43"},{"b":"400000","o":"976
251"},\{"b":"400000","o":"970382"},\{"b":"400000","o":"8F2896"},\{"b":"400000","o":"8F5731"},\{"b":"400000","o":"96F2D6"},\{"b":"400000","o":"67DD5D"},\{"b":"400000","o":"408872"},\{"b":"400000","o":"3D6BC9"},{"
b":"7FE42E35A000","o":"21AF5"},\{"b":"400000","o":"406689"}],"processInfo":{ "mongodbVersion" : "3.0.6", "gitVersion" : "1ef45a23a4c5e3480ac919b28afcba3c615488f2", "uname" : { "sysname" : "Linux", "release
" : "3.10.0-229.1.2.el7.x86_64", "version" : "#1 SMP Fri Mar 27 03:04:26 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "2C8EE1A0F536BE67BDBA4481FE55CEEE8AA469
50" }, \{ "b" : "7FFF192FE000", "elfType" : 3, "buildId" : "64DE62EAA6D0191EAD9358297D64406988D7ED66" }, { "b" : "7FE42F99A000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "12F30315D4F4A
2FE58B1977405C8B5515861E66B" }, \{ "b" : "7FE42F72D000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "B54FE20525AE27B81127E04A2B006FD758E42E55" }, { "b" : "7FE42F346000", "path" : "/lib64/li
bcrypto.so.10", "elfType" : 3, "buildId" : "D3ED02D380B3CDCF52EC6E23DD35CDF03B6E046A" }, { "b" : "7FE42F13E000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "7376A07360DC57189A8F92B20AA4AA1CA
EA80551" }, \{ "b" : "7FE42EF3A000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "4DFEE4EA9AE8FDD4C71BD4CCC0727222F19DF810" }, { "b" : "7FE42EC33000", "path" : "/lib64/libstdc++.so.6", "elfTyp
e" : 3, "buildId" : "405EACD649720B8668FFBBA197CBF030A7EF6296" }, \{ "b" : "7FE42E931000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "A1AA62B29765BE03A36BF927B047EEEF8696EEC6" }, { "b" : "7FE
42E71B000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "5D3D7256AE68BCFF41E312A24825ED80ECA88A73" }, { "b" : "7FE42E35A000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "C31FF
E7942BFD77B2FCA8F9BD5709D387A86D3BC" }, \{ "b" : "7FE42FBB6000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9866E1D2BA61EBB4CE4F009FACDAACC24EF3B804" }, { "b" : "7FE42E10E000", "pa
th" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "34672D541C8C9C5C1C25CB4F3F332CC9D3E604AD" }, { "b" : "7FE42DE2B000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "45CB7F6CD32
2F5B55FF8B635F7EC1578631CCAEA" }, \{ "b" : "7FE42DC27000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7FE42D9F5000", "path" : "/lib
64/libk5crypto.so.3", "elfType" : 3, "buildId" : "23A2D854538903E2B84EF0882046DD95522C8B59" }, { "b" : "7FE42D7DF000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "E45643F27F3B3E960F3691AFC6EC
27A98EF7B46B" }, \{ "b" : "7FE42D5D0000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "F4A3D5E7E23F871751CA8F250421F8CF83447AD2" }, { "b" : "7FE42D3CC000", "path" : "/lib64/libkeyutil
s.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FE42D1B2000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "AC596E865AF0D14B10F7B707F47D2031AD6D
68DC" }, \{ "b" : "7FE42CF8D000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "82FF6B18E1E42825CC2D060F969479AD4AF2F62C" }, { "b" : "7FE42CD2C000", "path" : "/lib64/libpcre.so.1", "elfTyp
e" : 3, "buildId" : "AE64AA461A26E01F60408013D361749D56DD0AE1" }, \{ "b" : "7FE42CB07000", "path" : "/lib64/liblzma.so.5", "elfType" : 3, "buildId" : "218D03D1F6CF1A099A4D467B5E8ECF4F2BF45750" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5c569]
 mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefb431]
 mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0xedfee2]
 mongod(_ZN5mongo21WiredTigerRecordStore8Iterator7_locateERKNS_8RecordIdEb+0x151) [0xd754f1]
 mongod(_ZN5mongo21WiredTigerRecordStore8IteratorC2ERKS0_PNS_16OperationContextERKNS_8RecordIdERKNS_20CollectionScanParams9DirectionEb+0x9C) [0xd7560c]
 mongod(_ZNK5mongo21WiredTigerRecordStore11getIteratorEPNS_16OperationContextERKNS_8RecordIdERKNS_20CollectionScanParams9DirectionE+0x43) [0xd75f43]
 mongod(_ZN5mongo21WiredTigerRecordStoreC1EPNS_16OperationContextERKNS_10StringDataES5_bllPNS_28CappedDocumentDeleteCallbackEPNS_20WiredTigerSizeStorerE+0x2C1) [0xd76251]
 mongod(_ZN5mongo18WiredTigerKVEngine14getRecordStoreEPNS_16OperationContextERKNS_10StringDataES5_RKNS_17CollectionOptionsE+0x132) [0xd70382]
 mongod(_ZN5mongo22KVDatabaseCatalogEntry14initCollectionEPNS_16OperationContextERKSsb+0x276) [0xcf2896]
 mongod(_ZN5mongo15KVStorageEngineC1EPNS_8KVEngineERKNS_22KVStorageEngineOptionsE+0x5E1) [0xcf5731]
 mongod(+0x96F2D6) [0xd6f2d6]
 mongod(_ZN5mongo23GlobalEnvironmentMongoD22setGlobalStorageEngineERKSs+0x30D) [0xa7dd5d]
 mongod(_ZN5mongo13initAndListenEi+0x422) [0x808872]
 mongod(main+0x139) [0x7d6bc9]
 libc.so.6(__libc_start_main+0xF5) [0x7fe42e37baf5]
 mongod(+0x406689) [0x806689]
----- END BACKTRACE -----
2018-10-30T18:06:59.129+0800 I - [initandlisten]
 
***aborting after invariant() failure

 

其他的数据(44T大小)实在是太重要了,希望你能有办法帮我启动mongodb

经过我自己的调研,我认为是WiredTiger.wt文件和WiredTiger.turtle 在我删除了local目录后导致在启动WiredTiger引擎时,验证相关信息时不匹配,但是我不知道如何修改这两个文件,或者我还需要提供什么信息



 Comments   
Comment by william.wu [ 01/Nov/18 ]

Hi, Daniel,

Thanks for your replay. I am very sad to hear this, For me, it will be an unforgettable lesson.

I had to try use the wt utility to dump remain data to new files, then load them to a new mongod instance. But I can only recover Part of the collections(.wt files in dbpath) for some is too large, and my disk capacity is not enough for its dump file. So I submited this issue to ask for another way to recover data.

Thanks a lot.

William

Comment by Danny Hatcher (Inactive) [ 31/Oct/18 ]

Hello William,

Unfortunately, we do not have any automated tools to recover from cases like this. We strongly recommend never deleting files or folders in your dbpath without ensuring that it has been backed up to an alternate location.

Thank you,

Danny

Comment by william.wu [ 31/Oct/18 ]

I have rm local dir in dbpath, after this , I can not start mongodb

but, the data remain in dbpath is very important to me , and is very large ,which has 44T ,

I think it is my WiredTiger.turtle and WiredTiger.wt need to be change, but I don not know how to do it.

so, I need you help, please

Generated at Thu Feb 08 04:47:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.