[SERVER-32451] Cannot start mongod with a missing wiredTiger database Created: 22/Dec/17  Updated: 26/Jan/18  Resolved: 22/Dec/17

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.2.17
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Greg Assignee: Kelsey Schubert
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File WiredTiger.wt     Text File dump.txt    
Issue Links:
Duplicate
duplicates SERVER-19815 Improved mongod --repair option for W... Closed
Operating System: ALL
Steps To Reproduce:

(In a clean, empty directory):
mongod --dbpath . --storageEngine wiredTiger --directoryperdb
(In another console):
mongo
>use tmp
>db.tmp.insert(

{'a':4}

)

Then, disconnect and stop the mongod process.

Remove the folder "tmp" and its contents in the dbpath location.

Restart mongod (same command as before)

Participants:

 Description   

I cannot restart the mongo server when a wiredtiger file is missing. Ideally, I'd like to drop/ignore this database allowing me to access any others.

2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] MongoDB starting : pid=30787 port=27018 dbpath=. 64-bit host=greg-laptop
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] db version v3.2.17
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] git version: 186656d79574f7dfe0831a7e7821292ab380f667
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] allocator: tcmalloc
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] modules: none
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] build environment:
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] distmod: ubuntu1604
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] distarch: x86_64
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] target_arch: x86_64
2017-12-21T17:14:15.690-0800 I CONTROL [initandlisten] options: { net:

{ port: 27018 }

, storage:

{ dbPath: ".", directoryPerDB: true, engine: "wiredTiger" }

}
2017-12-21T17:14:15.708-0800 W - [initandlisten] Detected unclean shutdown - ./mongod.lock is not empty.
2017-12-21T17:14:15.708-0800 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
2017-12-21T17:14:15.708-0800 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=18G,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-12-21T17:14:16.230-0800 E STORAGE [initandlisten] WiredTiger (2) [1513905256:230271][30787:0x7fd6533e5c80], file:tmp/collection-2-8425982379140367284.wt, WT_SESSION.open_cursor: ./tmp/collection-2-8425982379140367284.wt: handle-open: open: No such file or directory
2017-12-21T17:14:16.230-0800 E STORAGE [initandlisten] no cursor for uri: table:tmp/collection-2-8425982379140367284
2017-12-21T17:14:16.230-0800 F - [initandlisten] Invalid access at address: 0x58
2017-12-21T17:14:16.237-0800 F - [initandlisten] Got signal: 11 (Segmentation fault).



 Comments   
Comment by Greg [ 22/Dec/17 ]

For future googlers tearing their hair out. I managed to recover from this...

Build and install wiredtiger: http://source.wiredtiger.com/2.6.0/build-posix.html

In the mongodb dir:
$ wiredtiger/wt -v -h . -C "extensions=[wiredtiger/ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R list
to get the collection names that its looking for. In this case:
table:MassDB_compil912/collection-24-7867037548174310433 and table:MassDB_compil912/collection-28-7867037548174310433

Copy over .wt files from somewhere else and change the name to match these names, with a .wt extension at the end. I made a new empty db in another mongod, and then copied over these empty files

$ wiredtiger/wt -v -h . -C "extensions=[wiredtiger/ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R salvage table:MassDB_compil912/collection-24-7867037548174310433
$ wiredtiger/wt -v -h . -C "extensions=[wiredtiger/ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R salvage table:MassDB_compil912/collection-28-7867037548174310433

Then restarted mongod and dropped the dbs.

Thanks in part to http://www.alexbevi.com/blog/2016/02/10/recovering-a-wiredtiger-collection-from-a-corrupt-mongodb-installation/ as I'd never have figure out this snappy extension shit...

Comment by Greg [ 22/Dec/17 ]

Thanks Kelsey,
I see this ticket was created more than two years ago. So to be clear, there is no way to recover from this and my only option is to start fresh? Is there a way to backup any other existing databases? I cannot use mongodump because I cannot start the mongod process, but I still have all of the other (non-corrupted) .wt files from the other databases...

Comment by Kelsey Schubert [ 22/Dec/17 ]

Hi stuppie,

Thanks for the report. This improvement is tracked in SERVER-19815. Please feel free to vote for it and watch it for updates.

Kind regards,
Kelsey

Generated at Thu Feb 08 04:30:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.