[SERVER-24435] I download mongodb backup from s3. WT_NOTFOUND: item not found, terminating Created: 07/Jun/16  Updated: 31/May/17  Resolved: 17/Apr/17

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 3.2.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: aedanlee Assignee: Kelsey Schubert
Resolution: Incomplete Votes: 0
Labels: WT_NOTFOUND:
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux


Attachments: PNG File 1.png     File WiredTiger.turtle     File WiredTiger.wt     PNG File screenshot-1.png     PNG File screenshot-2.png    
Operating System: Linux
Participants:

 Description   

[ec2-user@ip-172-31-55-219 baby]$ mongod  --directoryperdb --dbpath ./db
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] MongoDB starting : pid=29020 port=27017 dbpath=./db 64-bit host=ip-172-31-55-219
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] db version v3.2.6
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] git version: 05552b562c7a0b3143a729aaa0838e558dc49b25
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.0-fips 29 Mar 2010
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] modules: none
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] build environment:
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten]     distmod: amazon
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten]     distarch: x86_64
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2016-06-07T13:42:03.291+0000 I CONTROL  [initandlisten] options: { storage: { dbPath: "./db", directoryPerDB: true } }
2016-06-07T13:42:03.312+0000 I -        [initandlisten] Detected data files in ./db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-06-07T13:42:03.312+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-06-07T13:42:03.327+0000 E STORAGE  [initandlisten] WiredTiger (-31803) [1465306923:327233][29020:0x7fae5728ad80], txn-recover: Recovery failed: WT_NOTFOUND: item not found
2016-06-07T13:42:03.329+0000 I -        [initandlisten] Assertion: 28595:-31803: WT_NOTFOUND: item not found
2016-06-07T13:42:03.330+0000 I STORAGE  [initandlisten] exception in initAndListen: 28595 -31803: WT_NOTFOUND: item not found, terminating
2016-06-07T13:42:03.330+0000 I CONTROL  [initandlisten] dbexit:  rc: 100



 Comments   
Comment by Ramon Fernandez Marina [ 17/Apr/17 ]

kasperpro, it's always best to open a separate ticket, can you please do so?

Thanks,
Ramón.

Comment by Andrew [ 17/Apr/17 ]

Hi, Thomas,
Actually I have the same issue. Will try to explain what happening.
My server broke down and all I had is copy of database files in /data/db. No proper mongodump, only files.
After restoring my VPS from snapshot, I tried to run mongo, get error (see in attachment)

Answering to your question:
1. Which version of MongoDB was the original mongod using?
Original was 3.2.4. However, I already updated to 3.4.3 with hope that this will be fixed.

2. What method do you use to create backups?
That is the problem, that I didnt have mongodumps. Only *.wt files.

3. Have you ever manipulated (copied or moved) the underlying database files? If so, was the mongod running?
No I didnt touch files. After restoring from snapshot, server state was same as before server fault.

Please find all screenshots and WiredTiger.wt/ WiredTiger.turtle files.
Is it any chance to restore from unclean shutdown only with *.wt files.?

Comment by Kelsey Schubert [ 01/Aug/16 ]

Hi aedanlee,

We haven’t heard back from you for some time, so I’m going to mark this ticket as resolved. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Regards,
Thomas

Comment by Kelsey Schubert [ 21/Jun/16 ]

Hi aedanlee,

We still need the information I requested to continue to investigate. If this is still an issue for you, would you please provide the answers to my questions and attach the files to this ticket?

Thank you,
Thomas

Comment by Kelsey Schubert [ 07/Jun/16 ]

Hi aedanlee,

To get a better understanding of what is going on here, I have a number of questions about mongod that was backed up.

  1. Which version of MongoDB was the original mongod using?
  2. What method do you use to create backups?
  3. Have you ever manipulated (copied or moved) the underlying database files? If so, was the mongod running?
  4. What kind of underlying storage mechanism are you using? Are the storage devices attached locally or over the network? Are the disks SSDs or HDDs? What kind of RAID and/or volume management system are you using?
  5. Have you ever restored this instance from backups?

We may also need the following information concerning your current data files:

  • an upload of WiredTiger.wt
  • an upload of WiredTiger.turtle
  • output of ls -l against the journal directory

Thank you,
Thomas

Generated at Thu Feb 08 04:06:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.