[SERVER-23603] Dropping Index in WT Corrupted Database Created: 07/Apr/16  Updated: 06/Dec/22  Resolved: 08/Feb/17

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: John Page Assignee: Backlog - Storage Execution Team
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Storage Execution
Operating System: ALL
Participants:

 Description   

Running WiredTiger with a large "2d" index on a co-ordinate pair on field p
Created a 2nd index on

{ p: "2d", d : 1}

as foreground.
Database is under lead quering and updating.
All good

Dropped the original index - database quit immediately.
Will not restart - log says

2016-04-07T16:22:48.349+0000 I CONTROL  [initandlisten] options: { config: "/data/MDBWorldOSM/automation-mongod.conf", net: { bindIp: "0.0.0.0", port: 27017 }, processManagement: { fork: true }, storage: { dbPath: "/data/MDBWorldOSM", engine: "wiredTiger", journal: { enabled: false }, wiredTiger: { collectionConfig: { blockCompressor: "zlib" } } }, systemLog: { destination: "file", path: "/data/mongod.log" } }
2016-04-07T16:22:48.370+0000 W -        [initandlisten] Detected unclean shutdown - /data/MDBWorldOSM/mongod.lock is not empty.
2016-04-07T16:22:48.370+0000 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2016-04-07T16:22:48.370+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=37G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),,log=(enabled=false),
2016-04-07T16:22:48.408+0000 I -        [initandlisten] Fatal assertion 28579 NoSuchKey: Unable to find metadata for table:index-0-3681633574245705750
2016-04-07T16:22:48.408+0000 I -        [initandlisten]

Database is WT runnign with nojournal - however as I understand it this can at worst lead to 60s of data loss/rollback.

Database available for inspection in-situ on AWS.



 Comments   
Comment by Alexander Gorrod [ 08/Feb/17 ]

john.l.page@mongodb.com Without more specific steps it would be very difficult to reproduce this failure. I'm going to close the ticket for now. If you encounter the problem again please reopen.

Comment by John Page [ 19/Jan/17 ]

This was nearly a year ago so I have no idea exactly what version it was, should be easy to reproduce to test though.

Comment by Michael Cahill (Inactive) [ 11/Apr/16 ]

john.page, what version of MongoDB are you running? There were several durability bugs fixed in SERVER-19274 and related tickets.

Comment by John Page [ 07/Apr/16 ]

My Guess is that the _mdb_catalog.wt has reverted to when the index existed - but it no longer does.
I removed 'old' gzipped logs and now don't have one showing why it crashed when I dropped the index.
Is there a recovery from this as that database will take a lot of rebuilding.

Generated at Thu Feb 08 04:03:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.