[SERVER-33161] Postpone WiredTigerKVEngine table drops until the _mdb_catalog change is checkpointed Created: 07/Feb/18  Updated: 29/Jan/19  Resolved: 29/Jan/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Xiangyu Yao (Inactive)
Resolution: Duplicate Votes: 0
Labels: nyc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-38910 Remove redundant rollback handling on... Closed
Duplicate
duplicates SERVER-38962 The second phase of two-phase drop sh... Closed
Related
related to SERVER-39007 Switch to use rhel62-large distro for... Closed
related to SERVER-29213 Have KVWiredTigerEngine implement Sto... Closed
is related to SERVER-33159 RTT storage recovery from unclean shu... Closed
is related to SERVER-38962 The second phase of two-phase drop sh... Closed
is related to SERVER-38985 assume all tables exist in storage en... Closed
Sprint: Storage NYC 2019-02-11
Participants:
Linked BF Score: 0

 Description   

Some prerequisite knowledge is documented in SERVER-33159.

Recover to a stable timestamp introduces a case, in crash recovery, where an `_mdb_catalog` collection document may exist without a corresponding WT table. SERVER-33159 describes how that can happen.

SERVER-33159 is a short-term fix that accepts this state can exist. This ticket is to return the happens-before relationship between catalog entries and their on-disk WT tables, in the face of stable timestamp + logging changes for 4.0+ replica set members.

This patch is for glue code to postpone performing the WT table drop until it is sure the collection drop has been persisted in a stable checkpoint.



 Comments   
Comment by Xiangyu Yao (Inactive) [ 29/Jan/19 ]

Closing this as a duplicate of SERVER-38962.

Comment by Dianna Hohensee (Inactive) [ 15/Jan/19 ]

This was reverted because there must be in-memory storage engine handling. The in-mem engine does not take persisted checkpoints, so the idents would never be dropped. Perhaps oldest_timestamp is required for in-memory – I can't quite remember what we do for recover to a stable timestamp with the in-memory storage engine, I think the oplog history might be truncated based on oldest_timestamp or something.

Comment by Githook User [ 15/Jan/19 ]

Author:

{'username': 'DiannaHohensee', 'email': 'dianna.hohensee@10gen.com', 'name': 'Dianna Hohensee'}

Message: Revert "SERVER-33161 make the second phase of index two-phase drop occur when drop reaches checkpointed instead of oldest"

This reverts commit 2f67f3c66271e724b48afa2db88e8b6c3317f6ab.
Branch: master
https://github.com/mongodb/mongo/commit/a5921570e5eb22cbacbe80d9ee153a2257b15bd1

Comment by Benety Goh [ 14/Jan/19 ]

The remaining work is described in SERVER-38985

Comment by Daniel Gottlieb (Inactive) [ 11/Jan/19 ]

Re-opening to note there are still some changes worth doing, notably removing the short-term fix introduced by SERVER-33159

Comment by Githook User [ 11/Jan/19 ]

Author:

{'username': 'DiannaHohensee', 'email': 'dianna.hohensee@10gen.com', 'name': 'Dianna Hohensee'}

Message: SERVER-33161 make the second phase of index two-phase drop occur when drop reaches checkpointed instead of oldest
Branch: master
https://github.com/mongodb/mongo/commit/2f67f3c66271e724b48afa2db88e8b6c3317f6ab

Comment by Judah Schvimer [ 01/Mar/18 ]

Per discussion with spencer and daniel.gottlieb, we could alternatively do this in replication by having the reaper/ dropDatabase only perform the actual collection drop (as opposed to the rename) after a stable checkpoint is taken a timestamp after the drop oplog entry. This will make it so replication recovery never sees an oplog entry that references the collection, and it is safe to just drop the collection at startup recovery without relaxing idempotency constraints during replication recovery.

Generated at Thu Feb 08 04:32:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.