[SERVER-39957] Two phase drop by rename should delay the second phase until drop optime is both checkpointed and majority committed Created: 05/Mar/19  Updated: 06/Dec/22  Resolved: 02/Apr/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Xiangyu Yao (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Problem/Incident
Related
Assigned Teams:
Storage Execution
Operating System: ALL
Sprint: Storage NYC 2019-03-11, Storage NYC 2019-03-25
Participants:
Linked BF Score: 33
Story Points: 3

 Description   

Currently, the old two-phase drop (by rename) executes the second phase (drop of the WT table file) when majority commit point moves past drop optime. If majority commit point is ahead of checkpoint and a crash happens after the second phase drop, on restart, the server will find the metadata of the collection still in the catalog (because it loads last checkpoint) but the actual WT file gets dropped.

On restart, the server can detect that this is from an unclean shutdown by examining the mongod.lock file. Then it can safely remove the metadata of those collections which do not have WT table files.

However, instead of crashing after the second phase drop, opening up backup cursor would cause similar issue which is harder to solve: there is also an inconsistency between WT table files and the catalog. But since we don't copy mongod.lock during backup, then the server does not trigger the code which reconciles the catalog. Then it tries to open a WT file which does not exist and hit this fassert.

To fix this problem, we should delay the second phase until drop optime is checkpointed.



 Comments   
Comment by Githook User [ 13/Mar/19 ]

Author:

{'name': 'Mathew Robinson', 'username': 'chasinglogic', 'email': 'chasinglogic@gmail.com'}

Message: Revert "SERVER-39957 Two-phase drop by rename should wait until dropOptime is both checkpointed and majority committed"

This reverts commit c7e6cd6803e584a6951469e74af93ec3a7a47148.
Branch: master
https://github.com/mongodb/mongo/commit/f0e594510eb449b037a788b3aa057e911ab008b9

Comment by Githook User [ 12/Mar/19 ]

Author:

{'name': 'Xiangyu Yao', 'username': 'xy24', 'email': 'xiangyu.yao@mongodb.com'}

Message: SERVER-39957 Two-phase drop by rename should wait until dropOptime is both checkpointed and majority committed
Branch: master
https://github.com/mongodb/mongo/commit/c7e6cd6803e584a6951469e74af93ec3a7a47148

Generated at Thu Feb 08 04:53:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.