[SERVER-43925] Proactively close newly empty databases Created: 09/Oct/19  Updated: 07/Feb/24

Status: Backlog
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Backlog - Storage Execution Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-43890 Remove two-phase database drops Backlog
Issue split
split from SERVER-33272 The DatabaseHolder::close() function ... Closed
Related
related to SERVER-85975 Clear database state when dropping la... Open
is related to SERVER-77281 Investigate removing use of the Datab... Backlog
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Participants:
Linked BF Score: 0

 Description   

Currently, the kv engine does not check to see if a database is empty after a collection drop. Thus, empty databases can persist until mongod restarts (or the catalog is reloaded via replication rollback to stable timestamp). This situation is usually benign, but a database entry's presence does do one thing: it enforces that no other databases that are spelled the same but differ in capitalization can be created.
Because recover-to-timestamp exists, it can result in some replica set nodes having a catalog entry for an empty database while other nodes do not have such an entry. This can result in the ability to crash secondary nodes that have an empty database entry that a primary node does not have - by attempting to create a database with a different case than the existing empty database entry.

To fix this, we can check to see if a database is empty (i.e., no collection entries remain in the catalog) after a collection drop. If it is indeed empty, we can attempt to close the database by locking it in X mode and calling DatabaseHolder::close().



 Comments   
Comment by Gregory Wlodarek [ 09/Oct/19 ]

After discussing with the team, we're going to set this ticket aside as there's no urgency for it right now and needs more work to be done before tackling it.
To be able to do this ticket we'll need to do the following beforehand in order:

  • Get rid of readConcernMajority=false.
  • Get rid of two-phase database drops. The database object today doesn't contain any special state except for the isDropPending flag. We can remove the dropDatabase oplog entry and rollback handler.

This is problematic to do today because if we remove the Database in the DatabaseHolder when the last collection is dropped, then we lose our isDropPending flag (a member of the Database object class). This flag prevents new collections from being created on the database while waiting for the second phase of collection drops to be majority committed when using majority read concern.
While waiting for the majority point to be reached, a new collection can be created, which will implicitly open the database with the same name while there are collections still drop pending for that database name. Once the majority point is reached, the newly created collection would be forcefully one phase dropped.

Timeline of events if we implicitly close empty databases today

Client 1:

  • dropDatabase command executed, isDropPending flag set.
  • First phase drop for Coll A
    • Replicate...
  • First phase drop for Coll B
    • Replicate...
    • Close Database in DatabaseHolder since it's empty now.
    • Replicate...

Client 2:

  • Create collection C on the same database (the isDropPending flag would've prevented this).
    • Replicate openDB() and createCollection()...

Client 1:

  • Majority point reached, second phase of drops executed and committed across all nodes.
  • Forcefully one phase drops collection C created by 'Client 2'.

Another issue that would occur is during rollback. When the last collection is first phase dropped and rollback occurs, the Database object will no longer exist in the DatabaseHolder.

Generated at Thu Feb 08 05:04:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.