[SERVER-56023] listCollections can return empty metadata for a collection which has been concurrently dropped Created: 12/Apr/21  Updated: 29/Oct/23  Resolved: 20/May/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.0-rc0, 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Nicholas Zolnierz Assignee: Henrik Edin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-55352 Missing uuid in collection info retur... Closed
Problem/Incident
Related
related to SERVER-56002 listIndexes can read partial state fr... Closed
related to SERVER-56877 insert operations may fail to set ind... Closed
related to SERVER-57083 Coverity analysis defect 120111: Pars... Closed
related to SERVER-57324 dropDatabase should not modify Collec... Closed
related to SERVER-57775 collection metadata for multikey not ... Closed
related to SERVER-56999 Unify CollectionImpl creation interface Closed
related to SERVER-57127 Refactor multikey and other state out... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: Execution Team 2021-05-17, Execution Team 2021-05-31
Participants:
Linked BF Score: 158

 Description   

This is very similar to SERVER-56002, whereby a lock-free operation attempts to access the DurableCatalog while a concurrent operation is modifying it. For listCollections, its possible to lookup the collection in the CollectionCatalog just before a collection has been dropped, which means it no longer has an entry in the DurableCatalog. Then, when listCollections proceeds to read the collection options, it receives an empty object and returns it in the command response.

One possible downstream impact for this is a failed shardCollection command when try to extract the UUID of the collection to shard.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 24/May/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-56002 SERVER-56023 Store Collection metadata in the Collection and reply on the copy-on-write machinery to keep it in sync with the durable catalog.

All updates to the metadata needs to happen through the Collection, moved interfaces from the DurableCatalog to the Collection.
Removed back pointer to Collection in IndexCatalogEntryImpl, interfaces now correctly take a const or non-const Collection. This should make its iterface const-correct to avoid making bugs where the copy-on-write system for Collections are bypassed.

Multikey handle is special as it needs to happen without exclusive access to the Collection. Implemented isolation for the Collection metadata when multikey is changed. It handles multi-doc transactions and is only commited to the Collection instance after the write to the durable catalog successfully commits.

listCollections and listIndexes can now safetly read the metadata cache without needing to read from the durable catalog making them safe to do without Collection level locks.

(cherry picked from commit 11de948b0c50df7d12de09ae0f01e791fc5d70d7)
Branch: v5.0
https://github.com/mongodb/mongo/commit/50547a878aaa67722f69c05b05a35236f8f0def9

Comment by Githook User [ 24/May/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-56002 SERVER-56023 Store Collection metadata in the Collection and reply on the copy-on-write machinery to keep it in sync with the durable catalog.

(cherry picked from commit 15e97c021b4e1feba110f31a563076980794d0a5)
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/0637bb99c0e672df3f0710a970ae1bb2aad0a0ed

Comment by Githook User [ 20/May/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-56002 SERVER-56023 Store Collection metadata in the Collection and reply on the copy-on-write machinery to keep it in sync with the durable catalog.

All updates to the metadata needs to happen through the Collection, moved interfaces from the DurableCatalog to the Collection.
Removed back pointer to Collection in IndexCatalogEntryImpl, interfaces now correctly take a const or non-const Collection. This should make its iterface const-correct to avoid making bugs where the copy-on-write system for Collections are bypassed.

Multikey handle is special as it needs to happen without exclusive access to the Collection. Implemented isolation for the Collection metadata when multikey is changed. It handles multi-doc transactions and is only commited to the Collection instance after the write to the durable catalog successfully commits.

listCollections and listIndexes can now safetly read the metadata cache without needing to read from the durable catalog making them safe to do without Collection level locks.
Branch: master
https://github.com/mongodb/mongo/commit/11de948b0c50df7d12de09ae0f01e791fc5d70d7

Comment by Githook User [ 20/May/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-56002 SERVER-56023 Store Collection metadata in the Collection and reply on the copy-on-write machinery to keep it in sync with the durable catalog.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/15e97c021b4e1feba110f31a563076980794d0a5

Comment by Nicholas Zolnierz [ 12/Apr/21 ]

Note that as of SERVER-54918, the shardCollection path changed to not use listCollections and instead use AutoGetCollection to obtain the UUID of the target collection.

Generated at Thu Feb 08 05:38:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.