[SERVER-73766] Implement a ScopedLocalCatalogWriter utility Created: 08/Feb/23  Updated: 29/Oct/23  Resolved: 30/Mar/23

Status: Closed
Project: Core Server
Component/s: Sharding, Storage
Affects Version/s: None
Fix Version/s: 7.0.0-rc0

Type: Task Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Kaloian Manassiev
Resolution: Fixed Votes: 0
Labels: PM-2144-Milestone-0
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-73238 Introduce the shard_role_api library Closed
is depended on by SERVER-73298 Thread-through `ScopedCollectionOrVie... Closed
is depended on by SERVER-73756 Thread-through `ScopedCollectionOrVie... Closed
Problem/Incident
Related
related to SERVER-75286 Coverity analysis defect 135887: Unin... Closed
related to SERVER-75157 Complete TODO listed in SERVER-73766 Blocked
Backwards Compatibility: Fully Compatible
Sprint: Sharding EMEA 2023-02-20, Sharding EMEA 2023-03-06, Sharding EMEA 2023-03-20, Sharding EMEA 2023-04-03
Participants:
Linked BF Score: 135

 Description   

The CollectionWriter (and it's sibling AutoGetCollection::getWritableCollection) paradigms are really an attempt to provide a "see your own catalog writes" semantics for catalog changes performed while holding an AutoGetCollection. This ticket is proposing to implement a proper utility which ensures that all collection acquisitions held by an operation automatically see any catalog writes performed by the code path on which they have been placed.

The semantics of this utility will be:

  • It must be instantiated only within a WUOU and its lifetime must be smaller than that of the WUOW
  • Must must be instantiated only by passing a ScopedCollectionOrViewAcquisition obtained with at least MODE_X lock on the collection
  • On construction must invalidate the ScopedCollectionOrViewAcquisition to guarantee that it is not used while the underlying state is being modified
  • On destruction must make visible all the changes made (uncommitted, since within a WUOW) to the ScopedCollectionOrViewAcquisition
  • On rollback of the WUOW, must bring the respective AcquiredCollection to the old state (which means any active ScopedCollectionOrViewAcquisitions will start seeing the old state)
  •  (FUTURE) On commit must ensure that the committed state presented doesn't advance beyond the Sharding's Placement timestamp

Context

The AutoGetXXX utilities and the CollectionPtr are really caches of a particular collection from the CollectionCatalog, which itself is a cache of the persistent WT state on disk. The CollectionWriter utility serves two purposes:

  • Provide a programmatic C++ way to remove the constness of a CollectionPtr
  • Establish a "release"-barrier after which any caches of the CollectionCatalog (namely AutoGetCollection(s)) will reflect the changes that were made to the local catalog, pertaining to that collection

The ScopedCollectionOrView acquisition class is really a more complete cache of the entire catalog information for a collection (sharding + local) and at the end of PM-2144, no code paths should be interacting with the catalog outside of ScopedCollectionOrView. The ScopedLocalCatalogWriter described in this ticket will provide the release semantics described above, without dangling memory bugs like SERVER-73755, because it will operate on the TransactionResources' collection acquisitions rather than on stack variables.



 Comments   
Comment by Githook User [ 30/Mar/23 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-73766 Use ScopedCollectionAcquisition in the CollectionBulkLoaderImpl
Branch: master
https://github.com/mongodb/mongo/commit/fcd8862523226aba53062faed279c329d7f07ad5

Comment by Githook User [ 24/Mar/23 ]

Author:

{'name': 'Sviatlana Zuiko', 'email': 'sviatlana.zuiko@mongodb.com', 'username': 'szuiko'}

Message: Revert "SERVER-73766 Use ScopedCollectionAcquisition in the CollectionBulkLoaderImpl"

This reverts commit 7a62847796d3797278a7b5fb6215948cd325b45d.
Branch: master
https://github.com/mongodb/mongo/commit/56c6c67f6862825d3206251fc9b323509d6c1bd0

Comment by Githook User [ 22/Mar/23 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-73766 Use ScopedCollectionAcquisition in the CollectionBulkLoaderImpl
Branch: master
https://github.com/mongodb/mongo/commit/7a62847796d3797278a7b5fb6215948cd325b45d

Comment by Githook User [ 16/Mar/23 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-73766 Introduce kLocalCatalogOnlyWithPotentialDataLoss placement concern
Branch: master
https://github.com/mongodb/mongo/commit/757102d8184507a82a1bc83b6bbd844ff8e10ab6

Comment by Githook User [ 23/Feb/23 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-73766 Dependent cleanups
Branch: master
https://github.com/mongodb/mongo/commit/5ee40fa2dc77876c2885a19bc02ff15a1ea1dde8

Generated at Thu Feb 08 06:25:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.