[SERVER-43018] Formalize contract for safely accessing the catalog Created: 23/Aug/19  Updated: 29/Oct/23  Resolved: 06/Nov/19

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: 4.2.0
Fix Version/s: 4.3.1, 4.2.3

Type: Bug Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Suganthi Mani
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: HTML File sample_patch    
Issue Links:
Backports
Depends
is depended on by SERVER-40620 Return or log error if an index key i... Closed
Related
related to SERVER-40352 Transactions that perform untimestamp... Closed
related to SERVER-42956 ReadyIndexesIterator::_advance() does... Closed
related to SERVER-42497 Detect/log unintentional untimestampe... Closed
is related to SERVER-45421 Fix transactions_block_ddl.js to use ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.2
Sprint: Repl 2019-09-09, Repl 2019-09-23, Repl 2019-10-07, Repl 2019-10-21, Repl 2019-11-04, Repl 2019-11-18
Participants:
Linked BF Score: 12

 Description   

The MongoDB catalog is currently compromised of two layers:

  • An in-memory layer that represents "now"
  • A durable layer which is transparently (to the storage engine) backed in the same datastore as all of the other data.

A single read of the catalog can be serviced by both layers and the durable layer is typically keyed on the collection namespace. Thus allowing changes to the in-memory version while holding a transaction/snapshot open on the durable layer requires concurrency control to prevent wires getting crossed.

The proposed contract for catalog reads safe:

  • Reading with a timestamped (storage) transaction is always safe.
  • Reading without a timestamped transaction requires either:
    • Acquiring locks on all relevant collections prior to acquiring a snapshot for a catalog read.
    • Computing/Setting a "catalog conflicting timestamp". The snapshot acquired must include all writes up to this time. It's permissible for the snapshot to include additional writes at a later timestamp.

In cases where a read timestamp is not specified, the "catalog conflicting timestamp" must be substituted for comparison against "minimum visible timestamps" (1, 2 and 3).

This "catalog conflicting timestamp" mechanism can be removed when a versioned catalog backed by the durable catalog is in place.

A sample, incomplete patch is attached.



 Comments   
Comment by Githook User [ 12/Dec/19 ]

Author:

{'name': 'Suganthi Mani', 'email': 'suganthi.mani@mongodb.com', 'username': 'smani87'}

Message: SERVER-43018 Transactions that perform untimestamped reads should check min visible snapshot for any pending catalog changes in collection and index entries.

(cherry picked from commit 44e56dafcbcb624417960c0c03cf5176383efe46)
(cherry picked from commit d63202a259f038a8a8ef1af6f67c8cec396de4e5)
Branch: v4.2
https://github.com/mongodb/mongo/commit/586eb7e300b726fa26f89e237d3a1655b285a9ca

Comment by Githook User [ 06/Nov/19 ]

Author:

{'name': 'Suganthi Mani', 'username': 'smani87', 'email': 'suganthi.mani@mongodb.com'}

Message: SERVER-43018 Transactions that perform untimestamped reads should check min visible snapshot for any pending catalog changes in collection and index entries.
Branch: master
https://github.com/mongodb/mongo/commit/44e56dafcbcb624417960c0c03cf5176383efe46

Generated at Thu Feb 08 05:02:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.