[SERVER-44854] Ensure collections with commit timestamps have appropriately set minimum visible snapshots by the time the collections are visible in the catalog Created: 26/Nov/19  Updated: 29/Oct/23  Resolved: 16/Mar/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.4.0-rc0, 4.7.0

Type: Task Priority: Major - P3
Reporter: Maria van Keulen Assignee: Maria van Keulen
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
is related to SERVER-46900 Prohibit operations on system.profile... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: Execution Team 2019-12-30, Execution Team 2020-03-09, Execution Team 2020-03-23
Participants:

 Description   

SERVER-43859 exposed an issue where, for collections on standalones and non-replicated collections on replica sets, during collection creation, the object for the newly-created collection could be accessible prior to the storage transaction being committed. This was not a problem beforeĀ SERVER-43859, since the MODE_X collection lock was held throughout the creation, preventing access to the collection object before the command completed. As of that ticket, we no longer take the exclusive lock, so the issue needs to be otherwise addressed.



 Comments   
Comment by Githook User [ 17/Mar/20 ]

Author:

{'name': 'Maria van Keulen', 'email': 'maria@mongodb.com', 'username': 'mvankeulen94'}

Message: SERVER-44854 Make stronger visibility guarantees for new collections

(cherry picked from commit ce57a85479032380593268db65dfc9441c65cd9a)
Branch: v4.4
https://github.com/mongodb/mongo/commit/107f3397f5bf3a2cb14624fadbe85c05bd5caa03

Comment by Maria van Keulen [ 16/Mar/20 ]

As an update, independently of whether non-replicated collections get minimum visible snapshots set, the issue that was exposed due to taking IX locks instead of X locks for collections in SERVER-43859 was that the non-replicated collections were accessible in the catalog prior to storage transaction commit. As of SERVER-46285, this is no longer a concern.

The issue that still exists however (and existed prior to SERVER-43859), is that non-replicated collections can get created without a minimum visible snapshot set, even after storage transaction commit. This issue has to do with our criteria for setting the minimum visible snapshot.

This ticket (SERVER-44854) need only be concerned with collection visibility prior to storage transaction commit, and ensuring the minimum visible snapshot is set for all cases where we have a commit timestamp by the time the collection is visible in the catalog. Non-replicated collections do not have a commit timestamp, and will not necessarily have the minimum visible snapshot set. I've filed SERVER-46900 to address the interesting case there.

Comment by Githook User [ 16/Mar/20 ]

Author:

{'name': 'Maria van Keulen', 'username': 'mvankeulen94', 'email': 'maria@mongodb.com'}

Message: SERVER-44854 Make stronger visibility guarantees for new collections
Branch: master
https://github.com/mongodb/mongo/commit/ce57a85479032380593268db65dfc9441c65cd9a

Comment by Maria van Keulen [ 02/Mar/20 ]

We believe SERVER-46285 should have addressed the issue for non-replicated collections not having their minVisibleSnapshots set by the time collections are visible in the catalog.

The work for this ticket should be to confirm (possibly with an invariant) that all "special case" collections that can potentially receive a null minVisibleSnapshot initially have an appropriately set minVisibleSnapshot by the time they are accessible in the catalog.

Comment by Maria van Keulen [ 28/Feb/20 ]

SERVER-46241 will add a timestamp-related invariant. This invariant should be updated once the timestamping issues are resolved for non-replicated collections.

Comment by Maria van Keulen [ 11/Feb/20 ]

This ticket should also ensure that it is not possible to create/write to non-replicated collections inside multi-document transactions.

Comment by Daniel Gottlieb (Inactive) [ 26/Nov/19 ]

The two angles I know of that can solve this problem:

  1. Upgrade locks for those cases to MODE_X (reverting behavior)
  2. Faking a timestamp value for minVisibleSnapshot. Picking a value too large is safe, picking a value too small can lead to incorrect behavior.

While it may be pleasing to use the same solution for both standalone mode and unreplicated collections in replica sets, my hunch is it might be easiest to solve them differently:

  1. I think having collection creation upconvert locks from MODE_IX to MODE_X on standalones might be easier than trying to introduce timestamps.
  2. I think choosing fake timestamps for unreplicated collections in replica sets might be easier than knowing when to upconvert a lock to MODE_X.
Generated at Thu Feb 08 05:07:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.