[SERVER-47574] Ban non-transaction snapshot reads on capped collections Created: 15/Apr/20  Updated: 29/Oct/23  Resolved: 30/Apr/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: Lingzhi Deng Assignee: Lingzhi Deng
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File snapshot_reads_capped_collection.js    
Issue Links:
Documented
is documented by DOCS-13624 Investigate changes in SERVER-47574: ... Closed
Related
related to SERVER-47824 Ban transaction snapshot reads on cap... Closed
related to SERVER-63813 Allow transactions and snapshot reads... Backlog
Backwards Compatibility: Fully Compatible
Sprint: Repl 2020-05-04
Participants:

 Comments   
Comment by Githook User [ 30/Apr/20 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-47574: Ban non-transaction snapshot reads on capped collections
Branch: master
https://github.com/mongodb/mongo/commit/f1d2e697affa906307d8623bb3d3746acb1ae162

Comment by Tess Avitabile (Inactive) [ 28/Apr/20 ]

Yes, as we discussed, can you create a separate ticket for banning operations on capped collections inside transactions, and put that ticket into Needs Triage?

Comment by Lingzhi Deng [ 27/Apr/20 ]

Gotcha. Since this doesn't seem to be a new issue specific to snapshot reads outside of transaction, I will go ahead to ban snapshot reads on capped collections. tess.avitabile, should I also ban them inside transactions? I think we should and document the limitations.

Comment by Daniel Gottlieb (Inactive) [ 27/Apr/20 ]

Do you think it is reasonable to error out snapshot reads on capped collections?

Yes

Or is there an easy way to make them work (timestamping the delete?)?

I imagine that's relatively easy on primaries. I think secondaries would be harder: a delete associated with an insert at time 1 on a primary might happen on time 2 at a secondary, and time 3 on a different secondary! So the naive approach would likely result in a different problem to solve.

Comment by Lingzhi Deng [ 27/Apr/20 ]

Capped collections typically (always?) delete records with an unreplicated/untimestamped write. Those deletions will always become visible in a newer transaction.

Ah, yes, I see. I think this is the reason. And good point about bumping the minVisibleTimestamp when converting (if possible) a capped to a non-capped in order to ban reads at point in time before the conversion.

Do you think it is reasonable to error out snapshot reads on capped collections? Or is there an easy way to make them work (timestamping the delete?)?

Comment by Daniel Gottlieb (Inactive) [ 27/Apr/20 ]

Snapshot reads don't seem to work with capped collections possibly due to that capped collections use "truncate" to delete records.

The oplog gets called via WT's truncate command yes, but that's not true for all capped deletions, nor is it the reason why a transaction yield/restore at the same timestamp across a capped deletion gives different results. Capped collections typically (always?) delete records with an unreplicated/untimestamped write. Those deletions will always become visible in a newer transaction.

Moreover, it's possible to convertToCapped a non-capped collection into a capped one. I'm not sure if a capped collection can be made normal again. Presumably, if it could, we'd need to bump the minVisibleTimestamp such that the query of "is this a capped collection" is appropriately pseudo-versioned in the catalog.

Comment by Lingzhi Deng [ 27/Apr/20 ]

I think we may need to ban snapshot reads on capped collections in general. Snapshot reads don't seem to work with capped collections possibly due to that capped collections use "truncate" to delete records.

The documentation claims that "Reads from capped collections are still supported in transactions." However, for snapshot reads inside transactions, if a transaction uses snapshot with an atClusterTime (e.g. coming from mongos), it seems that it might not see all the documents at that point in time if documents were deleted. Snapshot transactions without atClusterTime seems to work probably because they hold onto the WT txn and the snapshot.

I attached the test file I used snapshot_reads_capped_collection.js. Grep for "<---" to see the command responses. For atCluserTime reads (both transaction and non-transaction), the find commands in the test return an empty result set.

CC tess.avitabile jesse

Generated at Thu Feb 08 05:14:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.