[SERVER-47574] Ban non-transaction snapshot reads on capped collections Created: 15/Apr/20 Updated: 29/Oct/23 Resolved: 30/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Lingzhi Deng | Assignee: | Lingzhi Deng |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Sprint: | Repl 2020-05-04 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Comments |
| Comment by Githook User [ 30/Apr/20 ] |
|
Author: {'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}Message: |
| Comment by Tess Avitabile (Inactive) [ 28/Apr/20 ] |
|
Yes, as we discussed, can you create a separate ticket for banning operations on capped collections inside transactions, and put that ticket into Needs Triage? |
| Comment by Lingzhi Deng [ 27/Apr/20 ] |
|
Gotcha. Since this doesn't seem to be a new issue specific to snapshot reads outside of transaction, I will go ahead to ban snapshot reads on capped collections. tess.avitabile, should I also ban them inside transactions? I think we should and document the limitations. |
| Comment by Daniel Gottlieb (Inactive) [ 27/Apr/20 ] |
Yes
I imagine that's relatively easy on primaries. I think secondaries would be harder: a delete associated with an insert at time 1 on a primary might happen on time 2 at a secondary, and time 3 on a different secondary! So the naive approach would likely result in a different problem to solve. |
| Comment by Lingzhi Deng [ 27/Apr/20 ] |
Ah, yes, I see. I think this is the reason. And good point about bumping the minVisibleTimestamp when converting (if possible) a capped to a non-capped in order to ban reads at point in time before the conversion. Do you think it is reasonable to error out snapshot reads on capped collections? Or is there an easy way to make them work (timestamping the delete?)? |
| Comment by Daniel Gottlieb (Inactive) [ 27/Apr/20 ] |
The oplog gets called via WT's truncate command yes, but that's not true for all capped deletions, nor is it the reason why a transaction yield/restore at the same timestamp across a capped deletion gives different results. Capped collections typically (always?) delete records with an unreplicated/untimestamped write. Those deletions will always become visible in a newer transaction. Moreover, it's possible to convertToCapped a non-capped collection into a capped one. I'm not sure if a capped collection can be made normal again. Presumably, if it could, we'd need to bump the minVisibleTimestamp such that the query of "is this a capped collection" is appropriately pseudo-versioned in the catalog. |
| Comment by Lingzhi Deng [ 27/Apr/20 ] |
|
I think we may need to ban snapshot reads on capped collections in general. Snapshot reads don't seem to work with capped collections possibly due to that capped collections use "truncate" to delete records. The documentation claims that "Reads from capped collections are still supported in transactions." However, for snapshot reads inside transactions, if a transaction uses snapshot with an atClusterTime (e.g. coming from mongos), it seems that it might not see all the documents at that point in time if documents were deleted. Snapshot transactions without atClusterTime seems to work probably because they hold onto the WT txn and the snapshot. I attached the test file I used snapshot_reads_capped_collection.js. Grep for "<---" to see the command responses. For atCluserTime reads (both transaction and non-transaction), the find commands in the test return an empty result set. |