[SERVER-47824] Ban transaction snapshot reads on capped collections Created: 28/Apr/20  Updated: 29/Oct/23  Resolved: 20/Jul/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Bug Priority: Major - P3
Reporter: Lingzhi Deng Assignee: Ali Mir
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-13775 Investigate changes in SERVER-47824: ... Closed
Problem/Incident
Related
related to SERVER-63813 Allow transactions and snapshot reads... Backlog
related to SERVER-40684 Ban transactions against capped colle... Closed
is related to SERVER-42372 Reads against capped collections aren... Closed
is related to SERVER-47574 Ban non-transaction snapshot reads on... Closed
Backwards Compatibility: Minor Change
Steps To Reproduce:

/* Tests capped collection reads with readConcern level snapshot.
 *
 * @tags: [requires_majority_read_concern, requires_fcv_46]
 */
(function() {
"use strict";
 
const replSet = new ReplSetTest({nodes: 1});
 
replSet.startSet();
replSet.initiate();
 
const collName = "coll";
const primary = replSet.getPrimary();
const primaryDB = primary.getDB('test');
 
assert.commandWorked(primaryDB.createCollection(collName, {capped: true, size: 32, max: 1}));
 
const insertTimestamp =
    assert
        .commandWorked(primaryDB.runCommand(
            {insert: collName, documents: [{_id: 0}], writeConcern: {w: "majority"}}))
        .operationTime;
 
// Perform an extra insert outside of the transaction to trigger capped collection cleanup.
assert.commandWorked(
    primaryDB.runCommand({insert: collName, documents: [{_id: 1}], writeConcern: {w: "majority"}}));
 
// Perform a local read outside of the transaction and it should only see {_id: 1}.
let cmdRes = assert.commandWorked(primaryDB.runCommand({find: collName}));
assert.sameMembers([{_id: 1}], cmdRes.cursor.firstBatch, tojson(cmdRes));
 
// TODO(SERVER-47824): Also ban transaction snapshot reads on capped collections because reading
// from a capped collection within a transaction with an atClusterTime could give wrong result.
session = primary.startSession({causalConsistency: false});
sessionDB = session.getDatabase('test');
session.startTransaction({readConcern: {level: 'snapshot', atClusterTime: insertTimestamp}});
cmdRes = assert.commandWorked(sessionDB.runCommand({find: collName}));
assert.sameMembers([{_id: 0}], cmdRes.cursor.firstBatch, tojson(cmdRes));;  // This would FAIL!
 
assert.commandWorked(session.abortTransaction_forTesting());
 
replSet.stopSet();
})();

Sprint: Repl 2020-06-15, Repl 2020-07-27
Participants:
Linked BF Score: 0

 Description   

Capped collections typically (always?) delete records with an unreplicated/untimestamped write. Those deletions will always become visible in a newer transaction. So if a transaction uses snapshot with an atClusterTime (e.g. coming from mongos), it may not see some deleted documents even if they existed at the point in time requested.

Snapshot transactions without atClusterTime might still work if the snapshot is open before the deletions happen probably because they hold onto the WT txn and the snapshot. But I am not 100% sure. Either way, it seems that we will still have to ban transaction snapshot reads on capped collections unless we choose to timestamp the capped collection deletions.



 Comments   
Comment by Githook User [ 20/Jul/20 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-47824 Exclude requires_capped in concurrency_sharded_multi_stmt_txn.yml
Branch: master
https://github.com/mongodb/mongo/commit/f93fc644d454c9a949fd31c140d4ba93a2e819ce

Comment by Githook User [ 20/Jul/20 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-47824 Ban transaction snapshot reads on capped collections
Branch: master
https://github.com/mongodb/mongo/commit/b2722ee46668782fa2ec86fe54547b9d98b5848b

Comment by Tess Avitabile (Inactive) [ 25/Jun/20 ]

Sounds good, thank you!

Comment by Evin Roesle [ 25/Jun/20 ]

I would prefer option 1 to Ban all operations on capped collections from transactions in 4.6+.

Comment by Tess Avitabile (Inactive) [ 25/Jun/20 ]

I ran lingzhi.deng's repro with a sharded cluster as well, to confirm that this problem exists for a sharded cluster (we don't expect atClusterTime to be used for transactions on replica sets). Thus the scope of the problem is that on version 4.2+, for transactions run with readConcern level snapshot on sharded clusters, reads from capped collections may return inconsistent data.

In SERVER-40684, we decided to ban all operations on capped collections from transactions in 4.2+, but we neglected to ban read operations.

Our options are:

  1. Ban all operations on capped collections from transactions in 4.6+.
  2. Ban all operations on capped collections from transactions and backport to 4.2.
  3. Close as Won't Fix.

evin.roesle, what is your preference?

Generated at Thu Feb 08 05:15:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.