[SERVER-35037] Create new concurrency suite that runs consecutive operations inside of a single transaction Created: 17/May/18  Updated: 29/Oct/23  Resolved: 03/Aug/18

Status: Closed
Project: Core Server
Component/s: Replication, Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.0.2, 4.1.2

Type: Task Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Samyukta Lanka
Resolution: Fixed Votes: 0
Labels: PM-988
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-35388 Improve misleading error messages for... Closed
Related
related to SERVER-72985 FSM workloads which use the connectio... Closed
is related to SERVER-35964 Delete UBSan concurrency_replication*... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Repl 2018-06-04, Repl 2018-06-18, Repl 2018-07-02, Repl 2018-07-16, Repl 2018-07-30, Repl 2018-08-13
Participants:

 Description   

We should be able to gain further coverage of transactions when operations are happening on the database concurrently by leveraging the existing FSM workloads.

Unlike the replica_sets_multi_stmt_txn_jscore_passthrough.yml test suite, it will be possible (and in some cases very probable) for operations to trigger a write conflict once run inside of a transaction. robert.guo proposed the idea of retrying a single state function's execution until the transaction commits. In order to make this approach compatible with an arbitrary FSM workload, we'll need to make a copy of the args.data object before the state function is called. This ensures a subsequent execution of the state function when the transaction aborts doesn't (speculatively) contain the effects of the aborted transaction. The args.data object should be reassigned to the possibly mutated value if the transactions commits.



 Comments   
Comment by Githook User [ 21/Aug/18 ]

Author:

{'name': 'Samy Lanka', 'email': 'samy.lanka@gmail.com', 'username': 'lankas'}

Message: SERVER-35037 Create new concurrency suite that runs consecutive operations inside of a single transaction

(cherry picked from commit 806b9c32dff674b72110f2acd8604e950afab102)
Branch: v4.0
https://github.com/mongodb/mongo/commit/d1898c5034aee912e6771a301c965c848882c658

Comment by Githook User [ 03/Aug/18 ]

Author:

{'name': 'Samy Lanka', 'email': 'samy.lanka@gmail.com', 'username': 'lankas'}

Message: SERVER-35037 Create new concurrency suite that runs consecutive operations inside of a single transaction
Branch: master
https://github.com/mongodb/mongo/commit/806b9c32dff674b72110f2acd8604e950afab102

Comment by Max Hirschhorn [ 17/May/18 ]

As I started up writing this ticket's description, I came to a realization where I no longer believe the jstests/libs/txns/txn_override.js override file from SERVER-33738 to be useful for the purposes of this new concurrency suite. If we attempted to split a single state function's execution into multiple transactions (e.g. if there was a catalog operation interleaved), then a second transaction needing to be retried would lead to the operations from the successfully committed first transaction being run again in a third transaction. The following patch is a sketch of how we could achieve running operations inside of a transaction with automatic retries. In order to support FSM workloads which run a mixture of transaction-compatible and transaction-incompatible commands, we should have some way to doing fn.call(args.data, args.db, args.collName, connCache) if the server responses with an error message indicating some command doesn't support being run inside of a transaction.

diff --git a/jstests/concurrency/fsm_libs/fsm.js b/jstests/concurrency/fsm_libs/fsm.js
index 0c395bc6c1..14acc7c092 100644
--- a/jstests/concurrency/fsm_libs/fsm.js
+++ b/jstests/concurrency/fsm_libs/fsm.js
@@ -1,5 +1,8 @@
 'use strict';
 
+// For withTxnAndAutoRetryOnWriteConflict.
+load('jstests/concurrency/fsm_workload_helpers/auto_retry_transaction.js');
+
 var fsm = (function() {
     // args.data = 'this' object of the state functions
     // args.db = database object
@@ -43,7 +46,20 @@ var fsm = (function() {
         for (var i = 0; i < args.iterations; ++i) {
             var fn = args.states[currentState];
             assert.eq('function', typeof fn, 'states.' + currentState + ' is not a function');
-            fn.call(args.data, args.db, args.collName, connCache);
+
+            if (TestData.runInsideTransaction) {
+                withTxnAndAutoRetryOnWriteConflict(args.db.getSession(), () => {
+                    // We make a deep copy of 'args.data' before calling the 'fn' state function so
+                    // that if the transaction aborts, then we haven't speculatively modified the
+                    // thread-local state.
+                    const data = Object.extend({}, args.data, true);
+                    fn.call(data, args.db, args.collName, connCache);
+                    args.data = data;
+                });
+            } else {
+                fn.call(args.data, args.db, args.collName, connCache);
+            }
+
             var nextState = getWeightedRandomChoice(args.transitions[currentState], Random.rand());
             currentState = nextState;
         }

Generated at Thu Feb 08 04:38:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.