Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-46238

Race between commitTransaction and transaction expiration leads to invariant

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.0.17
    • Affects Version/s: 4.0.16
    • Component/s: Replication
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Hide

      Add a sleep to the commitTransaction command:

      diff --git a/src/mongo/db/commands/txn_cmds.cpp b/src/mongo/db/commands/txn_cmds.cpp
      index 62fabeb627..9b1ed18870 100644
      --- a/src/mongo/db/commands/txn_cmds.cpp
      +++ b/src/mongo/db/commands/txn_cmds.cpp
      @@ -80,6 +80,8 @@ public:
               uassert(
                   ErrorCodes::CommandFailed, "commitTransaction must be run within a session", session);
       
      +        opCtx->sleepFor(Milliseconds(2000));
      +
               // commitTransaction is retryable.
               if (session->transactionIsCommitted()) {
                   // We set the client last op to the last optime observed by the system to ensure that
      

      Then run the following test:

      (function() {
      "use strict";
      
      var replTest = new ReplSetTest({nodes: 1});
      replTest.startSet();
      replTest.initiate();
      
      var primary = replTest.getPrimary();
      assert.commandWorked(primary.adminCommand({setParameter: 1, transactionLifetimeLimitSeconds: 1}));
      
      const session = primary.getDB("test").getMongo().startSession();
      const sessionDb = session.getDatabase("test");
      assert.commandWorked(sessionDb.createCollection("c"));
      
      session.startTransaction();
      assert.commandWorked(sessionDb.c.insert({a: 1}));
      assert.commandWorked(sessionDb.adminCommand({commitTransaction: 1, writeConcern: {w: "majority"}}));
      session.commitTransaction();
      
      replTest.stopSet();
      })();
      
      Show
      Add a sleep to the commitTransaction command: diff --git a/src/mongo/db/commands/txn_cmds.cpp b/src/mongo/db/commands/txn_cmds.cpp index 62fabeb627..9b1ed18870 100644 --- a/src/mongo/db/commands/txn_cmds.cpp +++ b/src/mongo/db/commands/txn_cmds.cpp @@ -80,6 +80,8 @@ public : uassert( ErrorCodes::CommandFailed, "commitTransaction must be run within a session" , session); + opCtx->sleepFor(Milliseconds(2000)); + // commitTransaction is retryable. if (session->transactionIsCommitted()) { // We set the client last op to the last optime observed by the system to ensure that Then run the following test: (function() { "use strict" ; var replTest = new ReplSetTest({nodes: 1}); replTest.startSet(); replTest.initiate(); var primary = replTest.getPrimary(); assert .commandWorked(primary.adminCommand({setParameter: 1, transactionLifetimeLimitSeconds: 1})); const session = primary.getDB( "test" ).getMongo().startSession(); const sessionDb = session.getDatabase( "test" ); assert .commandWorked(sessionDb.createCollection( "c" )); session.startTransaction(); assert .commandWorked(sessionDb.c.insert({a: 1})); assert .commandWorked(sessionDb.adminCommand({commitTransaction: 1, writeConcern: {w: "majority" }})); session.commitTransaction(); replTest.stopSet(); })();
    • Repl 2020-03-09

      The following race leads to invariant failure:

      The startPeriodicThreadToAbortExpiredTransactions thread kills the commitTransaction operation here. It then marks the transaction state as aborted here.

      The commitTransaction operation gets interrupted and triggers an abort-guard here. However, since the transaction state was already marked as aborted, we skip the destruction of the WriteUnitOfWork here. This means that the commitTransaction operation is still in a WriteUnitOfWork when it waits for writeConcern, triggering this invariant.

      This only occurs on 4.0, due to a refactor on 4.2, where transaction state can only be modified by the thread that checked out the session.

            Assignee:
            jason.chan@mongodb.com Jason Chan
            Reporter:
            tess.avitabile@mongodb.com Tess Avitabile (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: