Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-46238

Race between commitTransaction and transaction expiration leads to invariant

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 4.0.16
    • Fix Version/s: 4.0.17
    • Component/s: Replication
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      Add a sleep to the commitTransaction command:

      diff --git a/src/mongo/db/commands/txn_cmds.cpp b/src/mongo/db/commands/txn_cmds.cpp
      index 62fabeb627..9b1ed18870 100644
      --- a/src/mongo/db/commands/txn_cmds.cpp
      +++ b/src/mongo/db/commands/txn_cmds.cpp
      @@ -80,6 +80,8 @@ public:
               uassert(
                   ErrorCodes::CommandFailed, "commitTransaction must be run within a session", session);
       
      +        opCtx->sleepFor(Milliseconds(2000));
      +
               // commitTransaction is retryable.
               if (session->transactionIsCommitted()) {
                   // We set the client last op to the last optime observed by the system to ensure that
      

      Then run the following test:

      (function() {
      "use strict";
       
      var replTest = new ReplSetTest({nodes: 1});
      replTest.startSet();
      replTest.initiate();
       
      var primary = replTest.getPrimary();
      assert.commandWorked(primary.adminCommand({setParameter: 1, transactionLifetimeLimitSeconds: 1}));
       
      const session = primary.getDB("test").getMongo().startSession();
      const sessionDb = session.getDatabase("test");
      assert.commandWorked(sessionDb.createCollection("c"));
       
      session.startTransaction();
      assert.commandWorked(sessionDb.c.insert({a: 1}));
      assert.commandWorked(sessionDb.adminCommand({commitTransaction: 1, writeConcern: {w: "majority"}}));
      session.commitTransaction();
       
      replTest.stopSet();
      })();
      

      Show
      Add a sleep to the commitTransaction command: diff --git a/src/mongo/db/commands/txn_cmds.cpp b/src/mongo/db/commands/txn_cmds.cpp index 62fabeb627..9b1ed18870 100644 --- a/src/mongo/db/commands/txn_cmds.cpp +++ b/src/mongo/db/commands/txn_cmds.cpp @@ -80,6 +80,8 @@ public: uassert( ErrorCodes::CommandFailed, "commitTransaction must be run within a session", session); + opCtx->sleepFor(Milliseconds(2000)); + // commitTransaction is retryable. if (session->transactionIsCommitted()) { // We set the client last op to the last optime observed by the system to ensure that Then run the following test: (function() { "use strict";   var replTest = new ReplSetTest({nodes: 1}); replTest.startSet(); replTest.initiate();   var primary = replTest.getPrimary(); assert.commandWorked(primary.adminCommand({setParameter: 1, transactionLifetimeLimitSeconds: 1}));   const session = primary.getDB("test").getMongo().startSession(); const sessionDb = session.getDatabase("test"); assert.commandWorked(sessionDb.createCollection("c"));   session.startTransaction(); assert.commandWorked(sessionDb.c.insert({a: 1})); assert.commandWorked(sessionDb.adminCommand({commitTransaction: 1, writeConcern: {w: "majority"}})); session.commitTransaction();   replTest.stopSet(); })();
    • Sprint:
      Repl 2020-03-09
    • Case:

      Description

      The following race leads to invariant failure:

      The startPeriodicThreadToAbortExpiredTransactions thread kills the commitTransaction operation here. It then marks the transaction state as aborted here.

      The commitTransaction operation gets interrupted and triggers an abort-guard here. However, since the transaction state was already marked as aborted, we skip the destruction of the WriteUnitOfWork here. This means that the commitTransaction operation is still in a WriteUnitOfWork when it waits for writeConcern, triggering this invariant.

      This only occurs on 4.0, due to a refactor on 4.2, where transaction state can only be modified by the thread that checked out the session.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jason.chan Jason Chan
              Reporter:
              tess.avitabile Tess Avitabile
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: