Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-73915

TransactionCoordinatorService may stall primary step-up from completing when replica set shard steps down and back up quickly

    • Sharding NYC
    • Minor Change
    • ALL
    • 5
    • 3

      All TransactionCoordinators from the previous term when the node was primary must have exited before a node can finish stepping up as primary. The mechanisms for interrupting TransactionCoordinators involves interrupting active OperationContext and shutting down the txn::AsyncWorkScheduler's TaskExecutor. However the TransactionCoordinator also waits through the WaitForMajorityService and isn't guaranteed to be interrupted. This results in the node completing its member state PRIMARY transition but being unable to exit "drain mode" where the node can accepts writes as primary.

      One visible symptom of this behavior is for the following message to be logged every 5 seconds.

      [js_test:txn_two_phase_commit_basic] d20040| {"t":{"$date":"2023-02-10T22:40:06.714+00:00"},"s":"I",  "c":"TXN",      "id":22442,   "ctx":"OplogApplier-0","msg":"After 5 seconds of wait there are still sessions left with active coordinators which have not yet completed","attr":{"numSessionsLeft":1}}
      

            Assignee:
            david.chen@mongodb.com David Chen (Inactive)
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: