Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-48531

3 way deadlock can happen between chunk splitter, prepared transactions and stepdown thread.

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v4.4, v4.2
    • Sprint:
      Sharding 2020-06-15, Sharding 2020-06-29
    • Case:

      Description

      Currently step down kills all conflicting user operations and some internal operations that are marked killable using setSystemOperationKillable.

      • Write operation that takes global lock in IX and X mode.
      • Read operations that takes global lock in S mode.
      • Operations(read/write) that are blocked on prepare conflict.

      Step down hangs due to below three way deadlock

      1. Chunk splitter thread (_runAutosplit) performs read by holding RSTL in IX mode and is blocked by a prepared txn due to prepare conflict. ChunkSplitter internal threads are not marked killable. So, step down won't be able to kill/interrupt those internal read operations.
      2. Step down enqueues RSTL lock in X mode. And blocked behind chunk splitter internal thread.
      3. CommitTransaction cmd is waiting for RSTL lock to acquire in IX mode but blocked behind the step down thread.

        Attachments

          Activity

            People

            Assignee:
            kshitij.gupta Kshitij Gupta (Inactive)
            Reporter:
            suganthi.mani Suganthi Mani
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: