Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17416

Deadlock between MMAP V1 journal lock and oplog collection lock

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-rc11
    • Fix Version/s: 3.0.1, 3.1.0
    • Component/s: Concurrency, MMAPv1, Storage
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:

      Description

      It is possible to construct sequence of concurrent operations on multiple database in MMAP V1, which would result in deadlock, which is unresolvable by our current deadlock handling code.

      The deadlock cycle is below and the thread stacks are attached.

      Locker 4 (Thread 5) waits for resource {4611686018427387906: MMAPV1Journal, 2} held by [17337, 17340, ]
      	* {4611686018427387906: MMAPV1Journal, 2} held in S
       
      Locker 17337 (Thread 17343)  waits for resource {10558654665682534615: Collection, 1335282628827758807} held by [17328, ]
      	{2305843009213693953: Global, 1} held in IX
      	{4611686018427387906: MMAPV1Journal, 2} held in IX
      	{8810581537292689075: Database, 1893052509651607219} held in IX
      	{9643129350232766918: Collection, 419757313377991110} held in X
      	{8576409733318454219: Database, 1658880705677372363} held in IX
      	* {10558654665682534615: Collection, 1335282628827758807} held in X
       
      Locker 17340 (Thread 17346)  waits for resource {10558654665682534615: Collection, 1335282628827758807} held by [17328, 17337, ]
      	{2305843009213693953: Global, 1} held in IX
      	{4611686018427387906: MMAPV1Journal, 2} held in IX
      	{9033119116019004327: Database, 2115590088377922471} held in X
      	{13644805134446392231: Metadata, 2115590088377922471} held in X
      	{8576409733318454219: Database, 1658880705677372363} held in IX
      	* {10558654665682534615: Collection, 1335282628827758807} held in X
       
      Locker 17328 (Thread 17334)  waits for resource {4611686018427387906: MMAPV1Journal, 2} held by [4, ]
      	{2305843009213693953: Global, 1} held in IX
      	* {4611686018427387906: MMAPV1Journal, 2} held in IX
      	{8576409733318454219: Database, 1658880705677372363} held in IX
      	{10558654665682534615: Collection, 1335282628827758807} held in X

      1. stacks.txt
        14 kB
        Kaloian Manassiev

        Activity

        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

        Message: SERVER-17416 MMAP V1 journal thread should check for deadlocks

        Also improve the reporting of the Locker::dump function.
        Branch: master
        https://github.com/mongodb/mongo/commit/197cdebe7de9be23906c175b31f056c0351e9eb3

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'} Message: SERVER-17416 MMAP V1 journal thread should check for deadlocks Also improve the reporting of the Locker::dump function. Branch: master https://github.com/mongodb/mongo/commit/197cdebe7de9be23906c175b31f056c0351e9eb3
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

        Message: SERVER-17416 MMAP V1 journal thread should check for deadlocks

        Also improve the reporting of the Locker::dump function.

        (cherry picked from commit 197cdebe7de9be23906c175b31f056c0351e9eb3)

        Conflicts:
        src/mongo/db/repl/oplog.cpp
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/3d2e6a86bbd884b1723b5dda1c8a1d2e32015174

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'} Message: SERVER-17416 MMAP V1 journal thread should check for deadlocks Also improve the reporting of the Locker::dump function. (cherry picked from commit 197cdebe7de9be23906c175b31f056c0351e9eb3) Conflicts: src/mongo/db/repl/oplog.cpp Branch: v3.0 https://github.com/mongodb/mongo/commit/3d2e6a86bbd884b1723b5dda1c8a1d2e32015174

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: