Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22964

IX GlobalLock being held while wating for wt cache eviction

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      Linux
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      1. setup a two node replSet using the attached mongod.conf(only specify storage related)
      2. run test.sh
      3. It will hang after running the script for about a long time(2 days+)

      Show
      1. setup a two node replSet using the attached mongod.conf(only specify storage related) 2. run test.sh 3. It will hang after running the script for about a long time(2 days+)

      Description

      hi,
      I'm currently doing a pressure test for 3.2.3 using the attached test script. The testing environment is a two nodes replSet using WiredTiger. After running the script for long time(about 2 days). The mongod process will hang. As you can see in the attached thread stack, a lot of threads were waiting for the GlobalLock which was being held by someone(probably the thread doing profile, i.e. Thread 501). Looks like the thread locked it with MODE_IX and was doing a cappedDelete operation, making room for new profiling document. But it seems that it was waiting on WT's cache evict_waiter_cond and hangs there forever.

        Attachments

        1. diags1.tgz
          17.16 MB
        2. global_lock_hang.ts
          1.21 MB
        3. mongod.conf
          0.3 kB
        4. test.sh
          0.8 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: