Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29217

request stuck when dirty cache reach 20%

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.10
    • Component/s: WiredTiger
    • Labels:
      None

      mongostat data shown as belows

      insert query update delete getmore command % dirty % used flushes vsize   res   qr|qw   ar|aw netIn netOut conn           set repl                      time
          *0    *0     *0     *0       0    12|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  984b    27k    0 mgset-2080137  PRI 2017-05-15T15:21:29+08:00
          *0    *0     *0     *0       0     3|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  191b    22k    0 mgset-2080137  PRI 2017-05-15T15:21:30+08:00
          *0    *0     *0     *0       0     5|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  530b    24k    0 mgset-2080137  PRI 2017-05-15T15:21:31+08:00
          *0    *0     *0     *0       0     3|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  192b    22k    0 mgset-2080137  PRI 2017-05-15T15:21:32+08:00
          *0    *0     *0     *0       0     5|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  532b    23k    0 mgset-2080137  PRI 2017-05-15T15:21:33+08:00
          *0    *0     *0     *0       0     8|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  706b    24k    0 mgset-2080137  PRI 2017-05-15T15:21:34+08:00
          *0    *0     *0     *0       0     6|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  648b    25k    0 mgset-2080137  PRI 2017-05-15T15:21:35+08:00
          *0    *0     *0     *0       0    25|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128    2k    33k    0 mgset-2080137  PRI 2017-05-15T15:21:36+08:00
          *0    *0     *0     *0       0     4|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  527b    23k    0 mgset-2080137  PRI 2017-05-15T15:21:37+08:00
          *0    *0     *0     *0       0     2|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  133b    22k    0 mgset-2080137  PRI 2017-05-15T15:21:38+08:00
      insert query update delete getmore command % dirty % used flushes vsize   res   qr|qw   ar|aw netIn netOut conn           set repl                      time
          *0    *0     *0     *0       0     8|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  698b    25k    0 mgset-2080137  PRI 2017-05-15T15:21:39+08:00
          *0    *0     *0     *0       0     4|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  354b    23k    0 mgset-2080137  PRI 2017-05-15T15:21:40+08:00
          *0    *0     *0     *0       0    11|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128    1k    27k    0 mgset-2080137  PRI 2017-05-15T15:21:41+08:00
          *0    *0     *0     *0       0     2|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  134b    22k    0 mgset-2080137  PRI 2017-05-15T15:21:42+08:00
          *0    *0     *0     *0       0     5|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  532b    23k    0 mgset-2080137  PRI 2017-05-15T15:21:43+08:00
          *0    *0     *0     *0       0     3|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  189b    22k    0 mgset-2080137  PRI 2017-05-15T15:21:44+08:00
          *0    *0     *0     *0       0     4|0    20.1   79.6       0 24.9G 16.3G 1105|34 128|128  474b    23k    0 mgset-2080137  PRI 2017-05-15T15:21:45+08:00
      
      • Through the pstack result,I find that __wt_evict_thread_run never work even though dirty cache ratio reached 20%.
      • After I changed eviction_dirty_trigger to 30, the server recovered to normal state immediately.
      
      db.runCommand({setParameter: 1, wiredTigerEngineRuntimeConfig: "eviction_dirty_trigger=30"})db.runCommand({setParameter: 1, wiredTigerEngineRuntimeConfig: "eviction_dirty_trigger=30"})
      

            Assignee:
            kelsey.schubert@mongodb.com Kelsey Schubert
            Reporter:
            zyd_com@126.com Zhang Youdong
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: