Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33191

Cache-full hangs on 3.6

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 3.6.2, 3.6.5
    • 3.6.7, 4.0.1, 4.1.2
    • Storage
    • None
    • Fully Compatible
    • ALL
    • v4.0, v3.6
    • Storage NYC 2018-07-16, Storage NYC 2018-07-30

    Description

      We have seen a couple of cases in the field where under heavy cache pressure a node can get stuck with the WT cache full. In the cases seen so far the cache pressure has been due to replica set lag, which causes the majority commit point to lag, creating cache pressure.

      So far we don't have a reproducer and don't have FTDC data covering the inception of the issue (SERVER-32876) so this ticket for now is a placeholder to gather information.

      Attachments

        1. metrics.2018-05-30T17-31-46Z-00000
          59 kB
        2. repro-10MBx2.sh
          1 kB
        3. stacks.txt
          65 kB
        4. stuck.png
          stuck.png
          166 kB

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. SERVER-32876 Don't stall ftdc due to WT cache full

          • Critical - P2 - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. WT-4105 Optimize cache usage for update workload with history pinned

          • Major - P3 - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-36496 Cache pressure issues during oplog replay in initial sync

          • Major - P3 - Major loss of function.
          • Backlog

          Bug - A problem which impairs or prevents the functions of the product. SERVER-34938 Secondary slowdown or hang due to content pinned in cache by single oplog batch

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-34941 Add testing to cover cases where timestamps cause cache pressure

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-34942 Stuck with cache full during oplog replay in initial sync

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-36495 Cache pressure issues during recovery oplog application

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-37849 Poor replication performance and cache-full hang on secondary due to pinned content

          • Major - P3 - Major loss of function.
          • Backlog

          Bug - A problem which impairs or prevents the functions of the product. SERVER-35103 Checkpoint creates unevictable clean content

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-36221 [3.6] Performance regression on small updates to large documents

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-36238 replica set startup fails in wt_cache_full.js, initial_sync_wt_cache_full.js, recovery_wt_cache_full.js when journaling is disabled

          • Minor - P4 - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Task - A task that needs to be done. SERVER-36373 create test to fill WT cache during steady state replication

          • Major - P3 - Major loss of function.
          • Closed

          Task - A task that needs to be done. WT-4106 Optimize lookaside implementation to the point where I/O can be saturated

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. WT-4107 Reduce lookaside usage in update heavy workloads where history is pinned

          • Major - P3 - Major loss of function.
          • Closed
          mentioned in

          Page Loading...

          Activity

            People

              benety.goh@mongodb.com Benety Goh
              bruce.lucas@mongodb.com Bruce Lucas (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: