Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2910

test_inmem01 can abort due to stuck cache

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.10, 3.3.14
    • Labels:
      None
    • # Replies:
      10
    • Last comment by Customer:
      true

      Description

      See: http://build.wiredtiger.com:8080/job/wiredtiger-test-unit/5201/console

      Outputs:

      ==========
      cache dump
      file:inmem01.wt(<live>): 
      	internal: 11 pages, 0MB, 0/11 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      	leaf: 596 pages, 4MB, 409/187 clean/dirty pages, 0/4 clean/dirty MB, 0MB max page, 0MB max dirty page
      file:WiredTiger.wt(<live>): 
      	internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      	leaf: 1 pages, 0MB, 0/1 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      cache dump: total found = 5MB vs tracked inuse 5MB
      total dirty bytes = 4MB
      ==========
      test_inmem01.test_inmem01.test_insert_over_delete_replace(col) (subunit.RemotedTestCase)
      test_inmem01.test_inmem01.test_insert_over_delete_replace(col) ... ERROR
       
      ======================================================================
      ERROR: test_inmem01.test_inmem01.test_insert_over_delete_replace(col) (subunit.RemotedTestCase)
      test_inmem01.test_inmem01.test_insert_over_delete_replace(col)
      ----------------------------------------------------------------------
      _StringException: lost connection during test 'test_inmem01.test_inmem01.test_insert_over_delete_replace(col)'
       
      ----------------------------------------------------------------------
      Ran 7224 tests in 393.984s
      

      I have a reproducer which can cause the abort due to cache full on inmem, but I'm uncertain if it is the same reproduer - as I have been abortin with only 1MB dirty, rather than 4.

      My reproducer involves setting memory_page_max=4k down from memory_page_max=32k

        Issue Links

          Activity

          Hide
          david.hows David Hows added a comment -

          I've double checked by cherry-picking the recent commit (d318b8a5370f7182beeab4124cee04fdce271608) on top of 09f65fffa7375357264f55f75678540bb7ab5a08. This resolved the issue and made the test run quicker to boot!

          Show
          david.hows David Hows added a comment - I've double checked by cherry-picking the recent commit ( d318b8a5370f7182beeab4124cee04fdce271608 ) on top of 09f65fffa7375357264f55f75678540bb7ab5a08 . This resolved the issue and made the test run quicker to boot!
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2910 When running in-memory, only evict dirty pages. (#3044)

          • Don't relax the clean / dirty requirements when we get aggressive: if application threads are blocked by the dirty limit, evicting a clean page doesn't unblock them so there's no point trying.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2910 When running in-memory, only evict dirty pages. (#3044) Don't relax the clean / dirty requirements when we get aggressive: if application threads are blocked by the dirty limit, evicting a clean page doesn't unblock them so there's no point trying. Sanity check dirty eviction limits: once the cache is full, we should evict both clean and dirty pages. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/d318b8a5370f7182beeab4124cee04fdce271608
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger: 5bc03723a7e77c96b1d5e45a97173654872c727a from branch mongodb-3.4

          ref: deeb0f589a..5bc03723a7
          for: 3.3.13

          WT-2859 Improve statistics macros to not collect stats when statistics=none is set
          WT-2894 Create workload that shows negative scaling when overwhelmed
          WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags
          WT-2902 Enhance eviction to work better with update heavy YCSB load
          WT-2905 dead code: MongoDB Coverity #99881
          WT-2908 Add dry-run support to python unit-test
          WT-2910 test_inmem01 can abort due to stuck cache
          WT-2911 add support for gcc6
          WT-2912 make --enable-strict run on the zseries test box
          WT-2913 stuck cache after latest merge
          WT-2914 Fix link error on OS/X for checksum_init call
          WT-2916 whitespace check not working
          Branch: master
          https://github.com/mongodb/mongo/commit/7693fa59c4470db729d85af99fb9cc3b264fa8c8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 5bc03723a7e77c96b1d5e45a97173654872c727a from branch mongodb-3.4 ref: deeb0f589a..5bc03723a7 for: 3.3.13 WT-2859 Improve statistics macros to not collect stats when statistics=none is set WT-2894 Create workload that shows negative scaling when overwhelmed WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags WT-2902 Enhance eviction to work better with update heavy YCSB load WT-2905 dead code: MongoDB Coverity #99881 WT-2908 Add dry-run support to python unit-test WT-2910 test_inmem01 can abort due to stuck cache WT-2911 add support for gcc6 WT-2912 make --enable-strict run on the zseries test box WT-2913 stuck cache after latest merge WT-2914 Fix link error on OS/X for checksum_init call WT-2916 whitespace check not working Branch: master https://github.com/mongodb/mongo/commit/7693fa59c4470db729d85af99fb9cc3b264fa8c8
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2910 When running in-memory, only evict dirty pages. (#3044)

          • Don't relax the clean / dirty requirements when we get aggressive: if application threads are blocked by the dirty limit, evicting a clean page doesn't unblock them so there's no point trying.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2910 When running in-memory, only evict dirty pages. (#3044) Don't relax the clean / dirty requirements when we get aggressive: if application threads are blocked by the dirty limit, evicting a clean page doesn't unblock them so there's no point trying. Sanity check dirty eviction limits: once the cache is full, we should evict both clean and dirty pages. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/d318b8a5370f7182beeab4124cee04fdce271608
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Import wiredtiger: bb18c439152379fb528af217345be40819b01fc4 from branch mongodb-3.2

          ref: 911c940ada..bb18c43915
          for: 3.2.10

          WT-2859 Improve statistics macros to not collect stats when statistics=none is set
          WT-2894 Create workload that shows negative scaling when overwhelmed
          WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags
          WT-2902 Enhance eviction to work better with update heavy YCSB load
          WT-2905 dead code: MongoDB Coverity #99881
          WT-2908 Add dry-run support to python unit-test
          WT-2910 test_inmem01 can abort due to stuck cache
          WT-2911 add support for gcc6
          WT-2912 make --enable-strict run on the zseries test box
          WT-2913 stuck cache after latest merge
          WT-2914 Fix link error on OS/X for checksum_init call
          WT-2916 whitespace check not working
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/dfca65ea3a32b1e0c832ff7ee2d7b667819ec64d

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Import wiredtiger: bb18c439152379fb528af217345be40819b01fc4 from branch mongodb-3.2 ref: 911c940ada..bb18c43915 for: 3.2.10 WT-2859 Improve statistics macros to not collect stats when statistics=none is set WT-2894 Create workload that shows negative scaling when overwhelmed WT-2900 Add ARM8 build support to WiredTiger and fix ARM CRC assembler tags WT-2902 Enhance eviction to work better with update heavy YCSB load WT-2905 dead code: MongoDB Coverity #99881 WT-2908 Add dry-run support to python unit-test WT-2910 test_inmem01 can abort due to stuck cache WT-2911 add support for gcc6 WT-2912 make --enable-strict run on the zseries test box WT-2913 stuck cache after latest merge WT-2914 Fix link error on OS/X for checksum_init call WT-2916 whitespace check not working Branch: v3.2 https://github.com/mongodb/mongo/commit/dfca65ea3a32b1e0c832ff7ee2d7b667819ec64d

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                35 weeks, 1 day ago
                Date of 1st Reply: