Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6308

Eliminate redundant HS reads during shutdown

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      After fixing WT-6229, WT still issues hundreds of repeated reads of the same data from the HS file during shutdown.  This extra IO is bad for performance.  

      Although I detected this problem during shutdown, it may occur during regular checkpoint processing, as well.

      Using the test case Eric Mikie describes in BF-17346, I enabled verbose read debugging during shutdown by setting the WT_VERB_READ flag at the beginning of wt_txn_global_shutdown(). I sorted and counted the resulting messages to see how many times WT issues reads to the same block.  The following reads from WiredTigerHS.wt each occurred 600+ times:

      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=94208, len=12288
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=28672, len=12288
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=16384, len=12288
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=86016, len=8192
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=69632, len=16384
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=40960, len=12288
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=53248, len=16384
      file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
      /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=4096, len=12288
      

      This test uses a 1GB WT cache size and the total size of its data files is ~5MB. So it is unlikely that these reads are caused by capacity misses in the cache.  Rather, it seems like there are additional cases (similar to WT-6229) where WT evicts HS pages that should remain in the cache.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            keith.smith@mongodb.com Keith Smith
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: