Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Duplicate
-
None
-
None
-
None
-
None
Description
After fixing WT-6229, WT still issues hundreds of repeated reads of the same data from the HS file during shutdown. This extra IO is bad for performance.
Although I detected this problem during shutdown, it may occur during regular checkpoint processing, as well.
Using the test case Eric Mikie describes in BF-17346, I enabled verbose read debugging during shutdown by setting the WT_VERB_READ flag at the beginning of wt_txn_global_shutdown(). I sorted and counted the resulting messages to see how many times WT issues reads to the same block. The following reads from WiredTigerHS.wt each occurred 600+ times:
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=94208, len=12288 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=28672, len=12288 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=16384, len=12288 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=86016, len=8192 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=69632, len=16384 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=40960, len=12288 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=53248, len=16384 |
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner
|
/backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=4096, len=12288 |
This test uses a 1GB WT cache size and the total size of its data files is ~5MB. So it is unlikely that these reads are caused by capacity misses in the cache. Rather, it seems like there are additional cases (similar to WT-6229) where WT evicts HS pages that should remain in the cache.