Details
Improvement
Major - P3
Description
The current hazard pointer implementation in WiredTiger never reduces the number of active hazard pointers in a session from the high water mark. That means that if a session has many hazard pointers out at a point in time, all future walks will include up to the high water mark.
We could reduce the size in __wt_hazard_clear if the pointer being freed is the last in the list. Or even more simply clear the size in session reset.
Activity
Alexander Gorrod, the assumption I've always made was a cursor will never have more than 2 or 3 hazard pointers at any time (lock-coupling up & down the tree). Am I missing a case, or are there workloads where that's not true?
Alexander Gorrod, I read through your response on HELP-2683, I get it now, no need to reply.
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
Author:
{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger: 9cf2f89d6d95e1de797f05ab1fef28695f8bae7b from branch mongodb-3.2
ref: bb18c43915..9cf2f89d6d
for: 3.2.10
WT-2864 Reconfiguring the checkpoint server can lead to hangs
WT-2874 Change test_compact01 to avoid eviction
WT-2918 The dist scripts create C files s_whitespace complains about
WT-2919 Don't mask error returns from style checking scripts
WT-2921 Reduce the WT_SESSION hazard_size when possible
WT-2923 heap-use-after-free on address in compaction
WT-2924 Ensure we are doing eviction when threads are waiting for it
WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag
WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock
WT-2928 Eviction failing to switch queues can lead to starvation
Branch: v3.2
https://github.com/mongodb/mongo/commit/79d9b3ab5ce20f51c272b4411202710a082d0317
Author:
{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger: fc0e7abe82595e579573d42448632f7b36a2d154 from branch mongodb-3.4
ref: 5bc03723a7..fc0e7abe82
for: 3.3.15
WT-2864 Reconfiguring the checkpoint server can lead to hangs
WT-2874 Change test_compact01 to avoid eviction
WT-2918 The dist scripts create C files s_whitespace complains about
WT-2919 Don't mask error returns from style checking scripts
WT-2921 Reduce the WT_SESSION hazard_size when possible
WT-2923 heap-use-after-free on address in compaction
WT-2924 Ensure we are doing eviction when threads are waiting for it
WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag
WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock
WT-2928 Eviction failing to switch queues can lead to starvation
Branch: master
https://github.com/mongodb/mongo/commit/9dda827a3ae58beef36d53da1b55554cbd8744c4
A potential, untested change:
diff --git a/src/support/hazard.c b/src/support/hazard.cindex 8ac8f5f..2597be7 100644--- a/src/support/hazard.c+++ b/src/support/hazard.c@@ -168,7 +168,8 @@ __wt_hazard_clear(WT_SESSION_IMPL *session, WT_PAGE *page)* If this was the last hazard pointer in the session,* we may need to update our transactional context.*/- --session->nhazard;+ if (--session->nhazard == 0)+ WT_PUBLISH(session->hazard_size, 0);return (0);}