Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2921

Reduce the WT_SESSION hazard_size when possible

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.10, 3.3.15
    • Labels:
      None
    • # Replies:
      7
    • Last comment by Customer:
      true

      Description

      The current hazard pointer implementation in WiredTiger never reduces the number of active hazard pointers in a session from the high water mark. That means that if a session has many hazard pointers out at a point in time, all future walks will include up to the high water mark.

      We could reduce the size in __wt_hazard_clear if the pointer being freed is the last in the list. Or even more simply clear the size in session reset.

        Activity

        Hide
        alexander.gorrod Alexander Gorrod added a comment -

        A potential, untested change:

        diff --git a/src/support/hazard.c b/src/support/hazard.c
        index 8ac8f5f..2597be7 100644
        --- a/src/support/hazard.c
        +++ b/src/support/hazard.c
        @@ -168,7 +168,8 @@ __wt_hazard_clear(WT_SESSION_IMPL *session, WT_PAGE *page)
                                 * If this was the last hazard pointer in the session,
                                 * we may need to update our transactional context.
                                 */
        -                       --session->nhazard;
        +                       if (--session->nhazard == 0)
        +                               WT_PUBLISH(session->hazard_size, 0);
                                return (0);
                        }
        

        Show
        alexander.gorrod Alexander Gorrod added a comment - A potential, untested change: diff --git a/src/support/hazard.c b/src/support/hazard.c index 8ac8f5f..2597be7 100644 --- a/src/support/hazard.c +++ b/src/support/hazard.c @@ -168,7 +168,8 @@ __wt_hazard_clear(WT_SESSION_IMPL *session, WT_PAGE *page) * If this was the last hazard pointer in the session, * we may need to update our transactional context. */ - --session->nhazard; + if (--session->nhazard == 0) + WT_PUBLISH(session->hazard_size, 0); return (0); }
        Hide
        keith.bostic Keith Bostic added a comment - - edited

        Alexander Gorrod, the assumption I've always made was a cursor will never have more than 2 or 3 hazard pointers at any time (lock-coupling up & down the tree). Am I missing a case, or are there workloads where that's not true?

        Alexander Gorrod, I read through your response on HELP-2683, I get it now, no need to reply.

        Show
        keith.bostic Keith Bostic added a comment - - edited Alexander Gorrod , the assumption I've always made was a cursor will never have more than 2 or 3 hazard pointers at any time (lock-coupling up & down the tree). Am I missing a case, or are there workloads where that's not true? Alexander Gorrod , I read through your response on HELP-2683, I get it now, no need to reply.
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058) Branch: develop https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058) Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2921 Reset the count when the last hazard pointer is cleared. (#3058) Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/5a26a11f0da1941b38ca34c3ba2a4e1265392049
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

        Message: Import wiredtiger: 9cf2f89d6d95e1de797f05ab1fef28695f8bae7b from branch mongodb-3.2

        ref: bb18c43915..9cf2f89d6d
        for: 3.2.10

        WT-2864 Reconfiguring the checkpoint server can lead to hangs
        WT-2874 Change test_compact01 to avoid eviction
        WT-2918 The dist scripts create C files s_whitespace complains about
        WT-2919 Don't mask error returns from style checking scripts
        WT-2921 Reduce the WT_SESSION hazard_size when possible
        WT-2923 heap-use-after-free on address in compaction
        WT-2924 Ensure we are doing eviction when threads are waiting for it
        WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag
        WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock
        WT-2928 Eviction failing to switch queues can lead to starvation
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/79d9b3ab5ce20f51c272b4411202710a082d0317

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger: 9cf2f89d6d95e1de797f05ab1fef28695f8bae7b from branch mongodb-3.2 ref: bb18c43915..9cf2f89d6d for: 3.2.10 WT-2864 Reconfiguring the checkpoint server can lead to hangs WT-2874 Change test_compact01 to avoid eviction WT-2918 The dist scripts create C files s_whitespace complains about WT-2919 Don't mask error returns from style checking scripts WT-2921 Reduce the WT_SESSION hazard_size when possible WT-2923 heap-use-after-free on address in compaction WT-2924 Ensure we are doing eviction when threads are waiting for it WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock WT-2928 Eviction failing to switch queues can lead to starvation Branch: v3.2 https://github.com/mongodb/mongo/commit/79d9b3ab5ce20f51c272b4411202710a082d0317
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

        Message: Import wiredtiger: fc0e7abe82595e579573d42448632f7b36a2d154 from branch mongodb-3.4

        ref: 5bc03723a7..fc0e7abe82
        for: 3.3.15

        WT-2864 Reconfiguring the checkpoint server can lead to hangs
        WT-2874 Change test_compact01 to avoid eviction
        WT-2918 The dist scripts create C files s_whitespace complains about
        WT-2919 Don't mask error returns from style checking scripts
        WT-2921 Reduce the WT_SESSION hazard_size when possible
        WT-2923 heap-use-after-free on address in compaction
        WT-2924 Ensure we are doing eviction when threads are waiting for it
        WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag
        WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock
        WT-2928 Eviction failing to switch queues can lead to starvation
        Branch: master
        https://github.com/mongodb/mongo/commit/9dda827a3ae58beef36d53da1b55554cbd8744c4

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger: fc0e7abe82595e579573d42448632f7b36a2d154 from branch mongodb-3.4 ref: 5bc03723a7..fc0e7abe82 for: 3.3.15 WT-2864 Reconfiguring the checkpoint server can lead to hangs WT-2874 Change test_compact01 to avoid eviction WT-2918 The dist scripts create C files s_whitespace complains about WT-2919 Don't mask error returns from style checking scripts WT-2921 Reduce the WT_SESSION hazard_size when possible WT-2923 heap-use-after-free on address in compaction WT-2924 Ensure we are doing eviction when threads are waiting for it WT-2925 WT_THREAD_PANIC_FAIL is a WT_THREAD structure flag WT-2926 WT_CONNECTION.reconfigure can attempt unlock of not-locked lock WT-2928 Eviction failing to switch queues can lead to starvation Branch: master https://github.com/mongodb/mongo/commit/9dda827a3ae58beef36d53da1b55554cbd8744c4

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              30 weeks, 1 day ago
              Date of 1st Reply: