Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2955

Add statistics tracking the amount of time threads spend waiting for high level locks

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.11, 3.4.0-rc2
    • Labels:
      None
    • # Replies:
      9
    • Last comment by Customer:
      true
    • Sprint:
      Storage 2016-10-31

      Description

      Add statistics tracking the amount of time threads spend waiting for the dhandle list lock and possibly other high level locks in WiredTiger.

      Specific lock acquire counts might also be interesting. The full set of locks I think are worth considering are:

      • Checkpoint lock (WT_CONNECTION::checkpoint_lock)
      • Handle list lock (WT_CONNECTION::dhandle_lock)
      • Metadata lock (WT_CONNECTION::metadata_lock)
      • Schema lock (WT_CONNECTION::schema_lock)
      • Table lock (WT_CONNECTION::table_lock)

      I'm inclined to add a pair of statistics for each lock type, similar to:

          LockStat('lock_dhandle_count', 'Number of times handle lock was acquired'),
          LockStat('lock_dhandle_wait_time', 'Amount of time waiting to acquire handle lock'),
      

      It would also be interesting to distinguish between internal and application threads, but that would lead to a lot of new statistics.

      These statistics would be gathered as part of fast statistics, though we want to be careful re: adding gettime calls around lock acquisition. If these locks are acquired regularly it may be worthwhile not gathering the stats for performance reasons.

        Issue Links

          Activity

          Hide
          keith.bostic Keith Bostic added a comment -

          There's an open question of whether adding get-time-of-day calls into performance paths might make a difference: we believe it won't be a problem, based on the work in WT-2663.

          Show
          keith.bostic Keith Bostic added a comment - There's an open question of whether adding get-time-of-day calls into performance paths might make a difference: we believe it won't be a problem, based on the work in WT-2663 .
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger: ef9a7983ea47cea78400a4472a3d4e46735385c5 from branch mongodb-3.4

          ref: 6a31c2118c..ef9a7983ea
          for: 3.4.0-rc2

          WT-1592 Add ability to dump detailed cache information via statistics
          WT-2403 Enhance random cursor implementation for LSM trees
          WT-2880 Add support for Zstandard compression
          WT-2904 Fix a bug where the reported checkpoint size could be many times data size
          WT-2949 Add an option to wtperf to not close connection on shutdown
          WT-2954 Inserting multi-megabyte values can cause large in-memory pages
          WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
          WT-2956 utility tests -h option is always overridden by the default setup
          WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized
          WT-2963 Race setting max_entries during eviction
          WT-2965 test_wt2323_join_visibility can hang on OSX
          WT-2974 lint
          WT-2976 Add a statistic tracking how long application threads spend doing I/O
          WT-2977 Csuite LSM Random test can occasionally fail
          WT-2985 Race during checkpoint can cause a core dump
          WT-2987 Fix a bug where opening a cursor on an incomplete table drops core
          WT-2988 Fix a bug where __wt_epoch potentially returns garbage values.
          Branch: master
          https://github.com/mongodb/mongo/commit/0609d0ce2ef563d7a4cde77d46396fe5c92c6df1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: ef9a7983ea47cea78400a4472a3d4e46735385c5 from branch mongodb-3.4 ref: 6a31c2118c..ef9a7983ea for: 3.4.0-rc2 WT-1592 Add ability to dump detailed cache information via statistics WT-2403 Enhance random cursor implementation for LSM trees WT-2880 Add support for Zstandard compression WT-2904 Fix a bug where the reported checkpoint size could be many times data size WT-2949 Add an option to wtperf to not close connection on shutdown WT-2954 Inserting multi-megabyte values can cause large in-memory pages WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks WT-2956 utility tests -h option is always overridden by the default setup WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized WT-2963 Race setting max_entries during eviction WT-2965 test_wt2323_join_visibility can hang on OSX WT-2974 lint WT-2976 Add a statistic tracking how long application threads spend doing I/O WT-2977 Csuite LSM Random test can occasionally fail WT-2985 Race during checkpoint can cause a core dump WT-2987 Fix a bug where opening a cursor on an incomplete table drops core WT-2988 Fix a bug where __wt_epoch potentially returns garbage values. Branch: master https://github.com/mongodb/mongo/commit/0609d0ce2ef563d7a4cde77d46396fe5c92c6df1
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086)

          • WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks

          Sort the statistics categories so it's easier to find stuff, no real change.

          • Add counters and usec wait times to long-term locks (currently the
            checkpoint, handle-list, metadata, schema and table locks).
          • mutex.i:295:26: error: conversion to int64_t {aka long int}

            from long
            unsigned int may change the sign of the result [-Werror=sign-conversion]
            [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);

          • Rename the lock statistics so they group together.

          Split lock wait times between internal and application threads.

          • Separate the connection's dummy session initialization out into its own
            function, that way it's clear what we're doing.
          • The session's connection statistics are fixed when the session ID is
            allocated, so we can cache a pointer to them and avoid u_int divisions
            (which are currently about the slowest thing you can do on a modern
            architecture).
          • A slightly different change: instead of caching a reference to the
            connection statistics, cache the offset into the array of statistics
            pointers, that way we can avoid the integer division when updating
            almost all statistics.
          • Review comments:

          Add comments describing the use of statistics array offsets in lock
          tracking.

          Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.

          Whitespace cleanup.

          • __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
            if the connection open fails at any point, __wt_cache_destroy() will be
            called as part of that cleanup.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks (#3086) WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); Rename the lock statistics so they group together. Split lock wait times between internal and application threads. Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/13c4e64a162abdcdd19afa29cec52a8fe57977ff
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2

          ref: 9cf2f89d6d..b11ed312ce
          for: 3.2.11

          WT-1592 Dump detailed cache information via statistics
          WT-2403 Enhance random cursor implementation for LSM trees
          WT-2831 Skip creating a checkpoint if there have been no changes
          WT-2858 rename wtperf's CONFIG structure
          WT-2880 Add support for Zstandard compression
          WT-2895 Reduce the runtime of make check testing with disable long
          WT-2904 Fix a bug where the reported checkpoint size could be many times data size
          WT-2907 Bug in Java ConcurrentCloseTest case
          WT-2917 split wtperf's configuration into per-database and per-run parts
          WT-2920 Add statistic tracking application thread cache maintenance time
          WT-2931 Configure default in-memory dirty cache usage lower
          WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
          WT-2933 Fix a race between named snapshots and checkpoints
          WT-2937 test_inmem01 aborts due to stuck cache
          WT-2938 Assembly files should end in .sx, not .S
          WT-2941 Improve test/format to use faster key-generation functions
          WT-2942 verbose strings don't need newline
          WT-2946 dist/s_docs incompatible with OS X Xcode installation
          WT-2948 simplify error handling by making epoch time return never fail
          WT-2949 Add an option to wtperf to not close connection on shutdown
          WT-2950 Inserting multi-megabyte values can cause large in-memory pages
          WT-2954 Inserting multi-megabyte values can cause large in-memory pages
          WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
          WT-2956 utility tests -h option is always overridden by the default setup
          WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized
          WT-2963 Race setting max_entries during eviction
          WT-2965 test_wt2323_join_visibility can hang on OSX
          WT-2974 lint
          WT-2976 Add a statistic tracking how long application threads spend doing I/O
          WT-2977 Csuite LSM Random test can occasionally fail
          WT-2985 Race during checkpoint can cause a core dump
          WT-2987 Fix a bug where opening a cursor on an incomplete table drops core
          WT-2988 __wt_epoch potentially returns garbage values.
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2 ref: 9cf2f89d6d..b11ed312ce for: 3.2.11 WT-1592 Dump detailed cache information via statistics WT-2403 Enhance random cursor implementation for LSM trees WT-2831 Skip creating a checkpoint if there have been no changes WT-2858 rename wtperf's CONFIG structure WT-2880 Add support for Zstandard compression WT-2895 Reduce the runtime of make check testing with disable long WT-2904 Fix a bug where the reported checkpoint size could be many times data size WT-2907 Bug in Java ConcurrentCloseTest case WT-2917 split wtperf's configuration into per-database and per-run parts WT-2920 Add statistic tracking application thread cache maintenance time WT-2931 Configure default in-memory dirty cache usage lower WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration WT-2933 Fix a race between named snapshots and checkpoints WT-2937 test_inmem01 aborts due to stuck cache WT-2938 Assembly files should end in .sx, not .S WT-2941 Improve test/format to use faster key-generation functions WT-2942 verbose strings don't need newline WT-2946 dist/s_docs incompatible with OS X Xcode installation WT-2948 simplify error handling by making epoch time return never fail WT-2949 Add an option to wtperf to not close connection on shutdown WT-2950 Inserting multi-megabyte values can cause large in-memory pages WT-2954 Inserting multi-megabyte values can cause large in-memory pages WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks WT-2956 utility tests -h option is always overridden by the default setup WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized WT-2963 Race setting max_entries during eviction WT-2965 test_wt2323_join_visibility can hang on OSX WT-2974 lint WT-2976 Add a statistic tracking how long application threads spend doing I/O WT-2977 Csuite LSM Random test can occasionally fail WT-2985 Race during checkpoint can cause a core dump WT-2987 Fix a bug where opening a cursor on an incomplete table drops core WT-2988 __wt_epoch potentially returns garbage values. Branch: v3.2 https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                35 weeks ago
                Date of 1st Reply:

                  Agile