Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2693

Check open_cursor error paths for consistent handling

    Details

    • # Replies:
      9
    • Last comment by Customer:
      true

      Description

      In debugging WT-2689, I noticed that the error handling when opening cursors could be improved. Several files specifically say the call to wt_cursor_init is last to make error handling easier. But those that have additional calls do not have similar handling often.

        Issue Links

          Activity

          Hide
          sue.loverso Sue LoVerso added a comment -

          In particular, the handling of cursorp after it comes back from wt_cursor_init. Often it is equal to the cursor passed in except when it is a dump cursor. Here's a list of callers of wt_cursor_init and the error handling:

          • cur_backup.c:148: Call is last so error handling only needs a free.
          • cur_config.c:62: Call is last so error handling only needs a free.
          • cur_ds.c:498: Potential errors both before and after wt_cursor_init and open_cursor. Error handling needs careful study.
          • cur_dump.c:407: Call is last so error handling only needs a free.
          • cur_file.c:482: Call is last so error handling only needs a free.
          • cur_index.c:472: Potential errors both before and after wt_cursor_init and open_cursor. Error handling needs careful study.
          • cur_join.c:1337: Call is last but error handling does a close and not a free. Error handling needs review.
          • cur_log.c:394: Call is not last. Error handling needs review.
          • cur_metadata.c:537: Call is not last. Error handling needs review.
          • cur_stat.c:706: Call is last so error handling only needs a free.
          • cur_table.c:932: Potential errors both before and after wt_cursor_init and open_cursor. Error handling needs careful study.
          Show
          sue.loverso Sue LoVerso added a comment - In particular, the handling of cursorp after it comes back from wt_cursor_init . Often it is equal to the cursor passed in except when it is a dump cursor. Here's a list of callers of wt_cursor_init and the error handling: cur_backup.c:148: Call is last so error handling only needs a free. cur_config.c:62: Call is last so error handling only needs a free. cur_ds.c:498: Potential errors both before and after wt_cursor_init and open_cursor . Error handling needs careful study. cur_dump.c:407: Call is last so error handling only needs a free. cur_file.c:482: Call is last so error handling only needs a free. cur_index.c:472: Potential errors both before and after wt_cursor_init and open_cursor . Error handling needs careful study. cur_join.c:1337: Call is last but error handling does a close and not a free. Error handling needs review. cur_log.c:394: Call is not last. Error handling needs review. cur_metadata.c:537: Call is not last. Error handling needs review. cur_stat.c:706: Call is last so error handling only needs a free. cur_table.c:932: Potential errors both before and after wt_cursor_init and open_cursor . Error handling needs careful study.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4

          ref: d8fb874..a9e9696
          for: 3.3.11

          SERVER-24971 Excessive memory held by sessions when application threads do evictions
          WT-1162 Add latency to Jenkins wtperf tests and plots
          WT-2026 Maximum pages size at eviction too large
          WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
          WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
          WT-2380 Make scripts fail if code doesn't match style
          WT-2486 Update make check so that it runs faster
          WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
          WT-2648 cache-line alignment for new ports
          WT-2665 Limit allocator fragmentation in WiredTiger
          WT-2693 Check open_cursor error paths for consistent handling
          WT-2708 split child-update race with reconciliation/eviction
          WT-2711 Change statistics log configuration options
          WT-2728 Don't re-read log file headers during log_flush
          WT-2729 Focus eviction walks in largest trees
          WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
          WT-2731 Raw compression can create pages that are larger than expected
          WT-2732 Coverity analysis defect 99665: Redundant test
          WT-2737 Scrub dirty pages rather than evicting them
          WT-2738 Remove the ability to change the default checkpoint name
          WT-2739 pluggable file systems documentation cleanups
          WT-2743 Thread count statistics always report 0
          WT-2744 partial line even with line buffering set
          WT-2746 track checkpoint I/O separately from eviction I/O
          WT-2751 column-store statistics incorrectly calculates the number of entries
          WT-2752 Fixes to zipfian wtperf workload config
          WT-2755 flexelint configuration treats size_t as 4B type
          WT-2756 Upgrade the autoconf archive package to check for swig 3.0
          WT-2757 Column tables behave differently when column names are provided
          WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
          WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
          WT-2762 wtstats tool fails if checkpoint runs
          WT-2763 Unit test test_intpack failing on OSX
          WT-2764 Optimize checkpoints to reduce throughput disruption
          WT-2765 wt dump: indices need to be shown in the dump output
          WT-2767 test suite needs way to run an individual scenario
          WT-2769 Update documentation to reflect correct limits of memory_page_max
          WT-2770 Add statistics tracking schema operations
          WT-2772 Investigate log performance testing weirdness
          WT-2773 search_near in indexes does not find exact matches
          WT-2774 minor cleanups/improvements
          WT-2778 Python test suite: make scenario initialization consistent
          WT-2779 Raw compression created unexpectedly large pages on disk
          WT-2781 Enhance bulk cursor option with an option to return immediately on contention
          WT-2782 Missing a fs_directory_list_free in ex_file_system.c
          WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
          WT-2791 Enhance OS X Evergreen unit test
          WT-2793 wtperf config improvements
          WT-2796 Memory leak in reconciliation uncovered by stress testing
          WT-2798 Crash vulnerability with nojournal after create during checkpoint
          WT-2800 Illegal file format in test/format on PPC
          WT-2801 Crash vulnerability from eviction of metadata during checkpoint
          WT-2802 Transaction commit causes heap-use-after free
          WT-2803 Add verbose functionality to WT Evergreen tests
          WT-2804 Don't read values in a tree without a snapshot
          WT-2805 Infinite recursion if error streams fail
          WT-2806 wtperf allocation size off-by-one
          Branch: master
          https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4 ref: d8fb874..a9e9696 for: 3.3.11 SERVER-24971 Excessive memory held by sessions when application threads do evictions WT-1162 Add latency to Jenkins wtperf tests and plots WT-2026 Maximum pages size at eviction too large WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang WT-2380 Make scripts fail if code doesn't match style WT-2486 Update make check so that it runs faster WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros WT-2648 cache-line alignment for new ports WT-2665 Limit allocator fragmentation in WiredTiger WT-2693 Check open_cursor error paths for consistent handling WT-2708 split child-update race with reconciliation/eviction WT-2711 Change statistics log configuration options WT-2728 Don't re-read log file headers during log_flush WT-2729 Focus eviction walks in largest trees WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary WT-2731 Raw compression can create pages that are larger than expected WT-2732 Coverity analysis defect 99665: Redundant test WT-2737 Scrub dirty pages rather than evicting them WT-2738 Remove the ability to change the default checkpoint name WT-2739 pluggable file systems documentation cleanups WT-2743 Thread count statistics always report 0 WT-2744 partial line even with line buffering set WT-2746 track checkpoint I/O separately from eviction I/O WT-2751 column-store statistics incorrectly calculates the number of entries WT-2752 Fixes to zipfian wtperf workload config WT-2755 flexelint configuration treats size_t as 4B type WT-2756 Upgrade the autoconf archive package to check for swig 3.0 WT-2757 Column tables behave differently when column names are provided WT-2759 Releasing the hot-backup lock doesn't require the schema lock. WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default WT-2762 wtstats tool fails if checkpoint runs WT-2763 Unit test test_intpack failing on OSX WT-2764 Optimize checkpoints to reduce throughput disruption WT-2765 wt dump: indices need to be shown in the dump output WT-2767 test suite needs way to run an individual scenario WT-2769 Update documentation to reflect correct limits of memory_page_max WT-2770 Add statistics tracking schema operations WT-2772 Investigate log performance testing weirdness WT-2773 search_near in indexes does not find exact matches WT-2774 minor cleanups/improvements WT-2778 Python test suite: make scenario initialization consistent WT-2779 Raw compression created unexpectedly large pages on disk WT-2781 Enhance bulk cursor option with an option to return immediately on contention WT-2782 Missing a fs_directory_list_free in ex_file_system.c WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation WT-2791 Enhance OS X Evergreen unit test WT-2793 wtperf config improvements WT-2796 Memory leak in reconciliation uncovered by stress testing WT-2798 Crash vulnerability with nojournal after create during checkpoint WT-2800 Illegal file format in test/format on PPC WT-2801 Crash vulnerability from eviction of metadata during checkpoint WT-2802 Transaction commit causes heap-use-after free WT-2803 Add verbose functionality to WT Evergreen tests WT-2804 Don't read values in a tree without a snapshot WT-2805 Infinite recursion if error streams fail WT-2806 wtperf allocation size off-by-one Branch: master https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2693 Check open_cursor error paths for consistent handling (#2859)

          • WT-2693 Check open_cursor error paths for consistent handling

          There's a problem with cursor functions which don't call their typed
          close functions, relying instead on a simple free of the base cursor
          type because the __wt_cursor_init() function is the last function called
          before successful return. The problem is __wt_cursor_init() doesn't
          clean up after itself: for example, it allocates memory to hold the URI
          in WT_CURSOR.internal_uri, then has a number of subsequent potential
          error returns, and WT_CURSOR.internal_uri won't be free'd unless
          __wt_cursor_close() is called. In other words, we should always call
          the typed closed function (which in turn calls __wt_cursor_close()).

          Specific additional concerns:

          cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or
          clear the hot-backup flag if __wt_cursor_init() fails, which will hang
          the process. Remove the cleanup-on-error code from the __backup_start()
          function and rely instead on the backup cursor's close function to do
          the work. There's a problem if we fail the initial backup cursor test,
          in which case we don't want to do cleanup other than discarding the
          cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the
          WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot
          backup cursor flag, and use it to figure out how much cleanup needs to
          be done.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile
          has been initialized, change it to check for NULL.

          cursor/cur_file.c: the change to call the __curfile_close() function has
          a problem with incrementing the data-source's in-use counter. Instead
          of incrementing the counter in __wt_curfile_open() before calling
          __curfile_create(), increment it in __curfile_create() after the point
          where all failure paths close the cursor.

          cursor/cur_log.c: the change to call the __curlog_close() function means
          we may be required to clean up a cursor that has not yet acquired the
          log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG
          structure so the close function knows if the log archive lock needs to be
          released.

          cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(),
          we're closing the cursor, resetting it doesn't do anything useful.

          cursor/cur_metadata.c: the change to call the __curmetadata_close()
          function is slightly tricky, be cautious about closing the underlying
          WT_CURSOR_METADATA.file_cursor field, it may not be set when we call
          __curmetadata_close().

          • comment typo
          • fix a comment
          • Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag.
          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2693 Check open_cursor error paths for consistent handling (#2859) WT-2693 Check open_cursor error paths for consistent handling There's a problem with cursor functions which don't call their typed close functions, relying instead on a simple free of the base cursor type because the __wt_cursor_init() function is the last function called before successful return. The problem is __wt_cursor_init() doesn't clean up after itself: for example, it allocates memory to hold the URI in WT_CURSOR.internal_uri, then has a number of subsequent potential error returns, and WT_CURSOR.internal_uri won't be free'd unless __wt_cursor_close() is called. In other words, we should always call the typed closed function (which in turn calls __wt_cursor_close()). Specific additional concerns: cursor/cur_backup.c: __wt_curbackup_open() doesn't turn off backups or clear the hot-backup flag if __wt_cursor_init() fails, which will hang the process. Remove the cleanup-on-error code from the __backup_start() function and rely instead on the backup cursor's close function to do the work. There's a problem if we fail the initial backup cursor test, in which case we don't want to do cleanup other than discarding the cursors themselves. Add the WT_CURBACKUP_LOCKER flag to the WT_CURSOR_BACKUP structure, which isn't set until we're holding the hot backup cursor flag, and use it to figure out how much cleanup needs to be done. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with __wt_bulk_wrapup(), which assumes WT_CURSOR_BULK.reconcile has been initialized, change it to check for NULL. cursor/cur_file.c: the change to call the __curfile_close() function has a problem with incrementing the data-source's in-use counter. Instead of incrementing the counter in __wt_curfile_open() before calling __curfile_create(), increment it in __curfile_create() after the point where all failure paths close the cursor. cursor/cur_log.c: the change to call the __curlog_close() function means we may be required to clean up a cursor that has not yet acquired the log archive lock. Add the WT_CURLOG_ARCHIVE_LOCK flag to the WT_CURSOR_LOG structure so the close function knows if the log archive lock needs to be released. cursor/cur_log.c: remove the __curlog_reset() call from __curlog_close(), we're closing the cursor, resetting it doesn't do anything useful. cursor/cur_metadata.c: the change to call the __curmetadata_close() function is slightly tricky, be cautious about closing the underlying WT_CURSOR_METADATA.file_cursor field, it may not be set when we call __curmetadata_close(). comment typo fix a comment Add explicit comment on the life-cycle of the WT_CURBACKUP_LOCKER flag. Review comment, set a specific size on the flags field. Generally, the cursors all use uint8_t for flags, and I don't think it matters much what we use, use the same thing everywhere. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/f258ae5ac12a0394c8c8559fe6aeefff06092266
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2

          ref: dddca65..911c940ada
          for: 3.2.10

          SERVER-24971 Excessive memory held by sessions when application threads do evictions
          SERVER-25843 Coverity analysis defect 99856: Redundant test
          SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced
          SERVER-25846 Coverity analysis defect 99861: Dereference after null check
          WT-1162 Add latency to Jenkins wtperf tests and plots
          WT-2026 Maximum pages size at eviction too large
          WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration
          WT-2233 Investigate changing when the eviction server switches to aggressive mode.
          WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
          WT-2323 Allocate a transaction id at the beginning of join cursor iteration
          WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
          WT-2380 Make scripts fail if code doesn't match style
          WT-2486 Update make check so that it runs faster
          WT-2555 make format run on Windows
          WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
          WT-2631 nullptr is passed for parameters marked with attribute non-null
          WT-2638 ftruncate may not be supported
          WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor
          WT-2648 cache-line alignment for new ports
          WT-2665 Limit allocator fragmentation in WiredTiger
          WT-2678 The metadata should not imply that an empty value is true
          WT-2688 configure --enable-python doesn't check for availability of swig
          WT-2693 Check open_cursor error paths for consistent handling
          WT-2695 Integrate s390x accelerated crc32c support
          WT-2708 split child-update race with reconciliation/eviction
          WT-2711 Change statistics log configuration options
          WT-2719 add fuzz testing for WiredTiger options and reconfiguration.
          WT-2728 Don't re-read log file headers during log_flush
          WT-2729 Focus eviction walks in largest trees
          WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
          WT-2731 Raw compression can create pages that are larger than expected
          WT-2732 Coverity analysis defect 99665: Redundant test
          WT-2734 Improve documentation of eviction behavior
          WT-2737 Scrub dirty pages rather than evicting them
          WT-2738 Remove the ability to change the default checkpoint name
          WT-2739 pluggable file systems documentation cleanups
          WT-2743 Thread count statistics always report 0
          WT-2744 partial line even with line buffering set
          WT-2746 track checkpoint I/O separately from eviction I/O
          WT-2751 column-store statistics incorrectly calculates the number of entries
          WT-2752 Fixes to zipfian wtperf workload config
          WT-2755 flexelint configuration treats size_t as 4B type
          WT-2756 Upgrade the autoconf archive package to check for swig 3.0
          WT-2757 Column tables behave differently when column names are provided
          WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
          WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
          WT-2762 wtstats tool fails if checkpoint runs
          WT-2763 Unit test test_intpack failing on OSX
          WT-2764 Optimize checkpoints to reduce throughput disruption
          WT-2765 wt dump: indices need to be shown in the dump output
          WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache
          WT-2767 test suite needs way to run an individual scenario
          WT-2769 Update documentation to reflect correct limits of memory_page_max
          WT-2770 Add statistics tracking schema operations
          WT-2772 Investigate log performance testing weirdness
          WT-2773 search_near in indexes does not find exact matches
          WT-2774 minor cleanups/improvements
          WT-2778 Python test suite: make scenario initialization consistent
          WT-2779 Raw compression created unexpectedly large pages on disk
          WT-2781 Enhance bulk cursor option with an option to return immediately on contention
          WT-2782 Missing a fs_directory_list_free in ex_file_system.c
          WT-2783 wtperf multi-btree.wtperf dumps core on Mac
          WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
          WT-2787 Include src/include/wiredtiger_ext.h is problematic
          WT-2788 Java: freed memory overwrite during handle close can cause JNI crash
          WT-2791 Enhance OS X Evergreen unit test
          WT-2793 wtperf config improvements
          WT-2795 Update documentation around read-only configuration
          WT-2796 Memory leak in reconciliation uncovered by stress testing
          WT-2798 Crash vulnerability with nojournal after create during checkpoint
          WT-2800 Illegal file format in test/format on PPC
          WT-2801 Crash vulnerability from eviction of metadata during checkpoint
          WT-2802 Transaction commit causes heap-use-after free
          WT-2803 Add verbose functionality to WT Evergreen tests
          WT-2804 Don't read values in a tree without a snapshot
          WT-2805 Infinite recursion if error streams fail
          WT-2806 wtperf allocation size off-by-one
          WT-2807 Switch Jenkins performance tests to tcmalloc
          WT-2811 Reconciliation asserts that transaction time has gone backwards
          WT-2812 Error when reconfiguring cache targets
          WT-2813 small cache usage stuck even with large cache
          WT-2814 Enhance wtperf to support single-op truncate mode
          WT-2816 Improve WiredTiger eviction performance
          WT-2817 Investigate performance regression in develop, add workload to wtperf/runners
          WT-2818 The page visibility check when queuing pages for eviction is overly restrictive
          WT-2820 add gcc warn_unused_result attribute
          WT-2822 panic mutex and other functions that cannot fail
          WT-2823 support file handles without a truncate method
          WT-2824 wtperf displays connection and table create configurations twice
          WT-2826 clang38 false positive on uninitialized variable.
          WT-2827 checkpoint log_size configuration improvements
          WT-2828 Make long wtperf tests reflect mongoDB usage
          WT-2829 Switch automated testing to use enable-strict configure option
          WT-2832 Python test uses hard-coded temporary directory
          WT-2834 Join cursor: discrepancy with bloom filters
          WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup
          WT-2838 Don't free session handles on close if leak memory is configured
          WT-2839 lint: Ignoring return value of function
          WT-2840 clang analysis: garbage values
          WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig
          WT-2842 split wtperf's configuration into per-database and per-run parts
          WT-2843 Fix a bug in recovery if there is no filesystem truncate support
          WT-2846 Several bugs related to reconfiguring eviction server at runtime
          WT-2847 Merge fair locks into read/write locks.
          WT-2850 clang 4.1 attribute warnings when building
          WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup
          WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE
          WT-2862 Fix lint error in test case for forced eviction with multiple cursors
          WT-2863 Support UTF-8 paths on Windows
          WT-2865 eviction thread error failure
          WT-2866 Eviction server algorithm tuning
          WT-2867 Review and fix barrier usage in __lsm_tree_close
          WT-2868 Add sample_interval to checkpoint-stress wtperf config
          WT-2869 Performance regression on secondaries
          WT-2870 Rename wtperf checkpoint schema jobs
          WT-2871 __wt_verbose has the wrong GCC format attributes
          WT-2872 Recent stuck cache test/stress failures.
          WT-2873 Refactor CRC32 code
          WT-2875 Test test_wt2853_perf can run too long under valgrind
          WT-2876 Extend wtperf to support a log like table
          WT-2878 Verbose changes affected performance
          WT-2881 Add -Wpedantic to clang compiler warning flags
          WT-2883 wiredtiger_open with verbose=handleops recursive loop
          WT-2885 __wt_checkpoint_signal lint
          WT-2886 Decide how in-memory configuration and eviction_dirty_target interact
          WT-2888 Switch functions to return void where possible
          WT-2892 hot backup can race with block truncate
          WT-2896 Coverity #1362535: resource leak
          WT-2897 Checkpoints can become corrupted on failure
          WT-2901 Add option to disable checkpoint dirty stepdown phase
          WT-2903 Reduce the impact of checkpoint scrubbing on applications
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2 ref: dddca65..911c940ada for: 3.2.10 SERVER-24971 Excessive memory held by sessions when application threads do evictions SERVER-25843 Coverity analysis defect 99856: Redundant test SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced SERVER-25846 Coverity analysis defect 99861: Dereference after null check WT-1162 Add latency to Jenkins wtperf tests and plots WT-2026 Maximum pages size at eviction too large WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration WT-2233 Investigate changing when the eviction server switches to aggressive mode. WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact WT-2323 Allocate a transaction id at the beginning of join cursor iteration WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang WT-2380 Make scripts fail if code doesn't match style WT-2486 Update make check so that it runs faster WT-2555 make format run on Windows WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros WT-2631 nullptr is passed for parameters marked with attribute non-null WT-2638 ftruncate may not be supported WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor WT-2648 cache-line alignment for new ports WT-2665 Limit allocator fragmentation in WiredTiger WT-2678 The metadata should not imply that an empty value is true WT-2688 configure --enable-python doesn't check for availability of swig WT-2693 Check open_cursor error paths for consistent handling WT-2695 Integrate s390x accelerated crc32c support WT-2708 split child-update race with reconciliation/eviction WT-2711 Change statistics log configuration options WT-2719 add fuzz testing for WiredTiger options and reconfiguration. WT-2728 Don't re-read log file headers during log_flush WT-2729 Focus eviction walks in largest trees WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary WT-2731 Raw compression can create pages that are larger than expected WT-2732 Coverity analysis defect 99665: Redundant test WT-2734 Improve documentation of eviction behavior WT-2737 Scrub dirty pages rather than evicting them WT-2738 Remove the ability to change the default checkpoint name WT-2739 pluggable file systems documentation cleanups WT-2743 Thread count statistics always report 0 WT-2744 partial line even with line buffering set WT-2746 track checkpoint I/O separately from eviction I/O WT-2751 column-store statistics incorrectly calculates the number of entries WT-2752 Fixes to zipfian wtperf workload config WT-2755 flexelint configuration treats size_t as 4B type WT-2756 Upgrade the autoconf archive package to check for swig 3.0 WT-2757 Column tables behave differently when column names are provided WT-2759 Releasing the hot-backup lock doesn't require the schema lock. WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default WT-2762 wtstats tool fails if checkpoint runs WT-2763 Unit test test_intpack failing on OSX WT-2764 Optimize checkpoints to reduce throughput disruption WT-2765 wt dump: indices need to be shown in the dump output WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache WT-2767 test suite needs way to run an individual scenario WT-2769 Update documentation to reflect correct limits of memory_page_max WT-2770 Add statistics tracking schema operations WT-2772 Investigate log performance testing weirdness WT-2773 search_near in indexes does not find exact matches WT-2774 minor cleanups/improvements WT-2778 Python test suite: make scenario initialization consistent WT-2779 Raw compression created unexpectedly large pages on disk WT-2781 Enhance bulk cursor option with an option to return immediately on contention WT-2782 Missing a fs_directory_list_free in ex_file_system.c WT-2783 wtperf multi-btree.wtperf dumps core on Mac WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation WT-2787 Include src/include/wiredtiger_ext.h is problematic WT-2788 Java: freed memory overwrite during handle close can cause JNI crash WT-2791 Enhance OS X Evergreen unit test WT-2793 wtperf config improvements WT-2795 Update documentation around read-only configuration WT-2796 Memory leak in reconciliation uncovered by stress testing WT-2798 Crash vulnerability with nojournal after create during checkpoint WT-2800 Illegal file format in test/format on PPC WT-2801 Crash vulnerability from eviction of metadata during checkpoint WT-2802 Transaction commit causes heap-use-after free WT-2803 Add verbose functionality to WT Evergreen tests WT-2804 Don't read values in a tree without a snapshot WT-2805 Infinite recursion if error streams fail WT-2806 wtperf allocation size off-by-one WT-2807 Switch Jenkins performance tests to tcmalloc WT-2811 Reconciliation asserts that transaction time has gone backwards WT-2812 Error when reconfiguring cache targets WT-2813 small cache usage stuck even with large cache WT-2814 Enhance wtperf to support single-op truncate mode WT-2816 Improve WiredTiger eviction performance WT-2817 Investigate performance regression in develop, add workload to wtperf/runners WT-2818 The page visibility check when queuing pages for eviction is overly restrictive WT-2820 add gcc warn_unused_result attribute WT-2822 panic mutex and other functions that cannot fail WT-2823 support file handles without a truncate method WT-2824 wtperf displays connection and table create configurations twice WT-2826 clang38 false positive on uninitialized variable. WT-2827 checkpoint log_size configuration improvements WT-2828 Make long wtperf tests reflect mongoDB usage WT-2829 Switch automated testing to use enable-strict configure option WT-2832 Python test uses hard-coded temporary directory WT-2834 Join cursor: discrepancy with bloom filters WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup WT-2838 Don't free session handles on close if leak memory is configured WT-2839 lint: Ignoring return value of function WT-2840 clang analysis: garbage values WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig WT-2842 split wtperf's configuration into per-database and per-run parts WT-2843 Fix a bug in recovery if there is no filesystem truncate support WT-2846 Several bugs related to reconfiguring eviction server at runtime WT-2847 Merge fair locks into read/write locks. WT-2850 clang 4.1 attribute warnings when building WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE WT-2862 Fix lint error in test case for forced eviction with multiple cursors WT-2863 Support UTF-8 paths on Windows WT-2865 eviction thread error failure WT-2866 Eviction server algorithm tuning WT-2867 Review and fix barrier usage in __lsm_tree_close WT-2868 Add sample_interval to checkpoint-stress wtperf config WT-2869 Performance regression on secondaries WT-2870 Rename wtperf checkpoint schema jobs WT-2871 __wt_verbose has the wrong GCC format attributes WT-2872 Recent stuck cache test/stress failures. WT-2873 Refactor CRC32 code WT-2875 Test test_wt2853_perf can run too long under valgrind WT-2876 Extend wtperf to support a log like table WT-2878 Verbose changes affected performance WT-2881 Add -Wpedantic to clang compiler warning flags WT-2883 wiredtiger_open with verbose=handleops recursive loop WT-2885 __wt_checkpoint_signal lint WT-2886 Decide how in-memory configuration and eviction_dirty_target interact WT-2888 Switch functions to return void where possible WT-2892 hot backup can race with block truncate WT-2896 Coverity #1362535: resource leak WT-2897 Checkpoints can become corrupted on failure WT-2901 Add option to disable checkpoint dirty stepdown phase WT-2903 Reduce the impact of checkpoint scrubbing on applications Branch: v3.2 https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                40 weeks, 4 days ago
                Date of 1st Reply: