Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2531

in-memory tables are allocating unnecessary memory

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0
    • Labels:
      None
    • # Replies:
      12
    • Last comment by Customer:
      true

      Description

      In-memory tables are no longer "writing" their descriptor blocks, but they are truncating the "file" to the initial size, which causes an empty buffer to be allocated.

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}

          Message: Import wiredtiger-wiredtiger-2.8.0-134-g5047aab.tar.gz from wiredtiger branch mongodb-3.4

          ref: 9cf8eb2..5047aab

          SERVER-23504 Coverity analysis defect 98177: Resource leak
          WT-2330 in-memory configurations should not create on-disk collection files
          WT-2513 conversion from 'int64_t' to 'uint32_t'
          WT-2522 Incorrect format code in message
          WT-2525 in-memory configurations: miscellaneous cleanups
          WT-2527 OS X compile error, missing POSIX_FADV_WILLNEED #define
          WT-2528 style error in WiredTiger build
          WT-2529 The readonly test case is crashing with a stack overflow
          WT-2531 in-memory tables are allocating unnecessary memory
          WT-2532 WT_STREAM_APPEND and WT_STREAM_LINE_BUFFER flag overlap
          WT-2533 Ensure that in-memory tables don't report a zero size
          SERVER-23517 WiredTiger changes for MongoDB 3.3.5
          Branch: master
          https://github.com/mongodb/mongo/commit/de6f136d83b20f8a58ba6fe4ba02be229b6c9159

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: Import wiredtiger-wiredtiger-2.8.0-134-g5047aab.tar.gz from wiredtiger branch mongodb-3.4 ref: 9cf8eb2..5047aab SERVER-23504 Coverity analysis defect 98177: Resource leak WT-2330 in-memory configurations should not create on-disk collection files WT-2513 conversion from 'int64_t' to 'uint32_t' WT-2522 Incorrect format code in message WT-2525 in-memory configurations: miscellaneous cleanups WT-2527 OS X compile error, missing POSIX_FADV_WILLNEED #define WT-2528 style error in WiredTiger build WT-2529 The readonly test case is crashing with a stack overflow WT-2531 in-memory tables are allocating unnecessary memory WT-2532 WT_STREAM_APPEND and WT_STREAM_LINE_BUFFER flag overlap WT-2533 Ensure that in-memory tables don't report a zero size SERVER-23517 WiredTiger changes for MongoDB 3.3.5 Branch: master https://github.com/mongodb/mongo/commit/de6f136d83b20f8a58ba6fe4ba02be229b6c9159
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2531: in-memory tables are wasting space in truncation

          The in-memory truncate call wasn't extending the file (by setting the
          size of the WT_ITEM after truncation extended it).
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/659c977eb08d9c08ef903f5d876ac242daf424cc

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2531 : in-memory tables are wasting space in truncation The in-memory truncate call wasn't extending the file (by setting the size of the WT_ITEM after truncation extended it). Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/659c977eb08d9c08ef903f5d876ac242daf424cc
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2531: in-memory tables are wasting space in truncation

          Don't truncate the file when running in-memory, it will allocate relatively
          large chunks of buffer memory in the underlying OS layer that we'll never use.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/af4e3f14bd51f3c4149f64bcf4e2e249c61c5136

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2531 : in-memory tables are wasting space in truncation Don't truncate the file when running in-memory, it will allocate relatively large chunks of buffer memory in the underlying OS layer that we'll never use. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/af4e3f14bd51f3c4149f64bcf4e2e249c61c5136
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Merge pull request #2629 from wiredtiger/wt-2531

          WT-2531: in-memory tables are wasting space in truncation
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/7a5fce2b618d51e0bb42a12b0a2e44a3632d3d10

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Merge pull request #2629 from wiredtiger/wt-2531 WT-2531 : in-memory tables are wasting space in truncation Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/7a5fce2b618d51e0bb42a12b0a2e44a3632d3d10
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

          Message: Import wiredtiger-wiredtiger-2.8.0-201-g7ea2631.tar.gz from wiredtiger branch mongodb-3.2

          ref: 43e885a..7ea2631

          SERVER-23504 Coverity analysis defect 98177: Resource leak
          SERVER-23526 Replication relies on storage engines reporting a non-zero size for correctness
          SERVER-23588 mongod with WiredTiger won't start on Windows when built with --dbg=on --opt=off
          SERVER-23682 WiredTiger changes for MongoDB 3.2.6
          WT-2330 in-memory configurations should not create on-disk collection files
          WT-2507 Add upgrading documentation in preparation for 2.8 release.
          WT-2512 wtperf: MSVC complains about float conversion in throttle code
          WT-2513 conversion from 'int64_t' to 'uint32_t'
          WT-2517 wtperf uses setvbuf in a way that isn't supported on Windows
          WT-2522 Incorrect format code in message
          WT-2525 in-memory configurations: miscellaneous cleanups
          WT-2527 OS X compile error, missing POSIX_FADV_WILLNEED #define
          WT-2528 style error in WiredTiger build
          WT-2529 The readonly test case is crashing with a stack overflow
          WT-2531 in-memory tables are allocating unnecessary memory
          WT-2532 WT_STREAM_APPEND and WT_STREAM_LINE_BUFFER flag overlap
          WT-2533 Ensure that in-memory tables don't report a zero size
          WT-2534 Invalid transaction snapshots on PowerPC
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/7ee4e4e493c3785fea489ee3508ca18526709c16

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-201-g7ea2631.tar.gz from wiredtiger branch mongodb-3.2 ref: 43e885a..7ea2631 SERVER-23504 Coverity analysis defect 98177: Resource leak SERVER-23526 Replication relies on storage engines reporting a non-zero size for correctness SERVER-23588 mongod with WiredTiger won't start on Windows when built with --dbg=on --opt=off SERVER-23682 WiredTiger changes for MongoDB 3.2.6 WT-2330 in-memory configurations should not create on-disk collection files WT-2507 Add upgrading documentation in preparation for 2.8 release. WT-2512 wtperf: MSVC complains about float conversion in throttle code WT-2513 conversion from 'int64_t' to 'uint32_t' WT-2517 wtperf uses setvbuf in a way that isn't supported on Windows WT-2522 Incorrect format code in message WT-2525 in-memory configurations: miscellaneous cleanups WT-2527 OS X compile error, missing POSIX_FADV_WILLNEED #define WT-2528 style error in WiredTiger build WT-2529 The readonly test case is crashing with a stack overflow WT-2531 in-memory tables are allocating unnecessary memory WT-2532 WT_STREAM_APPEND and WT_STREAM_LINE_BUFFER flag overlap WT-2533 Ensure that in-memory tables don't report a zero size WT-2534 Invalid transaction snapshots on PowerPC Branch: v3.2 https://github.com/mongodb/mongo/commit/7ee4e4e493c3785fea489ee3508ca18526709c16

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 10 weeks, 5 days ago
                Date of 1st Reply: