ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • WT12.0.0, 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: Transactions
    • None
    • Storage Engines - Transactions
    • 78.984
    • None
    • None

      Problem. _wt_txn_snapshot_save (src/txn/txn.c:485, src/txn/txn.c:491-492 at develop 50c032ba6) passes byte sizes to _wt_calloc_def:

      WT_RET(__wt_calloc_def(session, sizeof(WT_TXN_SNAPSHOT), &txn->backup_snapshot_data));
      ...
      WT_ERR(__wt_calloc_def(session, sizeof(uint64_t) * S2C(session)->session_array.size,
        &txn->backup_snapshot_data->snapshot));
      

      _wt_calloc_def(session, number, addr) expands to _wt_calloc(session, number, sizeof(**(addr)), addr) (src/include/misc.h:147-148), which already multiplies the count by the element size:

      • The snapshot array gets 8 * session_array.size elements of uint64_t instead of session_array.size.
      • The WT_TXN_SNAPSHOT allocation gets sizeof(WT_TXN_SNAPSHOT) structures (32 on LP64, 1024 bytes) instead of one.

      WT-11972 introduced this code. No other call to _wt_calloc_def or _wt_realloc_def in src/ passes sizeof in its count argument.

      Effect. The save runs when an application thread refreshes its snapshot to evict a page (__wt_txn_snapshot_save_and_refresh, src/evict/evict_page.c:1558, statistic application thread snapshot refreshed for eviction) and when an application thread evicts under the checkpoint snapshot (src/evict/evict_page.c:1590). With MongoDB's configuration of session_max=33000, session_array.size is 33,043. Each save allocates and zeroes 2,114,752 bytes instead of 264,344: 1,850,408 extra bytes while the application thread is stuck in eviction.

      Proposed change. Use _wt_calloc_one for the WT_TXN_SNAPSHOT structure, and wt_calloc_def with session_array.size as the count for the array. The array is swapped with the transaction snapshot and refilled by a new snapshot or by a copy of the checkpoint snapshot. Neither holds more than session_array.size IDs: wt_txn_init allocates the transaction's own snapshot with that capacity, and _wt_txn_import_snapshot asserts it.

      Verification. Diagnostic build (HAVE_DIAGNOSTIC=1, ENABLE_STRICT=1, macOS arm64, AppleClang 21): test_app_thread_evict01 (drives the refresh path and asserts the statistic), the eviction and txn Python subsuites, and the test_format smoke test pass. There is no new test because the change does not alter behavior observable through the API.

      Pull request: https://github.com/wiredtiger/wiredtiger/pull/14941

            Assignee:
            Chenhao Qu
            Reporter:
            Mikhail Sotnikov (EXT)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: