ExportXMLWordPrintableJSON

    • Storage Engines - Foundations
    • 225.794
    • None
    • None

      Problem

      Existing layered cursor statistics can report zero or incomplete values through statistics:layered:... and statistics:table:..., while connection-level statistics correctly count the same operations.

      WT_STAT_CONN_DSRC_INCR in src/include/stat.h updates two independently stored counters:

      #define WT_STAT_CONN_DSRC_INCRV(session, fld, value) \
          do {                                           \
              WT_STAT_CONN_INCRV(session, fld, value);    \
              WT_STAT_DSRC_INCRV(session, fld, value);    \
          } while (0)
      

      The data-source update is conditional on the current handle having an initialized statistics array:

      if ((session)->dhandle != NULL && (session)->dhandle->stat_array != NULL)
          WT_STATP_DSRC_INCRV(session, (session)->dhandle->stats, fld, value);
      

      Layered cursor API entry selects clayered->dhandle through CURSOR_API_CALL and API_SESSION_PUSH in src/include/api.h. However, __wt_conn_dhandle_open in src/conn/conn_dhandle.c initializes data-source statistics for B-tree handles but not layered handles.

      Internal constituent opens can subsequently change session->dhandle. For example, _clayered_open_stable_int calls wt_open_cursor, which reaches _wt_curfile_open and resolves the stable B-tree handle. Some increments therefore reach constituent statistics, while increments made with the uninitialized layered handle selected are silently skipped.

      _curstat_layered_init in src/cursor/cur_stat.c initializes the result to zero and aggregates ingest/stable statistics through init_layered_constituent_stats. It does not include statistics owned by the logical layered handle. _wt_curstat_table_init in src/schema/schema_stat.c resolves a simple table's statistics through its underlying column-group source.

      Consequently, table-level values depend on cursor initialization and reuse, rather than solely on the operations performed.

      Examples

      With one layered table on a leader:

      Workload Statistic Connection count Table count
      Four successful next() calls followed by WT_NOTFOUND layered_curs_next 5 0
      Four successful search() calls on the same fresh cursor layered_curs_search 4 1

      Next operations

      __clayered_next in src/cursor/cur_layered.c increments the main counter before entering the iteration path:

      CURSOR_API_CALL(cursor, session, ret, next, clayered->dhandle);
      /* ... */
      WT_STAT_CONN_DSRC_INCR(session, layered_curs_next);
      WT_ERR(__clayered_iterate(clayered, WTI_CLAYERED_ITERATE_NEXT));
      WT_STAT_CLAYERED_READ_CONSTITUENT_INCR(session, clayered, layered_curs_next);
      

      The main increments occur with the layered handle selected, so their data-source updates are skipped. All five calls remain recorded at connection scope.

      Search operations

      __clayered_search in src/cursor/cur_layered.c enters the layered operation before incrementing its counter:

      CURSOR_API_CALL(cursor, session, ret, search, clayered->dhandle);
      /* ... */
      WT_ERR(__clayered_enter(clayered, WTI_CLAYERED_MODE_SEARCH, &op));
      /* ... */
      WT_STAT_CONN_DSRC_INCR(session, layered_curs_search);
      WT_ERR(__clayered_lookup(&op, &cursor->value));
      

      _clayered_enter calls _clayered_update_stable, which opens the stable cursor on first use. The first search therefore increments the stable handle's statistics and is included in table reporting. Subsequent searches reuse that cursor and increment with the uninitialized layered handle selected, losing their table-level increments.

      Similar underreporting affects _clayered_prev, clayered_search_near, clayered_insert, clayered_update, clayered_remove, and clayered_modify. _clayered_reserve also invokes a public search internally. The _stable and _ingest counters updated by WT_STAT_CLAYERED_READ_CONSTITUENT_INCR can likewise be incomplete because the macro uses the current session handle for storage.

      Expected Behavior

      Logical layered cursor statistics should be attributed consistently to the owning layered table, independently of which constituent is open or supplies the result. Table reporting should include all relevant increments without double-counting constituent activity.

      For a workload involving one table, connection and table deltas for the same metric should agree. Attempt counters and successful-result counters should retain their existing semantics: terminal next() calls count toward layered_curs_next but not its result-source variants.

      Connection-level statistics collected by regular FTDC remain populated. MongoDB's WiredTigerUtil::collectConnectionStatistics in src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp reads statistics:, which aggregates connection buckets independently of data-source statistics. The issue affects table-level attribution and reporting.

        1. test_layered_cursor30.py
          6 kB
          Alexander Pullen

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Alexander Pullen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: