Gate checkpoint progress logs on WT_PROGRESS_MSG_PERIOD

    • Type: Task
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: Checkpoints
    • None
    • Storage Engines - Persistence
    • 6.7
    • None
    • None

      Context

      Checkpoint progress is logged via WT_VERB_CHECKPOINT_PROGRESS from checkpoint_progress / checkpoint_prepare_progress in src/checkpoint/checkpoint_txn.c. Intermediate messages are currently triggered when wt_counter_backoff fires on write_pages, or when elapsed time exceeds WT_PROGRESS_MSG_PERIOD. The progress check is only invoked from wt_checkpoint_progress_stats after a dirty page is reconciled in sync_file.
      Example:

      [WT_VERB_CHECKPOINT_PROGRESS]: Checkpoint has been running for 12 seconds, wrote 10000 pages (40 MB), walked 50000 pages and checkpointed 3 files
      

      Motivation

      The write-page backoff produces noisy, rate-based logs that are hard to correlate with wall-clock time. After WT-18016, verify already reports on a time interval; checkpoint should match.

      We should also reconsider where the progress message is logged. These are some current gaps that make stall diagnosis challenging:

      • No heartbeat during scrub (__checkpoint_wait_reduce_dirty_cache), clean tree walks, or fsync / BM / HS / metadata sync — silence can mean either "between writes" or a real hang.

      Scope

      Primary

      1. Remove {}wt_counter_backoff {}from }}{_}{}{{{}_checkpoint_progress; gate intermediate messages only on WT_PROGRESS_MSG_PERIOD.
      2. Remove __wt_counter_backoff if it has no remaining callers.
      3. Update test/suite/test_verbose05.py for time-based gating.

      Secondary

      1. Poll progress from long-running loops that don't write: scrub wait, __sync_file per visited page (or every N visits), and optionally on checkpoint_state transitions for BM/HS/meta sync.
      2. Include checkpoint_state and current dhandle URI in the heartbeat; consider deltas / rates and reconcile vs sync time from existing accumulators.

            Assignee:
            Sean Watt
            Reporter:
            Sean Watt
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: