ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1025
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • Fully Compatible
    • ALL
    • Storage Execution 2026-11-09
    • 0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      PrimaryDrivenResumableIndexBuildTest._readResumeMetrics() in jstests/noPassthrough/libs/index_builds/primary_driven.js assumes one OTel metrics file per node and sums the last cumulative snapshot of every file in the metrics directory. However, the OTel file exporter rotates files for a single node:

      • at midnight UTC, because the file name embeds the date
        (mongodb-<pid>%Y%m%d%N-metrics.jsonl, see
        src/mongo/otel/metrics/metrics_initialization.cpp)
      • when a file reaches the 8MB chunk limit (the %N rotation index)

      Every rotated file retains its last cumulative snapshot, so after a rotation a single node's counter is summed once per file and the cluster-wide total is inflated. This makes the resume metric assertions in the primary-driven index build test library flaky for any run that crosses midnight UTC or triggers the chunk limit.

      Suggested fix

      Group the files by pid (parsed from the file name) and take the per-phase maximum across a pid's files as that node's current value. The counters are cumulative within a process, so the max equals the newest snapshot, and it also tolerates reading the directory right after a rotation when the new file may not have been flushed yet. Then sum across pids.

      Related work

      SERVER-127030 tracks replacing this reader with getLatestMetrics() or an equivalent once attributes are preserved there. That work is about attribute preservation and does not by itself address the rotation double-count described here. Any replacement reader must still aggregate the latest snapshot per node rather than summing across files or taking a single global snapshot.

            Assignee:
            Ernesto Rodriguez Reina
            Reporter:
            Ernesto Rodriguez Reina
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: