ExportXMLWordPrintableJSON

    • Type: New Feature
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • DB Integration & Observability
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Overview

      The IDL-compiler slice of the unified metrics framework (epic SPM-4962): a milestone-1 grammar subset with codegen for metrics structs, the section catalog, and serverStatus rendering. Constructs from later milestones are hard parse errors, so nothing is silently ignored.

      Grammar

      • Three top-level blocks: metrics_section, metrics_structs, and metrics (a fragment attached to a section root).
      • Node forms: leaf (kind + cpp_type), group (fields), and mount (struct_type). Mounts are legal only at the top level of a metrics: fragment.
      • A section root carries a mandatory fields mapping holding its subtrees, so child nodes are always visually distinct from section-root attributes; the only exception is a scalar struct_type that mounts a metrics_struct at the section root itself (no child nodes).
      • Leaf options: description (required), cpp_name, unit, optional, cache_exclusive, buckets, export, version.

      File naming

      • Metrics grammar files must be named <feature>_metrics.idl (ID125 plus a repo-scan test). The reverse check, a *_metrics.idl file with no metrics blocks, waits on legacy renames (SERVER-138506).
      • metrics_sections.idl is the framework-owned registry and the only file that may declare a metrics_section block. A block anywhere else is a parse error.
      • The file stem (minus _metrics) must be a lowercase identifier: it names the generated C++ namespace.

      Codegen

      • Output is <feature>_metrics_gen.h/cpp via the plain idl_generator bazel rule, which skips the explain-verbosity default dep for *_metrics.idl sources.
      • Struct members map to observability primitives: Counter, UpDownCounter, Gauge, HighWaterMark, AtomicHistogram, Histogram, SynchronizedValue. OptionalMetric wraps optional and CacheExclusive wraps cache_exclusive.
      • All generated metrics types (declared and fragment-derived structs) live in a per-file namespace, for example mongo::observability::quickwrite. Multiple files can contribute to the same section root without colliding on type names.
      • appendTo(BSONObjBuilder*) renders fields in IDL declaration order. Groups render as inline subobjects. Optional leaves render only after their first write.
      • Histograms render the legacy metrics-tree shape: a BSON array of lowerBound/count NumberLongs per bucket.
      • Fragment roots generate a <SectionRoot>Metrics struct.

      Section catalog

      • MetricsSectionDescription is hand-written in src/mongo/observability/metrics_section_description.h.
      • The generated metrics_sections_gen header emits only the k<Section>Section constants and the kMetricsSections array.
      • src/mongo/observability/metrics_sections.idl starts empty; each section root is declared by the migration that first populates it. Its description and include_by_default decision belong to the owner.

      Imports

      • Importing a file with metrics definitions adds that file's generated header to the consumer's includes, so imported metric types are reachable through the importing header.

      Binder validation

      • kind/cpp_type compatibility matrix and the unit enum.
      • buckets are required for histogram kinds and forbidden elsewhere. Boundaries must be integers in the signed 64-bit range, sorted ascending (ID127).
      • Node keys must be lowerCamel identifiers (ID124); the key is the BSON key and cpp_name renames the member.
      • Rejected collisions: sibling member names, duplicate metrics symbols (ID123), fragment-derived struct names, and group types against enclosing types.
      • Reserved generated names are rejected as member or struct names: appendTo, std, mongo, observability, BSONObjBuilder, BSONArrayBuilder.

      Deferred constructs

      gates/`when`/`state`, attributes, compose/`derived`/`alias_of`, families, collection/`sample`, cost, render_options, otel_name, appender, and cpp_accessor all fail with ID113 ("not supported by the metrics framework yet"). Each deferral carries a TODO naming its ticket or design tab.

      Ownership

      All metrics compiler logic lives in buildscripts/idl/idl/metrics/, owned by query-integration-observability through OWNERS.yml filters and the regenerated .github/CODEOWNERS. The server-programmability compiler files keep only thin hooks, plus the ERROR_ID_METRICS_* constants in idl/errors.py.

      Schemas

      buildscripts/idl/metrics_schema.yml (yaml-language-server via modeline comment; repo-wide glob promotion is SERVER-138506) and the idl_schema.yml extension, with build-time validation.

      Tests

      • 60 metrics compiler tests in {{tests/test_metrics_ {parser,binder,generator,yaml_schema}

        .py}}; the full IDL suite is 240 tests.

      • C++ end-to-end test src/mongo/observability/metrics_codegen_test.cpp with fixture codegen_test_metrics.idl (namespace mongo::observability::codegen_test), covering every kind, wrapper, and group.

      Known divergence from the design doc

      • Histograms render the legacy metrics-tree shape (array of lowerBound/count), not the count/sum/buckets object sketched in the Core Scalar tab. Byte compatibility wins; min and max are not rendered.
      • C++ keyword member and struct names are not rejected by the compiler; they fail at C++ compile time in the consumer. A hand-maintained keyword list seemed more brittle than that failure mode.

      Out of scope

      • Attributes (set_once, allowed_values): SERVER-138450.
      • Build-time section-root enforcement: SERVER-138451.
      • Repo-wide yaml-server glob and legacy *_metrics.idl renames: SERVER-138506.
      • Nested mounts inside structs/groups: SERVER-138534.
      • Gating, composition, families, collection/sample, cost, render_options: Phase 2 design tabs.

      Design source

      [Technical Design: Unified Metrics Framework|$GDOC], Core Scalar Framework tab: "Grammar", "Metrics Sections", "Metrics flow via metrics_structs"; Overview tab: "Metrics Flow / Writing metrics".

            Assignee:
            Finley Lau
            Reporter:
            Finley Lau
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: