-
Type:
New Feature
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
DB Integration & Observability
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Overview
The IDL-compiler slice of the unified metrics framework (epic SPM-4962): a milestone-1 grammar subset with codegen for metrics structs, the section catalog, and serverStatus rendering. Constructs from later milestones are hard parse errors, so nothing is silently ignored.
Grammar
- Three top-level blocks: metrics_section, metrics_structs, and metrics (a fragment attached to a section root).
- Node forms: leaf (kind + cpp_type), group (fields), and mount (struct_type). Mounts are legal only at the top level of a metrics: fragment.
- A section root carries a mandatory fields mapping holding its subtrees, so child nodes are always visually distinct from section-root attributes; the only exception is a scalar struct_type that mounts a metrics_struct at the section root itself (no child nodes).
- Leaf options: description (required), cpp_name, unit, optional, cache_exclusive, buckets, export, version.
File naming
- Metrics grammar files must be named <feature>_metrics.idl (ID125 plus a repo-scan test). The reverse check, a *_metrics.idl file with no metrics blocks, waits on legacy renames (SERVER-138506).
- metrics_sections.idl is the framework-owned registry and the only file that may declare a metrics_section block. A block anywhere else is a parse error.
- The file stem (minus _metrics) must be a lowercase identifier: it names the generated C++ namespace.
Codegen
- Output is <feature>_metrics_gen.h/cpp via the plain idl_generator bazel rule, which skips the explain-verbosity default dep for *_metrics.idl sources.
- Struct members map to observability primitives: Counter, UpDownCounter, Gauge, HighWaterMark, AtomicHistogram, Histogram, SynchronizedValue. OptionalMetric wraps optional and CacheExclusive wraps cache_exclusive.
- All generated metrics types (declared and fragment-derived structs) live in a per-file namespace, for example mongo::observability::quickwrite. Multiple files can contribute to the same section root without colliding on type names.
- appendTo(BSONObjBuilder*) renders fields in IDL declaration order. Groups render as inline subobjects. Optional leaves render only after their first write.
- Histograms render the legacy metrics-tree shape: a BSON array of lowerBound/count NumberLongs per bucket.
- Fragment roots generate a <SectionRoot>Metrics struct.
Section catalog
- MetricsSectionDescription is hand-written in src/mongo/observability/metrics_section_description.h.
- The generated metrics_sections_gen header emits only the k<Section>Section constants and the kMetricsSections array.
- src/mongo/observability/metrics_sections.idl starts empty; each section root is declared by the migration that first populates it. Its description and include_by_default decision belong to the owner.
Imports
- Importing a file with metrics definitions adds that file's generated header to the consumer's includes, so imported metric types are reachable through the importing header.
Binder validation
- kind/cpp_type compatibility matrix and the unit enum.
- buckets are required for histogram kinds and forbidden elsewhere. Boundaries must be integers in the signed 64-bit range, sorted ascending (ID127).
- Node keys must be lowerCamel identifiers (ID124); the key is the BSON key and cpp_name renames the member.
- Rejected collisions: sibling member names, duplicate metrics symbols (ID123), fragment-derived struct names, and group types against enclosing types.
- Reserved generated names are rejected as member or struct names: appendTo, std, mongo, observability, BSONObjBuilder, BSONArrayBuilder.
Deferred constructs
gates/`when`/`state`, attributes, compose/`derived`/`alias_of`, families, collection/`sample`, cost, render_options, otel_name, appender, and cpp_accessor all fail with ID113 ("not supported by the metrics framework yet"). Each deferral carries a TODO naming its ticket or design tab.
Ownership
All metrics compiler logic lives in buildscripts/idl/idl/metrics/, owned by query-integration-observability through OWNERS.yml filters and the regenerated .github/CODEOWNERS. The server-programmability compiler files keep only thin hooks, plus the ERROR_ID_METRICS_* constants in idl/errors.py.
Schemas
buildscripts/idl/metrics_schema.yml (yaml-language-server via modeline comment; repo-wide glob promotion is SERVER-138506) and the idl_schema.yml extension, with build-time validation.
Tests
- 60 metrics compiler tests in {{tests/test_metrics_
{parser,binder,generator,yaml_schema}
.py}}; the full IDL suite is 240 tests.
- C++ end-to-end test src/mongo/observability/metrics_codegen_test.cpp with fixture codegen_test_metrics.idl (namespace mongo::observability::codegen_test), covering every kind, wrapper, and group.
Known divergence from the design doc
- Histograms render the legacy metrics-tree shape (array of lowerBound/count), not the count/sum/buckets object sketched in the Core Scalar tab. Byte compatibility wins; min and max are not rendered.
- C++ keyword member and struct names are not rejected by the compiler; they fail at C++ compile time in the consumer. A hand-maintained keyword list seemed more brittle than that failure mode.
Out of scope
- Attributes (set_once, allowed_values): SERVER-138450.
- Build-time section-root enforcement: SERVER-138451.
- Repo-wide yaml-server glob and legacy *_metrics.idl renames: SERVER-138506.
- Nested mounts inside structs/groups: SERVER-138534.
- Gating, composition, families, collection/sample, cost, render_options: Phase 2 design tabs.
Design source
[Technical Design: Unified Metrics Framework|$GDOC], Core Scalar Framework tab: "Grammar", "Metrics Sections", "Metrics flow via metrics_structs"; Overview tab: "Metrics Flow / Writing metrics".
- blocks
-
SERVER-138595 End-to-end C++ test for metrics codegen in observability
-
- In Code Review
-
-
SERVER-138333 Registration codegen: per-file MONGO_INITIALIZER metrics registration
-
- Closed
-
-
SERVER-138332 Build the MetricsRegistry for materialization, validation, and unmaterialize()
-
- Needs Scheduling
-
-
SERVER-138450 Generate attribute support for metrics IDL (set_once and allowed_values)
-
- Needs Scheduling
-
-
SERVER-138451 Enforce that metrics fragments reference valid roots
-
- Needs Scheduling
-
-
SERVER-138534 Support nested mounting inside metrics_structs
-
- Needs Scheduling
-
-
SERVER-138506 Reserve *_metrics.idl naming for the metrics framework
-
- Needs Scheduling
-
- is blocked by
-
SERVER-138330 Productionize metric primitives for the unified metrics framework
-
- In Code Review
-
-
SERVER-138594 Metrics grammar binder validation in the IDL compiler
-
- In Code Review
-