-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
DB Integration & Observability
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Problem
BF-47116 exposed a fragile assumption in the metrics-filtering JS tests. They use the complete metrics response from an auth-disabled deployment as the expected schema for a separate auth-enabled deployment. These processes can legitimately report different fields because of sampling history, node role, or dynamically keyed metrics.
Consequently, a timing-dependent schema difference can fail the test even when filtering is correct. Maintaining exceptions for each such metric is error-prone, and broad subtree exceptions can hide meaningful differences.
Desired outcome
Verify metrics-filtering behavior without requiring independent deployments to have identical complete schemas. Adding a conditionally present, internal-only metric should not require another JS schema-comparison exception merely to keep the test passing.
Preserve checks that restricted users do not receive disallowed metrics, that representative approved metrics remain visible, and that bypass behavior works as intended.
For example, we could limit the end-to-end js testing to just check a couple specific fields are or are not included, to show that filtering is being applied. We could rely on unit testing to test that the filtering policy is correct, and make new metrics be hidden by default, possibly requiring a separate ticket/triage to make sure the filtering logic is updated accordingly. Note that some sub-sections are included/allowed wholesale, so new metrics in those subsections can be included by default without issue.
Scope and approach
- Focus on serverStatus. Include the related getDiagnosticData comparison only where its embedded serverStatus snapshot affects this test. getDiagnosticData returns a periodic diagnostic snapshot and bypasses metrics filtering; it is not another filtered-output case.
- Evaluate component tests that apply the real filtering policy to one fixed metrics document, including allowed, disallowed, unknown, absent, and dynamically keyed fields. Expected customer visibility must not be defined solely by asking the implementation for its own allowlist.
- Keep a smaller live JS test to verify that authentication, feature-flag state, the serverStatus command, and filtering work together. Do not use another process's full schema as its correctness oracle.
- Preserve the existing customer-exposure policy. Changing which metrics are approved for customer visibility is not part of this ticket.
- The fixture also covers other commands. Follow-up tickets may make sense for redesigning their tests; completing every command is not required here.
Acceptance criteria
- A legitimate difference in metric presence or dynamic keys between independent fixtures does not, by itself, fail the serverStatus filtering test.
- Tests detect disallowed fields in filtered output and verify representative permitted-field and bypass behavior.
- Component coverage exercises filtering against a fixed document and the real policy; live coverage demonstrates that the command and privilege paths actually apply filtering. The responsibilities of each are documented.
- The getDiagnosticData check does not assume its periodic snapshot is contemporaneous with a direct serverStatus response or that FTDC applies metrics filtering.
- The intended live test cases and feature-flag states are observed on the relevant CI task, rather than inferred from fixture loops alone.