ExportXMLWordPrintableJSON

    • Type: Investigation
    • Resolution: Won't Do
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Tools and Replicator

      Original Downstream Change Summary

      Downstream Changes Summary

      Project context
      ---------------
      Change stream updateLookup resolves the fullDocument for update events via a point lookup on the
      document's _id. With the optimization flag on (featureFlagChangeStreamOptimizedUpdateLookup), the
      lookup runs on a primary engine that can DECLINE the lookup (e.g. an _id shape it cannot handle);
      declines fall through to a universal fallback: the routed Aggregation executor, which runs a
      $match-on-_id aggregation and can serve any lookup (compound/object _id, collation, clustered
      collections, TooManyMatchingDocuments). With the flag off, the aggregation executor is the only
      engine used. Each engine is instrumented separately, so the metrics show which engine served
      which lookups.

      What this ticket adds
      ---------------------
      The previously zeroed metrics.changeStreams.updateLookup.aggregation.* serverStatus metrics
      (defined in SERVER-129512) are now populated: the aggregation executor records its cell on every
      lookup it serves - i.e. all lookups when the optimization is off, plus every decline-fallback
      when it is on. A sustained aggregation counter share therefore indicates the primary engines are
      declining.

      Metric semantics
      ----------------

      • found (counter): the lookup ran on the aggregation executor and resolved to a document.
      • notFound (counter): the lookup ran and did not resolve to a document (deleted/missing document;
        a dropped collection or database is mapped to not-found).
      • latencyMicros (histogram, microseconds): wall time of a single lookup attempt, recorded only
        for found/notFound.

      No new commands, options, response fields, or wire-protocol changes; no change to lookup results

      • observability wiring only.

        Description of Linked Ticket

        Context

        The Aggregation executor is the universal fallback and the entire lookup path when the IFR flag is
        off — the executor running in production today. Instrumenting it first lands a working end-to-end
        metric on the safest path.

      Scope

      • Add a SingleDocumentLookupStatsRecorder constructor parameter to AggregationSingleDocumentLookupExecutor; store it.
      • In performLookup, call recordFound / recordNotFound with the measured latency. The Aggregation executor always handles the lookup (it is the universal fallback and never returns kNotHandled), so it never records notHandled.
      • Update the stage-build factory (makeUpdateLookupStage site) to inject the updateLookupAggregationStats cell.

      Acceptance

      With the IFR flag OFF, a stream with updates increments changeStreams.updateLookup.aggregation.found / notFound and the latency histogram.

      Tests (definition of done)

      Create the shared replset jstest jstests/change_streams/change_stream_metrics_update_lookup.js:

      • tagged assumes_against_mongod_not_mongos + assumes_read_preference_unchanged + assumes_no_implicit_cursor_exhaustion (per the change_stream_metrics_* family).
      • baseline -> operate -> delta on serverStatus.
      • flag-off branch: assert the aggregation cell increments.
      • topology-agnostic invariants (found + notFound + notHandled reconcile; monotonic).
      • compute the expected cell as expectedEngine(flag, changeStreamPassthroughType()) so the Express and SBE wiring tickets can add their branches without restructuring.

      Dependencies

      Depends on the metrics infrastructure ticket, and AggregationSingleDocumentLookupExecutor (SERVER-128409).

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: