Streams: Consolidate stats, metrics, and memory tracking

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None

      This is a backlog ticket to think about how we can consolidate the different ways we track user-facing stats, internal metrics, and memory tracking.

      The streaming engine tracks both:

      The streaming engine also tracks memory consumption with individual gauges,     mongo::MemoryUsageHandle instances, and OperatorStats::memoryUsageBytes.

      Are there opportunities to simplify how we track these related things into a more singular abstraction?

      One can imagine the following:

      • Use prometheus style Counter, Gauge, Histogram, and Summary metric instances for tracking everything.
      • OperatorStats and MemoryUsageHandle can be a view on top of metric instances.

      It's not clear whether this refactoring is worth it anytime soon. But it's worth thinking about given how much code we have for stats and metrics.

            Assignee:
            Unassigned
            Reporter:
            Matthew Normyle
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: