Introduce BatchedEnrichmentStage, a generic buffering enrichment stage primitive

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • QE 2026-06-22, QE 2026-07-06
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      Add a reusable aggregation execution-stage base that buffers upstream events while
      holding no resources, enriches each one inside a bounded resource scope, then
      drains them. This lets any stage that enriches events via a point lookup amortize
      per-event resource acquisition without holding acquisitions or locks across
      getNext() calls. The primitive is enrichment-agnostic; it names no concrete
      consumer.

      Scope

      • BatchedEnrichmentStage base: fill -> enrich -> drain loop; constructor-injected
        limits (max buffered events, max buffered input bytes, max enriched-output bytes
        with mid-input suspend); a memory tracker over both buffers that feeds operation
        memory accounting and drives the input byte cap; per-iteration interrupt checks;
        pass-through of control documents in arrival order; guaranteed scope close even
        when enrichment throws; a test failpoint.
      • SingleDocumentLookupExecutor::releaseResources(): release attached state
        while cached plans survive; the PrimaryWithFallback combinator releases the primary
        before invoking the routed fallback and forwards the call to both children.
      • ScopedBatchedLookup: RAII scope that reattaches the executor on construction
        and releases + detaches it on destruction; move-safe.

      Out of scope

      Wiring any concrete stage onto the base.

      Testing

      Unit tests only: combinator release ordering and forwarding; ScopedBatchedLookup
      lifecycle including the exception path; stage fill/enrich/drain, the caps, output
      suspend, pause/EOF propagation, control pass-through, memory-tracker accuracy, and
      interrupt handling.

            Assignee:
            Denis Grebennicov
            Reporter:
            Denis Grebennicov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: