Create aggregation pipeline helpers

XMLWordPrintableJSON

    • Type: Epic
    • Resolution: Unresolved
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: Aggregation
    • Go Agg Builder
    • Go Drivers
    • None
    • Hide
      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?
      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • In Progress
    • 0
    • 5
    • 5.5
    • 100
    • 🟢 On Track
    • Hide

      Engineer(s): Matt Dale, Lin Borland

      What was accomplished since the last update?
      We implemented all of the operator functions in the mql specs and their corresponding tests, adding additional test coverage for operators with optional parameters.
      We established an optional parameters pattern and applied it to all relevant operators and stages.
      We removed field constructors that weren't being used in the package.
      We brainstormed solutions for handling multiple queries within the $match stage. We also began implementing more of the query operators, and writing tests for those.

      What's the focus over the next two weeks?
      1. Establish a usable pattern for handling multiple queries within the $match stage.
      2. Finish query operators and tests, completing the query builder package.
      3. (stretch goal) Implement the rest of the stages and tests in the mql specs.
      4. Begin putting together content on the aggregation pipeline builder API and intern final presentation.

      Any risks/blockers/impediments?
      None.

      Anything else to share?
       -

      Show
      Engineer(s): Matt Dale, Lin Borland What was accomplished since the last update? We implemented all of the operator functions in the mql specs and their corresponding tests, adding additional test coverage for operators with optional parameters. We established an optional parameters pattern and applied it to all relevant operators and stages. We removed field constructors that weren't being used in the package. We brainstormed solutions for handling multiple queries within the $match stage. We also began implementing more of the query operators, and writing tests for those. What's the focus over the next two weeks? 1. Establish a usable pattern for handling multiple queries within the $match stage. 2. Finish query operators and tests, completing the query builder package. 3. (stretch goal) Implement the rest of the stages and tests in the mql specs. 4. Begin putting together content on the aggregation pipeline builder API and intern final presentation. Any risks/blockers/impediments? None. Anything else to share?  -
    • Hide

      2026-07-09 - 🟢 On Track
      Engineer(s): Lin Borland, Matt Dale

      What was accomplished since the last update?
      We implemented the majority of the builder API for aggregation pipeline stages, accumulators, and operators. We also created a package for expressing query logic for use in the "$match" stage. Most implemented builder API functions have at least one test from the spec tests in https://github.com/mongodb/mql-specifications

      We started making usability improvements to the API, including experimenting with two simple optional parameter patterns, both of which were rejected by review as unusable.

      What's the focus over the next two weeks?
      1. Usability improvements for the API, including defining an easy-to-use pattern for optional params.
      2. Improve test coverage and add some Go "testable examples" that also act as user documentation.
      3. Developer experience demo.

      Any risks/blockers/impediments?
      None.

      Anything else to share?
       -

      Show
      2026-07-09 - 🟢 On Track Engineer(s): Lin Borland, Matt Dale What was accomplished since the last update? We implemented the majority of the builder API for aggregation pipeline stages, accumulators, and operators. We also created a package for expressing query logic for use in the "$match" stage. Most implemented builder API functions have at least one test from the spec tests in https://github.com/mongodb/mql-specifications We started making usability improvements to the API, including experimenting with two simple optional parameter patterns, both of which were rejected by review as unusable. What's the focus over the next two weeks? 1. Usability improvements for the API, including defining an easy-to-use pattern for optional params. 2. Improve test coverage and add some Go "testable examples" that also act as user documentation. 3. Developer experience demo. Any risks/blockers/impediments? None. Anything else to share?  -
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Defining aggregation pipelines using the Go driver can be a bad experience. The bson.D type (which is frequently used to define agg pipelines) does not provide compile-time safety for types used in aggregation pipelines and the syntax becomes difficult to read quickly as aggregation pipelines become more complex.

      Some other drivers, such as Java and .NET, provide helper functions that add a convenience (and sometimes type-safety) layer on top of the aggregation pipeline syntax, significantly improving the developer experience of using aggregation pipelines. Creating helper functions in the Go driver would provide a similar improvement in the experience of using aggregation pipelines in Go.

      Creating a comprehensive set of aggregation pipeline helpers is a very large task, so to help improve the experience for the majority of users faster, we should target creating helpers for the most popular aggregation pipelines. The helpers need to be able to interoperate with existing bson.D pipeline declaration logic so that users have the option to use a mixture of pipeline stages defined with the helpers and with bson.D.

      Definition of done:

      • Create a set of aggregation pipeline helpers that output aggregation pipeline stages.
      • The helper must be able to be combined with other aggregation pipeline stages defined as bson.D or bson.M (i.e. allow incrementally replacing pipeline stages defined with bson.D/bson.M with helper calls without replacing the entire pipeline).
      • Update all documentation to use the aggregation pipeline helpers instead of bson.D/bson.M where possible.

      Open questions:

      • Should we create a new package (e.g. aggregate.Match), a new type (e.g. mongo.Aggregate().Match), or something else?
      • Should the helper output bson.D or another type? It must be possible to mix helper output with other pipeline stages defined using bson.D/bson.M.

            Assignee:
            Lin Borland
            Reporter:
            Matt Dale
            None
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              6 weeks, 4 days
              None
              None

                Estimated:
                Original Estimate - 4 weeks
                4w
                Remaining:
                Time Spent - 2 weeks Remaining Estimate - 2 weeks
                2w
                Logged:
                Time Spent - 2 weeks Remaining Estimate - 2 weeks
                2w