Now that mongos supports exhaust cursors, we want to reintroduce driver support for OP_MSG exhaust as a performance optimization.

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Component/s: None
    • None
    • Needed

      We want to introduce support for OP_MSG exhaust cursors in drivers, now that the server-side implementation is complete across both mongod and mongos (see SERVER-57297). This enables the driver to opt into a more efficient communication pattern where the server streams multiple getMore responses without waiting for explicit client acknowledgments, reducing round-trip latency in workloads that consume large result sets.

      Although the original driver ticket (DRIVERS-535) was closed due to a lack of clear customer demand and incomplete support in sharded topologies, that landscape has now changed:

      • As of MongoDB 7.1, mongos supports exhaust getMores, removing a key blocker that previously limited driver use of this optimization.
      • Initial sync and oplog fetching in the server already use exhaust cursors to reduce latency, demonstrating proven server-side value.
      • Drivers implementing streaming features, such as topology updates or long-lived reads, could benefit from reduced latency and overhead.

      Our goal is to evaluate and implement driver support for exhaust cursors, starting with a driver that already exposes similar capabilities (e.g., Go). This includes:

      • Updating the CRUD spec to add an exhaust option to FindOptions and AggregateOptions.
      • Clarifying the behavior of the exhaustAllowed and moreToCome flags in the OP_MSG spec.
      • Implementing driver logic to detect exhaust support via isMaster/hello, opt in when appropriate, and correctly interpret streamed replies.
      • Limiting initial rollout to non-sharded collections or explicitly documenting the behavior in sharded environments, given mongos does not use exhaust when communicating with shards.

      We will treat this as an optional performance optimization, not a required feature, and only enable it when both client and server support it. Based on internal evaluation or customer interest, we may choose to expand support to more drivers or promote exhaust as a recommended option for specific workloads (e.g., high-throughput analytics queries or batch reads).

              Assignee:
              Unassigned
              Reporter:
              Garaudy Etienne
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: