Expose `cursor.bufferedCount` from `ChangeStream`

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: Change Streams
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Use Case: Enable efficient batch-processing of change streams

      We have been using change streams for some time now: we created a wrapper around a ChangeStream instance that deals with resume token management (incl. recycle/deployment scenarios). This works like a charm for some time now.

      Recently, we introduced a new use case where we need more throughput. We have a working solution that adds configurable batching, but we needed to patch the ChangeStream class of the mongodb library to make it work efficiently. This ticket is a request to add that to the official package so we can stop using our yarn patch-ed version of the library.

      Long story short: we need access to the (public) bufferedCount() function of the (private) ChangeStream.cursor field.

      The bufferedCount value allows use to avoid triggering getMore (as tryNext does) calls during processing during "slower" moments, when the stream contains less changes.

      Note: Using a ChangeStreamCursor directly would give us access to the bufferedCount(), but we prefer using the existing ChangeStream-based solution instead of reimplementing existing functionality.

      User Experience

      • What is the desired/expected outcome for the user once this ticket is implemented?
        We have access to theĀ  ChangeStream.bufferedCount

      Dependencies

      • none

      Risks/Unknowns

      Since we are exposing the public AbstractCursor.bufferedCount() value, this seems like low/no-risk. Especially since ChangeStreamCursor.resumeToken is already exposed from ChangeStream.resumeToken is a similar way (we might as well expose the cursor directly, but that won't be needed).

      We think it could be interesting to include this in all language drivers.

      Acceptance Criteria

      Implementation Requirements

      Add ChangeStream.bufferedCount.

      Testing Requirements

      Not sure if this would be relevant.

      Documentation Requirements

      TypeScript docs can take care of this.

            Assignee:
            Unassigned
            Reporter:
            gino heyman
            None
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: