changestream tryNext should update the resume token when returning a change

    • 3
    • Not Needed
    • None
    • Needed
    • Hide

      Following the root-cause fix, this is a follow-up to @DOCSP-51959. Please update the warning to name the specific driver version before which this issue occurs (i.e., all versions earlier than X.Y.Z).

      Show
      Following the root-cause fix, this is a follow-up to @DOCSP-51959. Please update the warning to name the specific driver version before which this issue occurs (i.e., all versions earlier than X.Y.Z).
    • None
    • None
    • None
    • None
    • None
    • None

      Node's change stream tryNext helper doesn't process change stream events (by calling the _processChange) helper, so the resume token for the change stream is not updated when a change is returned to the user.  This has been the behavior since tryNext was implemented (https://github.com/mongodb/node-mongodb-native/commit/634ae4f93013c824bc153f132f0dd6c9bd2cc127).  

      This can cause a number of issues

      • change streams aren't resumed properly if users are relying on `tryNext` in a loop
      • change streams can produce duplicate change events

      I believe this wasn't caught with existing tests for a few reasons:

      • our change stream test coverage isn't exhaustive (there is no test for resumability that checks for duplicate resume events).
      • our unified tests only test the `next` helper
      • most importantly, the resumability tests only test resuming on a single change event after initializing the change stream.  this is important because the `aggregate` call always sets the postBatchResumeToken if present, so a change stream that errors on the first change and resumes will resume properly and receive no duplicate events.

      Use Case

      As a... user of the tryNext api
      I want... resume tokens to be updated correctly on change events
      So that... I can resume the change stream from the correct point in time

      User Experience

      Dependencies

      • N/A

      Risks/Unknowns

      • N/A

      Acceptance Criteria

      Implementation Requirements

      • make change stream's tryNext update the resume token while processing changes

      Testing Requirements

      • Integration test to ensure that tryNext updates the resume token

      Documentation Requirements

      • Update docs to update unreliability callout for tryNext, noting that it's only present up to version 6.19.0 (double check against fix version)

      Follow Up Requirements

      • N/A

            Assignee:
            Sergey Zelenov
            Reporter:
            Bailey Pearson
            Bailey Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: