Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-93627

Improve error handling for change stream $source watch() errors

    • Atlas Streams
    • Sprint 61, Sprint 62

      See this section for how errors are categorized as "user errors" by the engine: https://docs.google.com/document/d/1HqpIU-y0E5Gal5iY4YjLT6CdHQdPzSzv2I5__otZZgY/edit?tab=t.0#heading=h.xaubq3o7hqau 

      This ticket involves fixing a number of change stream related errors that the engine currently calls "internal errors", when they are really "user errors".

      Issue 1: Invalid/un-parseable pipeline in $source.config.pipeline

      When watch is called with a $source.config.pipeline, we can get back lots of different errors from the target server (if the pipeline fails to parse, the agg layer on the target server might return lots of different error codes).

      For the "first start" of a stream processor, we should report such errors as a user error. The logic is: if there is no restore checkpoint, if $source.config.pipeline is set, any non network error return code from the .watch() call is a user error. 

      Issue 2: $replaceRoot

      Also, need to handle the $source.config.pipeline=[$replaceRoot] case, which leads to errors like the below. These should bubble up as user errors:

      Change stream $source petra.financialTransaction failed: PlanExecutor error during aggregation :: caused by :: Encountered an event whose _id field, which contains the resume token, was modified by the pipeline. Modifying the _id field of an event makes it impossible to resume the stream from that point. Only transformations that retain the unmodified _id field are allowed 

      Issue 3: Time series changestream

         error: Change stream $source db.coll failed: Cannot run aggregation on timeseries with namespace db.coll: generic server error 

       

            Assignee:
            guyjacques.isombe@mongodb.com Guy-Jacques Isombe
            Reporter:
            matthew.normyle@mongodb.com Matthew Normyle
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: