$changeStreamSplitLargeEvent returns Location7182803 error when used with non-renameable match expression

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 7.3.4, 7.0.13, 6.0.17, 8.0.0-rc11
    • Affects Version/s: 6.0.15
    • Component/s: None
    • None
    • Query Execution
    • Fully Compatible
    • ALL
    • v8.0, v7.3, v7.0, v6.0
    • Hide

      The following call currently returns Location7182803 error, while it is supposed either to pass or return ChangeStreamFatalError error with "cannot resume stream; the resume token was not found".

      testColl.watch(
    [{"$match": {"fullDocument.systemUuids": {"$size": 1}}},
     {"$changeStreamSplitLargeEvent": {}}],
    {
        "fullDocument": "whenAvailable",
        "resumeAfter": {
            "_data": "82666149BC000000012B042C0100296E5A10047986D56C8C194743B21A906C8EDE2A46463C6F7065726174696F6E54797065003C7570646174650046646F63756D656E744B65790046645F69640064650DD8A2FE91267B59DAB356000004"
        }
    });
      Show
      The following call currently returns Location7182803 error, while it is supposed either to pass or return ChangeStreamFatalError error with "cannot resume stream; the resume token was not found". testColl.watch( [{ "$match" : { "fullDocument.systemUuids" : { "$size" : 1}}}, { "$changeStreamSplitLargeEvent" : {}}], { "fullDocument" : "whenAvailable" , "resumeAfter" : { "_data" : "82666149BC000000012B042C0100296E5A10047986D56C8C194743B21A906C8EDE2A46463C6F7065726174696F6E54797065003C7570646174650046646F63756D656E744B65790046645F69640064650DD8A2FE91267B59DAB356000004" } });
    • QE 2024-06-24, QE 2024-07-08
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The $changeStreamSplitLargeEvent stage is expected to move before the $_internalChangeStreamEnsureResumeTokenPresent stage when the latter stage is present. If this does not happen the user assertion 7182803 is thrown.

      It appears that it will not happen when there is a non-renameable user-defined $match stage in the change stream pipeline. In such a case, this $match stage will remain between $_internalChangeStreamEnsureResumeTokenPresent and $changeStreamSplitLargeEvent stages after the optimization, which will block the $changeStreamSplitLargeEvent from moving ahead of $_internalChangeStreamEnsureResumeTokenPresent.

      Possible solutions:

      • Remove the assertion. This solution is not practical, because in the presence of non-renameable $match the $changeStreamSplitLargeEvent stage will not move before $_internalChangeStreamEnsureResumeTokenPresent and $_internalChangeStreamHandleTopologyChange and won't be pushed into the shard pipelines and will fail to split large events created on the shards with BSONObjectTooLarge error.
      • Always move $match before the two stages. This solution might interfere with SERVER-71565 and SERVER-67677 and change the behaviour of some change stream pipelines (resumable pipelines might become non-resumable).
      • Backport SERVER-75853. Large backports like this are risky and likely not solve the problem entirely. And we need also SERVER-73914 (thanks to catalin.sumanaru@mongodb.com for pointing at this).

      Solving this problem might also improve the performance of $match stages, because they will be "pushed-down" and eventually merge with other $match stages improving the overall performance.

      Regardless of this problem we should consider the following:

      • Document any additional limitations of $changeStreamSplitLargeEvent (see also SERVER-91313).
      • Use ChangeStreamFatalError instead of 7182803, because 7182803 is of NonspecificAssertionException category and it won't be logged with the default logging settings.
      • Add the $changeStreamSplitLargeEvent stage to the change stream fuzzer (DEVPROD-8085).

            Assignee:
            Romans Kasperovics
            Reporter:
            Romans Kasperovics
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: