Sharded local change-stream updateLookup fails getMore with NamespaceNotFound when the database was dropped, instead of a null fullDocument

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1022
    • Affects Version/s: 9.0.2
    • Component/s: None
    • None
    • Query Execution
    • Fully Compatible
    • ALL
    • v9.0
    • QE 2026-09-15, QE 2026-09-28
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      On a sharded cluster, a cluster-wide change stream with fullDocument: "updateLookup" aborts a getMore with NamespaceNotFound (26) when an update/replace event's database has been dropped before the stream reads that event:

      Error on remote shard localhost:28026 :: caused by :: Executor error during getMore
        :: caused by :: database nsnotfound_repro_db_0 not found
      

      The expected result is a null fullDocument. This is explicitly the intended behavior per SERVER-129059's own acceptance criteria, which state: "keep NamespaceNotFound and CollectionUUIDMismatch mapping to a null fullDocument".

      That mapping appears to cover a missing collection but not a missing database — dropping only the collection behaves correctly on the same build.

      Versions

      • Fails: 9.0.2
      • Works: 8.0.16-353-gb19191c (returns null post-images, drains cleanly)

      The boundary is consistent with SERVER-129059 / featureFlagChangeStreamOptimizedUpdateLookup (epic SPM-4535), which enabled the shard-local lookup path on sharded nodes and landed on master only. Not yet confirmed by bisect or by toggling the flag — see "Suggested next step" below.

      Reproduction

      Attached mongosh script, run against a mongos. Per iteration it creates a collection (sharded on _id: hashed), does insert/update/replace/delete for each of 10 documents, then tears down. The stream is opened afterwards with startAtOperationTime rewound to before the writes, which reproduces a lagging reader deterministically instead of racing the teardown. It uses raw aggregate + getMore commands rather than a driver watch() helper, so that driver-level resume logic does not mask the error.

      Isolation

      teardown 8.0.16-353-gb19191c 9.0.2
      dropDatabase null post-images NamespaceNotFound (26)
      dropDatabase + matchCollectionUUIDForUpdateLookup null post-images NamespaceNotFound (26)
      drop (collection only) null post-images null post-images
      none (documents deleted only) null post-images null post-images

      Notes:

      • Every variant deletes the documents before teardown, so the post-image is absent in all four rows. The missing-document path is therefore fine on 9.0; only a missing database errors.
      • matchCollectionUUIDForUpdateLookup makes no difference either way.

      Additional impact: the whole batch is lost

      The failing getMore returns zero events, even though the create, insert, and pre-drop update events all precede the doomed lookup in the same oplog range. A reader cannot skip past the bad event, because the events that were readable are discarded with it.

      How this was found

      MongoDB's Cluster-to-Cluster Sync (mongosync) embedded verifier opens a cluster-wide change stream with fullDocument: "updateLookup" and does not treat code 26 as retryable, so this is fatal to a sync run. It reproduced in mongosync CI against a 9.0 sharded source whenever a workload creates and drops databases while change event application is in progress.

      Suggested next step

      Re-running the attached script with featureFlagChangeStreamOptimizedUpdateLookup disabled would confirm or rule out SERVER-129059 as the origin; if the flag-off run drains cleanly, the gap is in the Express lookup body's error mapping for a nonexistent database.

            Assignee:
            Denis Grebennicov
            Reporter:
            Felipe Gasper
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: