updateLookup change stream returns null fullDocument for updates on a shard-local/uncataloged database

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1023
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • Fully Compatible
    • ALL
    • v9.0
    • QI 2025-02-17
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      An updateLookup change stream watching a collection in a database that exists only in a shard's local catalog (no config.databases entry — e.g. mongot's __mdb_internal_search materialized-view database, created directly on the shard it is attached to) silently returns fullDocument: null for update events, instead of the real post-image.

      Root cause

      The updateLookup shard-targeting sub-pipeline dispatch (sharded_agg_helpers.cpp, ShardedClusterLocalLookupEligibility::run()) routes the lookup through CatalogCache::getDatabase(), which throws NamespaceNotFound(26) for a database with no sharding-catalog entry. SERVER-134745 made the server swallow that NamespaceNotFound during pipeline construction and report a null post-image rather than propagate the error — correct for a genuinely-dropped collection, but wrong here: the database and document are both present, just shard-local by design.

      Impact

      Any consumer relying on updateLookup for such a database gets a false "document not found" signal for $set-only updates, even though the document exists. (For mongot specifically, this causes filter-only updates to autoEmbed vectorSearch filter fields to silently never reach the index.)

      Fix (as implemented)

      The fix is client-origin based, not namespace based: LocalLookupEligibilityFactoryImpl::makeLocalLookupEligibility now checks whether the connecting client is an internal client (i.e. routed via mongos or another cluster member), mirroring the existing MongoProcessInterface selection in mongod_process_interface_factory.cpp. A client that connected directly to a shard bypassed the routing protocol entirely, so it is treated as replica-set-like and receives AlwaysLocalEligibility. Only a routed, internal-client connection proceeds to ShardedClusterLocalLookupEligibility and consults the catalog cache.

      An earlier namespace-based approach (extending NamespaceString::isShardLocalNamespace() / isNamespaceAlwaysUntracked() to recognize _mdb_internal_search, plus a DDL guard in create_database_util.cpp) was implemented first but reverted during review: _mdb_internal_search also holds mongot's index catalog collections, which are legitimately registered and routed via mongos, so a database-wide predicate would have broken that registration path.

      Merged: https://github.com/10gen/mongo/pull/62686

            Assignee:
            Denis Grebennicov
            Reporter:
            Denis Grebennicov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: