rank_fusion_lookup_local_foreign_field_view.js fails with 12982600 in with_mongot_extension_* suites

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • Fully Compatible
    • v9.0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      `jstests/with_mongot/e2e/hybridSearch/rank_fusion_lookup_local_foreign_field_view.js` tests
      `$rankFusion` inside a `$lookup` sub-pipeline joined via `localField`/`foreignField` syntax against
      a view, verifying the view filter is correctly applied to hybrid search results (including the
      multi-input-pipeline case that desugars to `$unionWith`). All 3 subtests currently fail in the
      `with_mongot_extension_*` suites with:

      ```
      Location12982600: $lookup with $rankFusion/$scoreFusion cannot use localField/foreignField syntax.
      ```

      Root cause: `src/mongo/db/pipeline/document_source_lookup.cpp`, in
      `DocumentSourceLookUp::createFromBson` (~line 1401-1415):

      ```cpp
      if (lookupSpec.getIsHybridSearch() || hybrid_scoring_util::isHybridSearchPipeline(pipeline))

      { // If there is a hybrid search stage in our pipeline, then we should validate that we // are not running on a timeseries collection. // // If the hybrid search flag is set to true, this request may have // come from a mongos that does not know if the collection is a valid collection for // hybrid search. Therefore, we must validate it here. hybrid_scoring_util::assertForeignCollectionIsNotTimeseries(fromNs, pExpCtx); // TODO SERVER-121094 Remove this assertion when featureFlagExtensionsInsideHybridSearch is // removed (this whole code path will be dead code as well) uassert(12982600, "$lookup with $rankFusion/$scoreFusion cannot use localField/foreignField syntax.", localField.empty() && foreignField.empty()); }

      ```

      Unlike the SERVER-131118/SERVER-130918 family of failures (which are about the IFR-kickback-to-legacy-`$vectorSearch`
      retry path only breaking when `featureFlagExtensionsInsideHybridSearch` is off), this `uassert` has
      no flag check at all — it unconditionally rejects `localField`/`foreignField` syntax whenever the
      `$lookup` sub-pipeline is a hybrid search pipeline (contains `$rankFusion`/`$scoreFusion`), in any
      flag configuration. The TODO comment references `featureFlagExtensionsInsideHybridSearch`, but the
      code does not gate on it.

      Per `git blame`, this `uassert` was added deliberately in commit ca2c0f6510470a41a1adc934d4621d87d296e22f
      ("SERVER-129826 Ban hybrid search in lookup subpipelines with local/foreign field matches", #56792),
      which appears to be intentionally banning this combination for now, with `SERVER-121094` tracking
      follow-up removal once `featureFlagExtensionsInsideHybridSearch` is permanently on (at which point
      this whole code path becomes dead code). This test file was added/still exercises the banned
      combination and was never updated to expect the ban.

      In `buildscripts/resmokeconfig/matrix_suites/overrides/with_mongot_extension.yml`, this file is
      listed in the `exclude_incompatible_tests` block, positioned among the seven files excluded under
      the "TODO SERVER-130913" comment (extension `$vectorSearch` inside `$lookup`/`$graphLookup` hitting
      tassert 12197200) — but it has no comment of its own, and its actual failure (12982600) is unrelated
      to SERVER-130913 or SERVER-130918. It looks like it was swept into that list without separate
      triage.

      Open question to resolve before fixing: is unconditionally rejecting `localField`/`foreignField`
      syntax with `$rankFusion`/`$scoreFusion` in `$lookup` sub-pipelines the intended, permanent behavior
      introduced by SERVER-129826 (in which case this test is simply wrong and should be rewritten to
      assert failure with code 12982600 — similar to how `jstests/with_mongot/e2e/vector_search/lookup_vector_search_disallowed.js`
      asserts the analogous flag-off rejection for `$vectorSearch`), or was SERVER-129826 meant to be a
      temporary restriction pending SERVER-121094/`featureFlagExtensionsInsideHybridSearch` removal, in
      which case the localField/foreignField + hybrid-search combination is supposed to eventually work
      and this test's expectations are correct but blocked on that flag work. Read PR #56792 and
      SERVER-129826/SERVER-121094 directly to confirm intent before concluding either way — don't assume.
      Either way, the exclusion in `with_mongot_extension.yml` should be given its own comment/ticket
      reference instead of riding along with the SERVER-130913 group.

      Verified locally:
      ```
      python3 buildscripts/resmoke.py run --suites=with_mongot_extension_single_node --force-excluded-tests jstests/with_mongot/e2e/hybridSearch/rank_fusion_lookup_local_foreign_field_view.js
      ```
      All 3 subtests fail with code 12982600, no crash, fixture setup/teardown clean.
      </content>

            Assignee:
            Finley Lau
            Reporter:
            Finley Lau
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: