tassert(10978000) crash on router when extension $search is inside $lookup with featureFlagExtensionsInsideHybridSearch=false

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      When a router has `featureFlagSearchExtension=true` and `featureFlagExtensionsInsideHybridSearch=false`, running an aggregation with a `$lookup` whose sub-pipeline contains `$search` crashes the router with `tassert(10978000)`.

      The IFR kickback that should force this pipeline back to the legacy `$search` parser is placed on the post-desugar path (`DocumentSourceExtensionOptimizable::create()`). However, `$lookup` builds its sub-pipeline with `kOptionsMinimal` (`desugar=false`), so the pre-desugar wrapper `DocumentSourceExtensionForQueryShape` survives into wire dispatch — the exact state the tassert guards against. The same issue affects `$unionWith` sub-pipelines.

      *Affected versions*

      Regression introduced by `SERVER-117259` (`3d110ebc534c`, 2026-06-15), which lifted the pre-desugar wrapper's `LookupRequirement` from `kNotAllowed` to `kAllowed`, making the kickback in `SERVER-122433` unreachable on the `$lookup` sub-pipeline path.

      *Steps to reproduce*

      Start a sharded cluster with:
      ```
      featureFlagSearchExtension: true
      featureFlagExtensionsInsideHybridSearch: false
      ```
      Run:
      ```js
      db.coll.aggregate([{
      $lookup: {
      from: "coll",
      pipeline: [{ $search: {} }, { $project:

      { _id: 1 }

      }],
      as: "matches"
      }
      }])
      ```
      *Expected:* `ErrorCodes.SearchNotEnabled` (31082) — the router kicks back to the legacy parser, which fails because no mongot is running.
      *Actual:* Router crashes with `tassert(10978000)` in `DocumentSourceExtensionForQueryShape::serialize`.

      A self-contained reproducer exists at `jstests/noPassthrough/extensions/search_extension_in_lookup_sharded_wire_dispatch.js`.

      *Root cause*

      Two commits compounded the issue:

      1. *SERVER-122433* placed the `featureFlagExtensionsInsideHybridSearch` kickback on `DocumentSourceExtensionOptimizable::create()` (post-desugar), with a TODO noting it would only be reachable once SERVER-117259 lifted the pre-desugar constraint.
      2. *SERVER-117259* lifted that constraint and removed the TODO comment, without accounting for the fact that `DocumentSourceLookUp::initializeResolvedIntrospectionPipeline` builds sub-pipelines with `kOptionsMinimal` (`desugar=false`) — meaning the pre-desugar wrapper is never replaced by `DocumentSourceExtensionOptimizable`, and the kickback is never reached.

      *Proposed fix*

      Add the IFR kickback logic to the pre-desugar construction path in `expandableStageParamsToDocumentSourceFn` (`document_source_extension_optimizable.cpp`), before the `DocumentSourceExtensionForQueryShape` wrapper is constructed. Extract a shared helper (e.g., `throwHybridSearchKickbackForExtensionIfNecessary`) so the pre- and post-desugar sites stay in sync. This covers both `$lookup` and `$unionWith` sub-pipelines.

      No changes needed to the router/shard retry loop — it is already exception-driven and will handle the kickback correctly.

      Alternatives considered and rejected: flag-gating `constraints()` (no `ExpressionContext` access), lite-parse-time kickback (requires plumbing IFR context through `LiteParserOptions`), and forcing `kDesugarOnly` in `initializeResolvedIntrospectionPipeline` (high blast radius, doesn't fix `$unionWith`).

      *Test plan*

      • Add a new C++ unit test suite `DocumentSourceExtensionForQueryShapeInLookupKickbackTest` (symmetric with the existing `DocumentSourceExtensionOptimizableInLookupKickbackTest`) that constructs the pre-desugar wrapper via `expandableStageParamsToDocumentSourceFn` with `expCtx->setInLookup(true)` and asserts `IFRFlagRetry` is thrown.
      • Confirm the existing multiversion test `jstests/multiVersion/genericBinVersion/ifr_search_extension_sharded_upgrade.js` passes (`searchInLookup` scenario now receives `SearchNotEnabled`).
      • Run neighboring jstests to guard regressions: `search_ifr_flag_retry.js`, `extension_in_subpipeline_rejected.js`, `extension_in_lookup_flag_off.js`, `extension_in_lookup_with_views.js`.

      *Related*

      • *SERVER-117259* — regression source (lifted `LookupRequirement` on pre-desugar wrapper)
      • *SERVER-122433* — introduced the kickback (in the wrong layer)
      • *SPM-4488* — long-term plan to move to LiteParsed query shapes, which would eliminate the pre-desugar wrapper entirely

            Assignee:
            Charlie Swanson
            Reporter:
            Charlie Swanson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: