Summary

When $lookup.from is a view whose pipeline desugars to a mongot-style source stage plus $_internalSearchIdLookup, the foreign-view binding step binds the view's own pipeline onto that idLookup stage. On an unsharded deployment idLookup then prepends $match: <documentKey> to the bound view pipeline, displacing the PositionRequirement::kFirst source stage, and the query fails with:

Executor error during aggregate command on namespace: test.view_definition_variety_local :: caused by ::
$produceIds is only valid as the first stage in a pipeline
code: 40602, codeName: "Location40602"

On a sharded cluster the same query succeeds. $unionWith and $graphLookup are unaffected.

The topology divergence is the defect being reported here. Which of the two behaviors is correct is an open design question — see "Design fork" below. It is possible that the sharded behavior is the buggy one.

Observed in all four $lookup shapes on unsharded: pipeline syntax with 0-, 2-, and 3-stage user subpipelines, and localField/foreignField with no user subpipeline.

Root cause

Established with instrumented builds (LOGV2 tracing through the $lookup view-binding and idLookup execution paths) on jstests/extensions/view_definition_variety.js, run in extensions_standalone and extensions_sharded_cluster.

1. DocumentSourceLookUp::resolvedPipelineHelper builds the subpipeline correctly. Logged resolvedPipeline: [$readNDocuments], [$readNDocuments, $matchTopN, $addFields], [$readNDocuments, $addFields, $testBar, $sort], and for the localField/foreignField shape [$readNDocuments] with _fieldMatchPipelineIdx = 1, canSplitAtFieldMatch = true. The source stage is at index 0 and the join $match is correctly placed after it. Nothing is prepended ahead of the view's stage.
2. LookUpStage::buildPipeline runs makeLookupViewBinder (src/mongo/db/exec/agg/lookup_stage.cpp:92-104) → PipelineResolver::resolveInvolvedNamespacesOnLiteParsedPipeline(..., bindOnly=true). LiteParsedInternalSearchIdLookUp::bindResolvedNamespace (src/mongo/db/pipeline/search/lite_parsed_internal_search_id_lookup.h:81-89) stores the view's own desugared pipeline onto the idLookup stage. Logged: $_internalSearchIdLookup: {viewPipeline: [{$produceIds: {}}, \{$_internalSearchIdLookup: {}}]}{} — the stage is bound into itself.
3. IdLookupLocalReadExecutor (src/mongo/db/exec/agg/search/internal_search_id_lookup_local_read_executor.cpp:25-38) builds [{$match: documentKey}] and appends viewPipeline, producing [$match, $produceIds, $_internalSearchIdLookup]. Pipeline::validateCommon (src/mongo/db/pipeline/pipeline.cpp:283) throws 40602.

Because the throw happens in the per-document local read, the error surfaces as "Executor error during aggregate command" rather than as a parse failure.

Why unsharded only (standalone and replica set)

The router never marks the foreign namespace as a view, serializes from as the backing collection, and ships the subpipeline already desugared. Shard-side, isInvolvedNamespaceAView() is false, so makeLookupViewBinder returns early (lookup_stage.cpp:96-99) and idLookup receives no viewPipeline. Confirmed by contrast on the same query:

• unsharded: resolvedPipeline: [{$readNDocuments: {numDocs: 3}}], desugared to [$produceIds, {$_internalSearchIdLookup: {viewPipeline: [$produceIds, $_internalSearchIdLookup]]}} → throws.
• sharded: resolvedPipeline: [{$produceIds: {numDocs: 3}}, \{$_internalSearchIdLookup: {}}] → viewPipeline unset → succeeds (7/7 tests pass).

$unionWith and $graphLookup are unaffected because they do not run makeLookupViewBinder.

Design fork: which behavior is correct?

This needs to be decided before implementing, because the options require different fixes and opposite test expectations.

Option A — it should succeed. Precedent: a view defined as [{$readNDocuments: ...}] wrapped in a second view already works today (jstests/extensions/executable_extension_stage_in_view.js:76-95) — the generator stays at index 0 and later stages are appended after it. The sharded path also already succeeds. Under this option the unsharded path is broken and should be made to match sharded.

Option B — it should be rejected with a clear error. search_index_view_validation (src/mongo/db/query/search/search_index_view_validation.cpp:88-104) allows only $addFields/$set not touching _id, and $match containing only $expr, in a bound view pipeline — because a search-indexed view's transforms must be ones mongot can mirror. By that contract, a view definition containing a source generator is precisely what should be rejected, and the correct outcome is a "view definition is incompatible" error on all topologies. Under this option the unsharded 40602 is right in outcome but has a misleading message (it blames the user's stage ordering when the server bound the view into the stage internally), and the sharded path is the buggy one for silently accepting the shape.

Supporting evidence for Option B: the equivalent legacy shape is already banned. A view defined as [{$search: ...}] used as $lookup.from fails with 10623000, asserted at jstests/with_mongot/e2e/views/mongot_stage_in_view_definition.js:238-264.

Whichever option is chosen, the intentional 40602 at jstests/extensions/executable_extension_stage_in_view.js:66-75 must keep throwing. That is a different situation: there $readNDocuments appears in the user pipeline and declares kDefaultPrepend (the SDK default at src/mongo/db/extension/sdk/aggregation_stage.h:396-398; it never overrides getFirstStageViewApplicationPolicy), so the view legitimately precedes and displaces it.

Reachability (bears on priority)

Not reachable through shipped $search/$vectorSearch/$searchMeta: that shape is already banned with 10623000 (see above). Reaching 40602 requires a loaded extension exposing a source-generator stage that desugars to a mongot-style source plus $_internalSearchIdLookup, and a view defined as that stage. In-tree, only the test examples ($readNDocuments) do this. On that basis this looks like a latent gap in the extensions API surface rather than a regression reachable by shipped functionality.

Open question that could change the priority call, and which has NOT been tested: search_index_view_validation::validate() is invoked only from the legacy document_source_search.cpp / document_source_vector_search.cpp / document_source_search_meta.cpp. Extension search stages do not call it. If extension $search in a view definition therefore bypasses the 10623000 ban and lands on this 40602 path instead, this becomes 9.0-relevant, since featureFlagSearchExtension and featureFlagExtensionsInsideHybridSearch are both default: true, fcv_gated: false, incremental_rollout_phase: release. This should be the first thing checked.

Hypotheses already falsified by instrumentation

Recorded so they are not re-derived:

• "$lookup prepends a foreign source ahead of the view's stages" — false. Nothing is prepended; resolvedPipeline is correct. The comments currently in view_definition_matrix.js and extension_optimizations_in_union_with_lookup.js state this and are inaccurate; they should be corrected alongside the fix.
• Double view application via makePipelineFromViewDefinition on the CommandOnShardedViewNotSupportedOnMongod retry — false. A log in that catch block never fired.
• Generalizing shouldDiscardViewPrefix / isSourceStage beyond the mongot stage-name allowlist — wrong function. That code inspects the user subpipeline's first stage, not the view prefix, and never runs for this view. The defect is in view binding, not view prefixing.

Reproduction

bazel build install-dist-test
bazel build install-extensions
python3 buildscripts/resmoke.py run \
--suites=extensions_standalone \
jstests/extensions/view_definition_variety.js \
--runAllFeatureFlagTests

The "source extension" view definition in jstests/extensions/libs/view_definition_matrix.js is [{$readNDocuments: {numDocs: 3}}], which desugars to [$produceIds, $_internalSearchIdLookup]. Run the same test in extensions_sharded_cluster to see it pass.

Test pins to flip when this is fixed

Currently pinned in three places:

• jstests/extensions/libs/view_definition_matrix.js — lookupFailsWithCode: 40602 on the "source extension" definition
• jstests/extensions/view_definition_variety.js — assert.throws guarded by {
  Unknown macro: {!FixtureHelpers.isMongos(db)}

  }

• jstests/extensions/extension_optimizations_in_union_with_lookup.js — two kFirst: "lookupGap" placements plus a continue that skips $lookup entirely

All three fail loudly if the error stops occurring, so a fix will not go unnoticed. The third is easy to miss and would otherwise keep silently skipping $lookup coverage after a fix.

Suggested verification set: extensions_standalone, extensions_sharded_cluster, extensions_single_node, executable_extension_stage_in_view.js as the regression guard for the intentional 40602, and the mongot e2e view suites, since the affected code is shared mongot idLookup binding rather than extension-only code.

Related tickets

• SERVER-131308 — extension $vectorSearch in a $lookup subpipeline on a view is not first after view resolution. Closest ancestor; introduced the bindOnly resolution mode this bug flows through. Fixed the view-prefixing path, not the view-binding path.
• SERVER-131470 — extension $search in a sharded $lookup with a localField/foreignField join, also Location40602, but no view involved.
• SERVER-131212 — the view case in computeDesugaredMongotFieldMatchIdx, surfacing as Location12761201.
• SERVER-132879 / SERVER-133203 — join $match position bugs causing silent wrong results on sharded clusters. Adjacent in the same test matrix, but a different defect.