-
- Symptom
On a sharded cluster, an aggregation over a chain of views involving a bare `$lookup`
(`localField`/`foreignField`, no user `pipeline:`) could crash a shard with:
```
tassert 12792401: expected an empty subpipeline list when materializing a view subpipeline
src/mongo/db/pipeline/lite_parsed_document_source_nested_pipelines.h
LiteParsedDocumentSourceNestedPipelines<LiteParsedLookUp>::materializeViewSubpipeline()
```
First seen in `concurrency_sharded_replication_last_lts_new_old_old_new`
(`view_catalog_cycle_lookup.js`).
-
- Root cause: the binding loop runs twice on the same stage
View-subpipeline materialization for a bare `$lookup`/`$unionWith`/`$graphLookup` over a view
happens in one place — the binding loop inside `AggCatalogState::extendResolvedNamespaces`:
```cpp
for (auto& [_, entry] : resolvedNamespaces) {
if (auto* parsed = entry.getMutableParsedPipeline())
}
```
For each involved view, this calls `handleView → bindResolvedNamespace` on every stage. A bare
`$lookup` over a view (`_noUserPipeline == true`) responds by calling `materializeViewSubpipeline`,
which folds the view's pipeline into the stage's `_pipelines` (going from empty to size 1).
`materializeViewSubpipeline` asserts `_pipelines` is empty on entry — it assumes it runs *once per*
*stage object*.
During a single from-router aggregate execution on a shard, this loop runs in *two* resolution
passes:
-
-
- Pass 1 — memoized resolution
-
`AggCatalogState::getResolvedInvolvedNamespaces()` is memoized. Its first call runs
`resolveInvolvedNamespaces → extendResolvedNamespaces →` the binding loop, materializing the bare
`$lookup`'s view subpipeline into the *memoized* map's entry. That entry's stage now has
`_pipelines.size() == 1`.
-
-
- Pass 2 — proactive router resolution
-
`AggCatalogState::maybeProactivelyResolveInvolvedNamespaces()` runs on a from-router shard (or the
merging half of a sharded aggregation) and does two relevant things:
```cpp
auto resolvedMap = getResolvedInvolvedNamespaces(opCtx); // (A) COPY of the memoized map
...
if (mainIsView) {
...
extendResolvedNamespaces(opCtx, namespaces, resolvedMap); // (B) runs the binding loop AGAIN
}
```
- *(A)* `getResolvedInvolvedNamespaces()` returns a reference to the already-materialized memoized
map; `auto` makes `resolvedMap` a *copy*. The copy deep-clones each `ResolvedNamespace`, and
`LiteParsedPipeline`'s copy constructor clones each stage via `stage->clone()`, which copies
`_pipelines`. So the copied `$lookup` stage *already carries the materialized subpipeline*
(`_pipelines.size() == 1`), while `_noUserPipeline` is still `true`. - *(B)* `extendResolvedNamespaces` runs the same binding loop over that copy. It re-binds the
cloned `$lookup`, which already has a non-empty `_pipelines`, so `materializeViewSubpipeline` is
entered a second time and trips the `_pipelines.empty()` invariant → `tassert 12792401`.
So the "double call" is the binding loop in *pass 1 (memoized map)* and again in *pass 2 (a copy*
*that inherited pass 1's materialization)*.
-
- Why it only reproduced under multiversion + concurrency
Pass 2's re-binding is gated by two conditions:
1. *`mainIsView == true`* — the second `extendResolvedNamespaces` is inside `if (mainIsView)`, so
the shard's main namespace must itself be a view. In the FSM workload a concurrent
`remapViewToView` / `remapViewToCollection` makes a namespace the router treated as a collection
be a *view* by the time the shard (or the kickback retry) processes it.
2. *An older (last-LTS) router* — a latest-binary router pre-resolves foreign views for the shard,
so the shard skips this two-pass resolution. An older router does not, so the latest-binary shard
resolves involved namespaces itself and reaches the proactive pass.
A single-version cluster, or a static (non-concurrent) query, does not line up both conditions in
one execution, which is why the crash only surfaced in the mixed-version concurrency suite.
-
- Fix
Make the second pass not re-materialize an already-materialized stage. Two layers:
- *Structural:* `extendResolvedNamespaces` binds only the namespaces it newly added in this
call, rather than re-binding every entry in the (possibly pre-materialized) map. A pre-existing
entry can only reference namespaces that were also present before this call (view resolution is
transitive), so it never needs re-binding. - *Defensive (idempotency):* `LiteParsedDocumentSourceNestedPipelines` records
`_viewSubpipelineMaterialized` once `materializeViewSubpipeline` runs. The flag is copied
memberwise with `_pipelines` when the stage is cloned, so a re-bind of a copied stage is a no-op
instead of tripping the invariant.
Within one aggregate execution the resolved-namespace map is built from a single consistent catalog
snapshot, so skipping the redundant second materialization yields the identical view pipeline — no
view is lost, applied twice, or applied stale.