-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Optimization
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The $_internalValidateArrayness stage (inserted by the dependency graph validation passthrough suite) blocks SBE pushdown of subsequent stages, causing tests that rely on SBE plan caching/replanning to fail. Specifically, sbe_pushdown_replan_reason.js fails its $lookup test case because the validation stage inserted between $match and $lookup prevents $lookup from being pushed down to SBE.
Background
When internalEnableDependencyGraphValidation: true is set, the pipeline optimizer inserts $_internalValidateArrayness stages before each pipeline stage to validate the dependency graph's canPathBeArray() analysis at runtime. The optimized pipeline for the failing test becomes:
The SBE pushdown logic in pipelineStageIsCompatible() (sbe_pushdown.cpp) iterates stages sequentially and stops at the first unrecognized stage. Since $_internalValidateArrayness is not in the recognized stage list, it falls through to return false, breaking the pushdown chain and preventing $lookup (and any stage after the validation stage) from being pushed to SBE.
Impact
sbe_pushdown_replan_reason.js (lookup variant) must be excluded from the aggregation_dependency_graph_validation_passthrough suite
Any test relying on multi-stage SBE pushdown will fail or need exclusion when dependency graph validation is enabled
Reduces test coverage of the dependency graph validation suite
Proposed Solution
Make $_internalValidateArrayness SBE-compatible by treating it as a transparent passthrough in the pushdown logic. Since this stage only validates documents without modifying them (it's a streaming, no-op passthrough), it is safe to push it down alongside other stages. Implementation steps:
Add a validateArrayness field to CompatiblePipelineStages in sbe_pushdown.cpp
Add an else if (stageId == DocumentSourceInternalValidateArrayness::id) branch in pipelineStageIsCompatible() that unconditionally returns true (the stage has no expressions that could be SBE-incompatible)
Set the flag in findSbeCompatibleStagesForPushdown() — gated on SbeCompatibility::noRequirements since this is test-only infrastructure
Implement an SBE execution node for the stage (or have SBE skip it entirely since it's a passthrough that only needs to validate documents)
Alternatively, a simpler approach: modify the validation stage injection logic to skip insertion when the next stage would be pushed down to SBE, or have the pushdown logic skip over $_internalValidateArrayness stages transparently (treating them as invisible). This avoids needing a full SBE execution node.
- is depended on by
-
SERVER-125083 Introduce a new jstest suite for testing the dependency graph
-
- In Code Review
-