-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Optimization
-
None
-
None
-
None
-
None
-
None
-
None
-
None
If a $lookup + $unwind contains a large let list with the corresponding large $match containing $expr, it takes forever to run, with the execution time apparently O(N*N*logN) over the number of joined columns.
Perf reports most of the time being spent in the Expression constructor and getDefinedVariableIDs() more specifically :
+ 97.65% 0.00% conn114 mongod_with_debug [.] mongo::exec::agg::LookUpStage::unwindResult() ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::exec::agg::LookUpStage::buildPipeline(boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::Document const&) ◆ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::pipeline_factory::makePipeline(std::vector<mongo::BSONObj, std::allocator<mongo::BSONObj> > const&, boost::intrusive_ptr<mongo::Expressio▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::pipeline_factory::(anonymous namespace)::parseAndDesugarPipeline(std::vector<mongo::BSONObj, std::allocator<mongo::BSONObj> > const&, boo▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::Pipeline::parseFromLiteParsed(mongo::LiteParsedPipeline const&, boost::intrusive_ptr<mongo::ExpressionContext> const&, std::function<void▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::buildDocumentSource[abi:cxx11](mongo::LiteParsedDocumentSource const&, boost::intrusive_ptr<mongo::ExpressionContext> const&) ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::matchStageParamsToDocumentSourceFn[abi:cxx11](std::unique_ptr<mongo::StageParams, std::default_delete<mongo::StageParams> > const&, boost▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::DocumentSourceMatch::createFromBson(mongo::BSONElement, boost::intrusive_ptr<mongo::ExpressionContext> const&) ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::DocumentSourceMatch::DocumentSourceMatch(mongo::BSONObj const&, boost::intrusive_ptr<mongo::ExpressionContext> const&) ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::DocumentSourceMatch::rebuild(mongo::BSONObj) ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::MatchExpressionParser::parse(mongo::BSONObj const&, boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::ExtensionsCallback cons▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::(anonymous namespace)::parse(mongo::BSONObj const&, boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::ExtensionsCallback cons▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::(anonymous namespace)::parseExpr(mongo::StringData, mongo::BSONElement, boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::Ext▒ + 97.65% 0.00% conn114 mongod_with_debug [.] std::__detail::_MakeUniq<mongo::ExprMatchExpression>::__single_object std::make_unique<mongo::ExprMatchExpression, mongo::BSONElement, boost::in▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::ExprMatchExpression::ExprMatchExpression(mongo::BSONElement, boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::clonable_ptr<m▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::Expression::parseObject(mongo::ExpressionContext*, mongo::BSONObj, mongo::VariablesParseState const&) ▒ + 97.65% 0.00% conn114 mongod_with_debug [.] mongo::ExpressionNaryBase<mongo::ExpressionAnd>::parse(mongo::ExpressionContext*, mongo::BSONElement, mongo::VariablesParseState const&) ▒ + 97.60% 0.00% conn114 mongod_with_debug [.] mongo::ExpressionCompare::parse(mongo::ExpressionContext*, mongo::BSONElement, mongo::VariablesParseState const&, mongo::ExpressionCompare::CmpO▒ - 97.45% 0.01% conn114 mongod_with_debug [.] mongo::Expression::Expression(mongo::ExpressionContext*, std::vector<boost::intrusive_ptr<mongo::Expression>, std::allocator<boost::intrusive_pt▒ - 97.44% mongo::Expression::Expression(mongo::ExpressionContext*, std::vector<boost::intrusive_ptr<mongo::Expression>, std::allocator<boost::intrusive_ptr<mongo::Expression> > >&&) ▒ - 77.31% mongo::VariablesParseState::getDefinedVariableIDs() const ▒ - 53.84% mongo::VariablesParseState::getDefinedVariableIDs() const ▒ std::pair<std::_Rb_tree_iterator<long>, bool> std::_Rb_tree<long, long, std::_Identity<long>, std::less<long>, std::allocator<long> >::_M_insert_unique<long const&>(long const&) ▒ + 23.47% std::pair<std::_Rb_tree_iterator<long>, bool> std::_Rb_tree<long, long, std::_Identity<long>, std::less<long>, std::allocator<long> >::_M_insert_unique<long const&>(long const&) ▒ + 20.13% std::_Rb_tree<long, long, std::_Identity<long>, std::less<long>, std::allocator<long> >::_M_erase(std::_Rb_tree_node<long>*) ▒ + 77.31% 2.39% conn114 mongod_with_debug [.] mongo::VariablesParseState::getDefinedVariableIDs() const ▒ + 74.92% 53.49% conn114 mongod_with_debug [.] std::pair<std::_Rb_tree_iterator<long>, bool> std::_Rb_tree<long, long, std::_Identity<long>, std::less<long>, std::allocator<long> >::_M_insert▒ + 65.17% 0.01% conn114 mongod_with_debug [.] mongo::ExpressionFieldPath::parse(mongo::ExpressionContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > con▒ + 65.12% 0.01% conn114 mongod_with_debug [.] mongo::ExpressionFieldPath::ExpressionFieldPath(mongo::ExpressionContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocat▒ + 20.13% 11.40% conn114 mongod_with_debug [.] std::_Rb_tree<long, long, std::_Identity<long>, std::less<long>, std::allocator<long> >::_M_erase(std::_Rb_tree_node<long>*)