[SERVER-30189] Reduce calls to allocator for large $in expressions Created: 17/Jul/17  Updated: 30/Oct/23  Resolved: 09/Aug/17

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 3.2.19, 3.4.9, 3.5.12

Type: Improvement Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Tess Avitabile (Inactive)
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File screenshot-1.png    
Issue Links:
Backports
Related
related to SERVER-35693 Parsing of $in takes quadratic time d... Closed
related to SERVER-35699 Avoid holding locks during query parsing Backlog
related to SERVER-37176 IndexBoundsBuilder::unionize() can be... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4, v3.2
Sprint: Query 2017-08-21
Participants:
Case:

 Description   

When a large number of concurrent queries with large $in arrays are executed they may bottleneck on the allocator; typical stacks below.

Thread 1140 (Thread 0x2b451dae6700 (LWP 81974)):
#0  0x00000000014a6f85 in base::internal::SpinLockDelay(int volatile*, int, int) ()
#1  0x00000000014a6e57 in SpinLock::SlowLock() ()
#2  0x00000000014a9393 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int) ()
#3  0x00000000014b4d2a in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long) ()
#4  0x0000000001c11bd8 in tc_new ()
#5  0x0000000000d0097d in std::_Rb_tree<mongo::BSONElement, mongo::BSONElement, std::_Identity<mongo::BSONElement>, mongo::BSONElementCmpWithoutField, std::allocator<mongo::BSONElement> >::_M_copy(std::_Rb_tree_node<mongo::BSONElement> const*, std::_Rb_tree_node<mongo::BSONElement>*) ()
#6  0x0000000000d009e8 in std::_Rb_tree<mongo::BSONElement, mongo::BSONElement, std::_Identity<mongo::BSONElement>, mongo::BSONElementCmpWithoutField, std::allocator<mongo::BSONElement> >::_M_copy(std::_Rb_tree_node<mongo::BSONElement> const*, std::_Rb_tree_node<mongo::BSONElement>*) ()
#7  0x0000000000cff349 in mongo::ArrayFilterEntries::copyTo(mongo::ArrayFilterEntries&) const ()
#8  0x0000000000cff4e7 in mongo::InMatchExpression::copyTo(mongo::InMatchExpression*) const ()
#9  0x0000000000cff579 in mongo::InMatchExpression::shallowClone() const ()
#10 0x0000000000d13878 in mongo::AndMatchExpression::shallowClone() const ()
#11 0x0000000000ecc26b in mongo::QueryPlanner::planFromCache(mongo::CanonicalQuery const&, mongo::QueryPlannerParams const&, mongo::CachedSolution const&, mongo::QuerySolution**) ()
#12 0x0000000000e7c4e7 in mongo::(anonymous namespace)::prepareExecution(mongo::OperationContext*, mongo::Collection*, mongo::WorkingSet*, mongo::CanonicalQuery*, unsigned long, mongo::PlanStage**, mongo::QuerySolution**) ()
#13 0x0000000000e7d434 in mongo::getExecutor(mongo::OperationContext*, mongo::Collection*, std::unique_ptr<mongo::CanonicalQuery, std::default_delete<mongo::CanonicalQuery> >, mongo::PlanExecutor::YieldPolicy, unsigned long) ()
#14 0x0000000000e7dfbb in mongo::getExecutorFind(mongo::OperationContext*, mongo::Collection*, mongo::NamespaceString const&, std::unique_ptr<mongo::CanonicalQuery, std::default_delete<mongo::CanonicalQuery> >, mongo::PlanExecutor::YieldPolicy) ()
#15 0x0000000000b4af7e in mongo::FindCmd::run(mongo::OperationContext*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj&, int, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, mongo::BSONObjBuilder&) ()
 
Thread 1072 (Thread 0x2b4522d38700 (LWP 82047)):
#0  0x00000000014a6f85 in base::internal::SpinLockDelay(int volatile*, int, int) ()
#1  0x00000000014a6e57 in SpinLock::SlowLock() ()
#2  0x00000000014a8efd in tcmalloc::CentralFreeList::InsertRange(void*, void*, int) ()
#3  0x00000000014b4ed8 in tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int) ()
#4  0x00000000014b4f7d in tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned long) ()
#5  0x0000000001c12080 in tc_delete ()
#6  0x0000000000b40a1b in std::_Rb_tree<mongo::BSONElement, mongo::BSONElement, std::_Identity<mongo::BSONElement>, mongo::BSONElementCmpWithoutField, std::allocator<mongo::BSONElement> >::_M_erase(std::_Rb_tree_node<mongo::BSONElement>*) ()
#7  0x0000000000b40a0f in std::_Rb_tree<mongo::BSONElement, mongo::BSONElement, std::_Identity<mongo::BSONElement>, mongo::BSONElementCmpWithoutField, std::allocator<mongo::BSONElement> >::_M_erase(std::_Rb_tree_node<mongo::BSONElement>*) ()
#8  0x0000000000b40a0f in std::_Rb_tree<mongo::BSONElement, mongo::BSONElement, std::_Identity<mongo::BSONElement>, mongo::BSONElementCmpWithoutField, std::allocator<mongo::BSONElement> >::_M_erase(std::_Rb_tree_node<mongo::BSONElement>*) ()
#9  0x0000000000d007aa in mongo::InMatchExpression::~InMatchExpression() ()
#10 0x0000000000d1110a in mongo::ListOfMatchExpression::~ListOfMatchExpression() ()
#11 0x0000000000d135df in mongo::AndMatchExpression::~AndMatchExpression() ()
#12 0x0000000000eca1f6 in mongo::QueryPlanner::plan(mongo::CanonicalQuery const&, mongo::QueryPlannerParams const&, std::vector<mongo::QuerySolution*, std::allocator<mongo::QuerySolution*> >*) ()
#13 0x0000000000e7bf48 in mongo::(anonymous namespace)::prepareExecution(mongo::OperationContext*, mongo::Collection*, mongo::WorkingSet*, mongo::CanonicalQuery*, unsigned long, mongo::PlanStage**, mongo::QuerySolution**) ()
#14 0x0000000000e7d434 in mongo::getExecutor(mongo::OperationContext*, mongo::Collection*, std::unique_ptr<mongo::CanonicalQuery, std::default_delete<mongo::CanonicalQuery> >, mongo::PlanExecutor::YieldPolicy, unsigned long) ()
#15 0x0000000000e7dfbb in mongo::getExecutorFind(mongo::OperationContext*, mongo::Collection*, mongo::NamespaceString const&, std::unique_ptr<mongo::CanonicalQuery, std::default_delete<mongo::CanonicalQuery> >, mongo::PlanExecutor::YieldPolicy) ()
#16 0x0000000000b4af7e in mongo::FindCmd::run(mongo::OperationContext*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj&, int, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, mongo::BSONObjBuilder&) ()



 Comments   
Comment by Githook User [ 11/Jan/18 ]

Author:

{'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile', 'username': 'tessavitabile'}

Message: SERVER-30189 Reduce calls to allocator for large $in expressions

(cherry picked from commit 50a1afafc816097ed57804ff7033dffd85dbe160)
(cherry picked from commit ef408a20e6fd71ffc97f3602dd65ef5b03de6a45)
Branch: v3.2
https://github.com/mongodb/mongo/commit/d131ca7d6a2b0442620346561d27c6b00f832bf0

Comment by David Storch [ 11/Jan/18 ]

johnnyshields, we are currently evaluating the possibility of a backport to the 3.2 branch. Please continue to watch this ticket for updates.

Comment by Johnny Shields [ 03/Jan/18 ]

Hi, would it be possible to backport this patch to the 3.2 series? According to SERVER-32465 we believe it affected us in production on version 3.2.12.

Comment by Githook User [ 10/Aug/17 ]

Author:

{'username': 'tessavitabile', 'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile'}

Message: SERVER-30189 Reduce calls to allocator for large $in expressions

(cherry picked from commit 50a1afafc816097ed57804ff7033dffd85dbe160)
Branch: v3.4
https://github.com/mongodb/mongo/commit/ef408a20e6fd71ffc97f3602dd65ef5b03de6a45

Comment by Githook User [ 09/Aug/17 ]

Author:

{'username': 'tessavitabile', 'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile'}

Message: SERVER-30189 Reduce calls to allocator for large $in expressions
Branch: master
https://github.com/mongodb/mongo/commit/50a1afafc816097ed57804ff7033dffd85dbe160

Generated at Thu Feb 08 04:22:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.