-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: 5.0.2, 4.2.17, 4.4.10
-
Component/s: None
-
None
-
Fully Compatible
-
ALL
-
QE 2021-11-15, QE 2021-11-29, QE 2021-12-13, QE 2021-12-27, QE 2022-01-10, QE 2022-04-04, QE 2022-02-07, QE 2022-02-21, QE 2022-03-07, QE 2022-03-21, QE 2022-01-24, QE 2022-04-18, QE 2022-05-02, QO 2022-05-16, QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QE 2022-10-17, QE 2022-10-31
-
35
Repro script attached. It creates a document of this form:
doc = {
other: "xxxxxxx",
array: new Array(100000).fill("xxxxxxxxxxxxxx")
}
Then runs an aggregation that uses $project in one of two ways:
- exclude the array field using {$project: {array: 0}}, or
- include only the other field using {$project: {other: 1}}.
The exclude version is much slower and also scales very poorly as the number of threads increases, even though both are computing the same result. Following table shows time per $project operation for the two cases and for single threaded vs multi-threaded, showing both
- the poor single-threaded performance for the exclusive version, adn
- the poor scaling; good scaling would have the multi-threaded time per operation approximately the same as the single-threaded case, up to the number of CPUs.
nthreads=1 nthreads=40
exclusive: {$project: {array: 0}} 18 ms 631 ms
inclusive: {$project: {other: 1}} 1 ms 2 ms
FTDC data and PMP (stack trace) profiling taken from a customer incident show that the scaling bottleneck is the allocator, so it appears that the exclusive projection is doing a very large number of memory allocations, which could explain both the lower single-threaded performance and the poor scaling. In the multi-threaded case a typical stack for the exclusion project shows it waiting for access to the allocator central cache in this stack:
#0 0x0000561f79a845e6 in base::internal::SpinLockDelay(int volatile*, int, int) () #1 0x0000561f79a84413 in SpinLock::SlowLock() () #2 0x0000561f79a86495 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int) () #3 0x0000561f79a902c2 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned int, int, void* (*)(unsigned long)) () #4 0x0000561f798ff9bd in mongo::mongoMalloc(unsigned long) () #5 0x0000561f796db81d in mongo::RCString::create(mongo::StringData) () #6 0x0000561f796d47f9 in mongo::ValueStorage::putString(mongo::StringData) () #7 0x0000561f796d9430 in mongo::Value::Value(mongo::BSONElement const&) () #8 0x0000561f796d91a7 in mongo::Value::Value(mongo::BSONElement const&) () #9 0x0000561f796c4fbf in mongo::DocumentStorage::constructInCache(mongo::BSONElement const&) () #10 0x0000561f796c50c3 in mongo::DocumentStorageIterator::constructInCache() () #11 0x0000561f789bf160 in mongo::projection_executor::ProjectionNode::applyProjections(mongo::Document const&, mongo::MutableDocument*) const () #12 0x0000561f789c1492 in mongo::projection_executor::ProjectionNode::applyToDocument(mongo::Document const&) const () #13 0x0000561f789b9cc5 in mongo::projection_executor::ExclusionProjectionExecutor::applyProjection(mongo::Document const&) const () #14 0x0000561f7832d3b9 in mongo::projection_executor::ProjectionExecutor::applyTransformation(mongo::Document const&) () #15 0x0000561f787caa88 in mongo::DocumentSourceSingleDocumentTransformation::doGetNext() () #16 0x0000561f782db0a1 in mongo::DocumentSource::getNext() () #17 0x0000561f787d7edc in mongo::Pipeline::getNext() ()
- related to
-
SERVER-70353 Support fast path projection for exclusion projection executor
-
- Closed
-