Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major - P3
-
Resolution: Fixed
-
Affects Version/s: 3.4.0-rc3
-
Component/s: Aggregation Framework
-
Labels:None
-
Backwards Compatibility:Fully Compatible
-
Operating System:ALL
-
Backport Completed:
-
Steps To Reproduce:
-
Sprint:Query 2016-12-12
Description
Have a collection "teststructure" with the following documents:
/* 1 */
|
{
|
"_id" : ObjectId("58077488ff3d2f357d79b86c"),
|
"parent" : null,
|
"name" : "New York",
|
"type" : "city",
|
}
|
|
|
/* 2 */
|
{
|
"_id" : ObjectId("58077488ff3d2f357d79b86d"),
|
"parent" : ObjectId("58077488ff3d2f357d79b86c"),
|
"name" : "Building One",
|
"type" : "building"
|
}
|
|
|
/* 3 */
|
{
|
"_id" : ObjectId("58077488ff3d2f357d79b86e"),
|
"parent" : ObjectId("58077488ff3d2f357d79b86d"),
|
"name" : "Room 1",
|
"type" : "room"
|
}
|
Have a collection "objects" with the following documents that a reference field "ts" to a document from "teststructure" collection:
/* 1 */
|
{
|
"_id" : ObjectId("58077488ff3d2f357d79b891"),
|
"name" : "test1",
|
"ts" : ObjectId("58077488ff3d2f357d79b86e"),
|
"resource" : {
|
"rid" : "uuid1",
|
"cl" : "desks"
|
}
|
}
|
|
|
/* 2 */
|
{
|
"_id" : ObjectId("33065488ff3d2f357d70b753"),
|
"name" : "test2",
|
"ts" : ObjectId("58077488ff3d2f357d79b86e"),
|
"resource" : {
|
"rid" : "uuid2",
|
"cl" : "desks"
|
}
|
}
|
The following query (which should retrive recursivelly the desks that are assigned in leaf nodes - rooms) crashes mongodb service:
db.teststructure.aggregate([
|
{
|
$match: {
|
"_id": ObjectId('58077488ff3d2f357d79b86c') |
}
|
},
|
{
|
$graphLookup: {
|
from: "teststructure", |
startWith: ObjectId('58077488ff3d2f357d79b86c'), |
connectFromField: "_id", |
connectToField: "parent", |
maxDepth: parseInt(100),
|
as: "path", |
depthField: "depth" |
}
|
},
|
{
|
$graphLookup: {
|
from: "objects", |
startWith: { if: { $eq: ["$path.type", 'room'] }, then: ObjectId('58077488ff3d2f357d79b86c'), else: "$path._id" }, |
connectFromField: "_id", |
connectToField: "ts", |
maxDepth: parseInt(100),
|
as: "desks", |
depthField: "depth" |
}
|
}
|
])
|
Run the above query and mongodb crashes. More details (about logs) in steps to reproduce.
Then we have to manually restart mongodb server using "sudo service mongod restart".
The problem seems to be related with the usage of the conditional operator in startWith expression from $graphLookup.
Issue Links
- is related to
-
SERVER-25535
Remove injectExpressionContext() methods from pipeline directory
-
- Closed
-
Activity
Salinca Andreea, this is to let you know that this bug report has made it into the 3.4 Bug Hunt winners.
Our Community Team will be in touch with you soon. Thanks for helping us make MongoDB 3.4 better.
Regards,
Ramón.
It looks like the $graphLookup stage just forgets to forward the new ExpressionContext to the startWith expression. This should be done as part of DocumentSourceGraphLookup::doInjectExpressionContext(). This patch fixed it locally:
diff --git a/src/mongo/db/pipeline/document_source_graph_lookup.cpp b/src/mongo/db/pipeline/document_source_graph_lookup.cpp
|
index 90475df..9bfd0a3 100644
|
--- a/src/mongo/db/pipeline/document_source_graph_lookup.cpp
|
+++ b/src/mongo/db/pipeline/document_source_graph_lookup.cpp
|
@@ -474,6 +474,7 @@ void DocumentSourceGraphLookUp::doInjectExpressionContext() {
|
|
_frontier = pExpCtx->getValueComparator().makeUnorderedValueSet();
|
_cache.setValueComparator(pExpCtx->getValueComparator());
|
+ _startWith->injectExpressionContext(pExpCtx);
|
}
|
|
void DocumentSourceGraphLookUp::doDetachFromOperationContext() { |
I was originally surprised that the fuzzer didn't catch this, but I suspect this was difficult to find because not many expression actually use the ExpressionContext, only those that might need to check what the active collation is such as $eq, $gt, etc. Further, it would need to involve such an expression which does not resolve to an ExpressionConstant. Usually, the only way to do this is to involve a field path like "$path._id" in the comparison somehow, but in this case we hit SERVER-27100 which prevented the {$eq: [1, 2]} from being optimized to just false.
@Ramon Fernandez, I am happy that I could contribute to MongoDb 3.4 and that I got into 3.4 Bug Hunt winners.
Also, I appriciate a lot the fast feedback from your team! Great job!
Thanks,
Andreea
Author:
{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: SERVER-27201 fix DocumentSourceGraphLookup to correctly propagate ExpressionContext
Branch: master
https://github.com/mongodb/mongo/commit/7411de92863b81fdf0f7c69aba8178a070633991
Author:
{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: SERVER-27201 fix DocumentSourceGraphLookup to correctly propagate ExpressionContext
(cherry picked from commit 7411de92863b81fdf0f7c69aba8178a070633991)
Branch: v3.4
https://github.com/mongodb/mongo/commit/2910650d6927788ca0a4f500a76bf232d2213ed6
It looks like the ExpressionContext used to perform the $graphLookup is nullptr. Here's the symbolized output from a debug build:
$ python buildscripts/mongosymb.py ./mongod --symbolizer-path=$(which llvm-symbolizer-3.6) < backtrace.json/data/mci/toolchain-builder/build-gcc-v2.sh-cAV/x86_64-mongodb-linux/libsanitizer/asan/../../../../gcc-5.3.0/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:2943:0: __interceptor_backtrace.part.167/home/maxh/debugging/mongo/src/mongo/util/stacktrace_posix.cpp:172:0: mongo::printStackTrace(std::ostream&)/home/maxh/debugging/mongo/src/mongo/util/signal_handlers_synchronous.cpp:178:0: mongo::(anonymous namespace)::printSignalAndBacktrace(int)/home/maxh/debugging/mongo/src/mongo/util/signal_handlers_synchronous.cpp:235:0: mongo::(anonymous namespace)::abruptQuit(int)??:0:0: ??/build/eglibc-oGUzwX/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0: gsignal/build/eglibc-oGUzwX/eglibc-2.19/stdlib/abort.c:89:0: abort/build/eglibc-oGUzwX/eglibc-2.19/assert/assert.c:92:0: __assert_fail_base/build/eglibc-oGUzwX/eglibc-2.19/assert/assert.c:101:0: __assert_fail/home/maxh/debugging/mongo/src/third_party/boost-1.60.0/boost/smart_ptr/intrusive_ptr.hpp:174:0: boost::intrusive_ptr<mongo::ExpressionContext>::operator->() const/home/maxh/debugging/mongo/src/mongo/db/pipeline/expression.cpp:738:0: mongo::ExpressionCompare::evaluateInternal(mongo::Variables*) const/home/maxh/debugging/mongo/src/mongo/db/pipeline/expression.cpp:1239:0: mongo::ExpressionObject::evaluateInternal(mongo::Variables*) const/home/maxh/debugging/mongo/src/mongo/db/pipeline/document_source_graph_lookup.cpp:360:0: mongo::DocumentSourceGraphLookUp::performSearch()/home/maxh/debugging/mongo/src/mongo/db/pipeline/document_source_graph_lookup.cpp:96:0: mongo::DocumentSourceGraphLookUp::getNext()/home/maxh/debugging/mongo/src/mongo/db/pipeline/pipeline.cpp:346:0: mongo::Pipeline::getNext()/home/maxh/debugging/mongo/src/mongo/db/exec/pipeline_proxy.cpp:110:0: mongo::PipelineProxyStage::getNextBson()/home/maxh/debugging/mongo/src/mongo/db/exec/pipeline_proxy.cpp:71:0: mongo::PipelineProxyStage::doWork(unsigned long*)/home/maxh/debugging/mongo/src/mongo/db/exec/plan_stage.cpp:46:0: mongo::PlanStage::work(unsigned long*)/home/maxh/debugging/mongo/src/mongo/db/query/plan_executor.cpp:425:0: mongo::PlanExecutor::getNextImpl(mongo::Snapshotted<mongo::BSONObj>*, mongo::RecordId*)/home/maxh/debugging/mongo/src/mongo/db/query/plan_executor.cpp:345:0: mongo::PlanExecutor::getNext(mongo::BSONObj*, mongo::RecordId*)/home/maxh/debugging/mongo/src/mongo/db/commands/pipeline_command.cpp:113:0: mongo::(anonymous namespace)::handleCursorCommand(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::ClientCursorPin*, mongo::PlanExecutor*, mongo::AggregationRequest const&, mongo::BSONObjBuilder&)/home/maxh/debugging/mongo/src/mongo/db/commands/pipeline_command.cpp:565:0: mongo::(anonymous namespace)::PipelineCommand::runParsed(mongo::OperationContext*, mongo::NamespaceString const&, mongo::AggregationRequest const&, mongo::BSONObj&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, mongo::BSONObjBuilder&)/home/maxh/debugging/mongo/src/mongo/db/commands/pipeline_command.cpp:640:0: mongo::(anonymous namespace)::PipelineCommand::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj&, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, mongo::BSONObjBuilder&)/home/maxh/debugging/mongo/src/mongo/db/commands/dbcommands.cpp:1535:0: mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*)/home/maxh/debugging/mongo/src/mongo/db/commands/dbcommands.cpp:1452:0: mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*)/home/maxh/debugging/mongo/src/mongo/db/run_commands.cpp:73:0: mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*)/home/maxh/debugging/mongo/src/mongo/db/instance.cpp:271:0: mongo::(anonymous namespace)::receivedRpc(mongo::OperationContext*, mongo::Client&, mongo::DbResponse&, mongo::Message&)/home/maxh/debugging/mongo/src/mongo/db/instance.cpp:618:0: mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&)/home/maxh/debugging/mongo/src/mongo/db/service_entry_point_mongod.cpp:135:0: mongo::ServiceEntryPointMongod::_sessionLoop(std::shared_ptr<mongo::transport::Session> const&)/home/maxh/debugging/mongo/src/mongo/db/service_entry_point_mongod.cpp:103:0: mongo::ServiceEntryPointMongod::startSession(std::shared_ptr<mongo::transport::Session>)::{lambda(std::shared_ptr<mongo::transport::Session> const&)#1}::operator()(std::shared_ptr<mongo::transport::Session> const&) const/opt/mongodbtoolchain/v2/include/c++/5.3.0/functional:1871:0: std::_Function_handler<void (std::shared_ptr<mongo::transport::Session> const&), mongo::ServiceEntryPointMongod::startSession(std::shared_ptr<mongo::transport::Session>)::{lambda(std::shared_ptr<mongo::transport::Session> const&)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptr<mongo::transport::Session> const&)/opt/mongodbtoolchain/v2/include/c++/5.3.0/functional:2271:0: std::function<void (std::shared_ptr<mongo::transport::Session> const&)>::operator()(std::shared_ptr<mongo::transport::Session> const&) const/home/maxh/debugging/mongo/src/mongo/transport/service_entry_point_utils.cpp:78:0: mongo::(anonymous namespace)::runFunc(void*)/build/eglibc-oGUzwX/eglibc-2.19/nptl/pthread_create.c:312:0: start_thread/build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111:0: clone