Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23073

Aggregation checks the output collection's sharding state without collection lock

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.3.11
    • Affects Version/s: 3.2.4, 3.3.2
    • Component/s: Aggregation Framework
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Query 12 (04/04/16), Query 17 (07/15/16), Query 18 (08/05/16)

      The aggregation framework checks the output collection's sharding state without holding collection lock. This might cause it to see inconsistent state if the collection is currently being made sharded.

      This was caught by changing the version checking code to go through the newly introduced CollectionShardingState class instead of accessing the sharded collection metadata directly.

      While this is not a pressing issue it would be nice to fix it so we can switch to storing the per-collection sharding information as decoration on the collection itself.

      This is the call stack in question:

      [MongoDFixture:job0] 2016-03-10T15:59:04.172-0500 I -        [conn2] Invariant failure txn->lockState()->isCollectionLockedForMode(ns, MODE_IS) src\mongo\db\s\collection_sharding_state.cpp 64
[MongoDFixture:job0] 2016-03-10T15:59:04.172-0500 I -        [conn2]
[MongoDFixture:job0]
[MongoDFixture:job0] ***aborting after invariant() failure
[MongoDFixture:job0]
[MongoDFixture:job0]
[MongoDFixture:job0] 2016-03-10T15:59:04.992-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\util\stacktrace_windows.cpp(174)                                   mongo::printStackTrace+0x5b
[MongoDFixture:job0] 2016-03-10T15:59:04.992-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\util\signal_handlers_synchronous.cpp(181)                          mongo::`anonymous namespace'::printSignalAndBacktrace+0x84
[MongoDFixture:job0] 2016-03-10T15:59:04.992-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\util\signal_handlers_synchronous.cpp(238)                          mongo::`anonymous namespace'::abruptQuit+0x39
[MongoDFixture:job0] 2016-03-10T15:59:04.992-0500 I CONTROL  [conn2] MSVCR120D.dll                                                                                   raise+0x35f
[MongoDFixture:job0] 2016-03-10T15:59:04.992-0500 I CONTROL  [conn2] MSVCR120D.dll                                                                                   abort+0x40
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\util\assert_util.cpp(154)                                          mongo::invariantFailed+0x13d
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\s\collection_sharding_state.cpp(64)                             mongo::CollectionShardingState::get+0xc5
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\s\collection_sharding_state.cpp(59)                             mongo::CollectionShardingState::get+0x3a
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\pipeline\pipeline_d.cpp(87)                                     mongo::`anonymous namespace'::MongodImplementation::isSharded+0x42
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\pipeline\document_source_out.cpp(63)                            mongo::DocumentSourceOut::prepTempCollection+0x184
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\pipeline\document_source_out.cpp(129)                           mongo::DocumentSourceOut::getNext+0x159
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\exec\pipeline_proxy.cpp(126)                                    mongo::PipelineProxyStage::getNextBson+0x8d
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\exec\pipeline_proxy.cpp(72)                                     mongo::PipelineProxyStage::doWork+0x178
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\exec\plan_stage.cpp(43)                                         mongo::PlanStage::work+0x6d
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\query\plan_executor.cpp(403)                                    mongo::PlanExecutor::getNextImpl+0x789
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\query\plan_executor.cpp(323)                                    mongo::PlanExecutor::getNext+0x7c
[MongoDFixture:job0] 2016-03-10T15:59:04.993-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\commands\pipeline_command.cpp(95)                               mongo::handleCursorCommand+0x23d
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\commands\pipeline_command.cpp(306)                              mongo::PipelineCommand::run+0xe8a
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\dbcommands.cpp(1493)                                            mongo::Command::run+0xc70
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\dbcommands.cpp(1356)                                            mongo::Command::execCommand+0xfd4
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\commands.cpp(498)                                               mongo::runCommands+0x743
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\instance.cpp(307)                                               mongo::`anonymous namespace'::receivedRpc+0x23c
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\instance.cpp(531)                                               mongo::assembleResponse+0x877
[MongoDFixture:job0] 2016-03-10T15:59:04.994-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\db\db.cpp(174)                                                     mongo::MyMessageHandler::process+0x158
[MongoDFixture:job0] 2016-03-10T15:59:05.006-0500 I CONTROL  [conn2] mongod.exe     ...\src\mongo\util\net\message_server_port.cpp(231)                              mongo::PortMessageServer::handleIncomingMsg+0x509
[MongoDFixture:job0] 2016-03-10T15:59:05.016-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\functional(1150)  std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64>::_Do_call<,0>+0x6e
[MongoDFixture:job0] 2016-03-10T15:59:05.036-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\functional(1138)  std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64>::operator()<>+0x56
[MongoDFixture:job0] 2016-03-10T15:59:05.048-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\functional(1150)  std::_Bind<0,void,std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64> >::_Do_call<>+0x35
[MongoDFixture:job0] 2016-03-10T15:59:05.056-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\functional(1138)  std::_Bind<0,void,std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64> >::operator()<>+0x56
[MongoDFixture:job0] 2016-03-10T15:59:05.074-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\thr\xthread(196)  std::_LaunchPad<std::_Bind<0,void,std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64> > >::_Run+0x51
[MongoDFixture:job0] 2016-03-10T15:59:05.091-0500 I CONTROL  [conn2] mongod.exe     c:\program files (x86)\microsoft visual studio 12.0\vc\include\thr\xthread(188)  std::_LaunchPad<std::_Bind<0,void,std::_Bind<1,void * __ptr64,void * __ptr64 (__cdecl*const)(void * __ptr64),mongo::`anonymous namespace'::MessagingPortWithHandler * __ptr64> > >::_Go+0x28
[MongoDFixture:job0] 2016-03-10T15:59:05.099-0500 I CONTROL  [conn2] MSVCP120D.dll                                                                                   std::_Pad::_Release+0xd9
[MongoDFixture:job0] 2016-03-10T15:59:05.107-0500 I CONTROL  [conn2] MSVCR120D.dll                                                                                   beginthreadex+0x1f5
[MongoDFixture:job0] 2016-03-10T15:59:05.113-0500 I CONTROL  [conn2] MSVCR120D.dll                                                                                   endthreadex+0x1d7
[MongoDFixture:job0] 2016-03-10T15:59:05.121-0500 I CONTROL  [conn2] KERNEL32.DLL                                                                                    BaseThreadInitThunk+0x22
[MongoDFixture:job0] 2016-03-10T15:59:05.128-0500 F -        [conn2] Got signal: 22 (SIGABRT).
[MongoDFixture:job0] 2016-03-10T15:59:05.136-0500 I CONTROL  [conn2] writing minidump diagnostic file E:\workspace\mongo\mongod.2016-03-10T20-59-05.mdmp

            Assignee:
            charlie.swanson@mongodb.com Charlie Swanson
            Reporter:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: