Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24053

Null pointer crash in BackgroundSync::producerThread at shutdown

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.3.6
    • Component/s: Replication
    • None
    • Replication
    • ALL
    • Repl 18 (08/05/16)
    • 0

      This was hit while I was running sharding js tests locally on my dev machine. The call stack is:

      0:023> k
...
Child-SP          RetAddr           Call Site
00000092`695ff2f0 00007ff6`0cc1e2e8 mongod!std::unique_ptr<mongo::executor::NetworkInterface,std::default_delete<mongo::executor::NetworkInterface> >::get+0x28 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\memory @ 1412]
00000092`695ff320 00007ff6`0cc1df51 mongod!std::unique_ptr<mongo::executor::NetworkInterface,std::default_delete<mongo::executor::NetworkInterface> >::operator*+0x28 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\memory @ 1403]
00000092`695ff350 00007ff6`0d0c99e8 mongod!std::unique_ptr<mongo::executor::NetworkInterface,std::default_delete<mongo::executor::NetworkInterface> >::operator->+0x31 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\memory @ 1407]
00000092`695ff390 00007ff6`0c9ef19b mongod!mongo::executor::ThreadPoolTaskExecutor::startup+0x38 [e:\workspace\mongo\src\mongo\executor\thread_pool_task_executor.cpp @ 132]
00000092`695ff3f0 00007ff6`0cb4f3d4 mongod!mongo::repl::BackgroundSync::producerThread+0x7b [e:\workspace\mongo\src\mongo\db\repl\bgsync.cpp @ 192]
00000092`695ff690 00007ff6`0cb4fdfc mongod!std::_Invoker_pmf_pointer::_Call<void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64>+0x34 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1365]
00000092`695ff6c0 00007ff6`0cb4f8a0 mongod!std::invoke<void (__cdecl mongo::repl::BackgroundSync::*& __ptr64)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64>+0x4c [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1468]
00000092`695ff700 00007ff6`0cb4f596 mongod!std::_Invoke_ret<void (__cdecl mongo::repl::BackgroundSync::*& __ptr64)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64>+0x50 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1499]
00000092`695ff740 00007ff6`0cb4f2ca mongod!std::_Call_binder<std::_Unforced,0,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,std::tuple<mongo::repl::BackgroundSync * __ptr64>,std::tuple<> >+0x66 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional @ 831]
00000092`695ff780 00007ff6`0cb4f4c0 mongod!std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64>::operator()<>+0x6a [c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional @ 885]
00000092`695ff7e0 00007ff6`0cb4ff10 mongod!std::_Invoker_functor::_Call<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> >+0x30 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1400]
00000092`695ff810 00007ff6`0cb4f6ec mongod!std::invoke<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> >+0x30 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1468]
00000092`695ff840 00007ff6`0cb5227a mongod!std::_LaunchPad<std::unique_ptr<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> >,std::default_delete<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> > > > >::_Execute<0>+0x3c [c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread @ 241]
00000092`695ff870 00007ff6`0cb51e88 mongod!std::_LaunchPad<std::unique_ptr<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> >,std::default_delete<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> > > > >::_Run+0x6a [c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread @ 247]
00000092`695ff8c0 00007ff6`0be882ed mongod!std::_LaunchPad<std::unique_ptr<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> >,std::default_delete<std::tuple<std::_Binder<std::_Unforced,void (__cdecl mongo::repl::BackgroundSync::*)(void) __ptr64,mongo::repl::BackgroundSync * __ptr64 & __ptr64> > > > >::_Go+0x28 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread @ 233]
00000092`695ff8f0 00007ffc`b895a1bc mongod!std::_Pad::_Call_func+0x2d [c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread @ 210]
00000092`695ff920 00007ffc`b8959e13 ucrtbased!__crt_unique_heap_ptr<char const ,__crt_internal_free_policy>::detach+0x5c
00000092`695ff960 00007ffc`e4008102 ucrtbased!register_onexit_function+0xd3
00000092`695ff9b0 00007ffc`e41bc5b4 kernel32!BaseThreadInitThunk+0x22
00000092`695ff9e0 00000000`00000000 ntdll!RtlUserThreadStart+0x34

      It looks like shutdown is running at the same time on thread 0, which is the startup thread, which implies that this is a failed server startup:

         0  Id: ea8.3a44 Suspend: 0 Teb: 00000092`678b3000 Unfrozen
Child-SP          RetAddr           Call Site
00000092`677deee8 00007ffc`e128aadf ntdll!NtWaitForSingleObject+0x14
00000092`677deef0 00007ffc`c32e2844 KERNELBASE!WaitForSingleObjectEx+0x8f
00000092`677def90 00007ff6`0c3729d0 MSVCP140D!_Thrd_join+0x24 [f:\dd\vctools\crt\crtw32\stdcpp\thr\cthread.c @ 50]
00000092`677defd0 00007ff6`0cb4b0db mongod!std::thread::join+0xb0 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\thread @ 194]
00000092`677df040 00007ff6`0cb538a6 mongod!mongo::repl::ReplicationCoordinatorExternalStateImpl::shutdown+0x19b [e:\workspace\mongo\src\mongo\db\repl\replication_coordinator_external_state_impl.cpp @ 167]
00000092`677df140 00007ff6`0ec6f50e mongod!mongo::repl::ReplicationCoordinatorImpl::shutdown+0x296 [e:\workspace\mongo\src\mongo\db\repl\replication_coordinator_impl.cpp @ 580]
00000092`677df2b0 00007ff6`0ec720fa mongod!shutdownTask+0xae [e:\workspace\mongo\src\mongo\db\db.cpp @ 980]
00000092`677df610 00007ff6`0ec73ab0 mongod!std::_Invoker_functor::_Call<void (__cdecl*& __ptr64)(void)>+0x2a [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1400]
00000092`677df640 00007ff6`0ec72694 mongod!std::invoke<void (__cdecl*& __ptr64)(void)>+0x30 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1468]
00000092`677df670 00007ff6`0ec75a75 mongod!std::_Invoke_ret<void,void (__cdecl*& __ptr64)(void)>+0x34 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits @ 1485]
00000092`677df6a0 00007ff6`0c9eb7b4 mongod!std::_Func_impl<void (__cdecl*)(void),std::allocator<int>,void>::_Do_call+0x35 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional @ 214]
00000092`677df6e0 00007ff6`0d473cd1 mongod!std::_Func_class<void>::operator()+0x54 [c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional @ 280]
00000092`677df720 00007ff6`0d473ac3 mongod!mongo::`anonymous namespace'::runTasks+0x51 [e:\workspace\mongo\src\mongo\util\exit.cpp @ 59]
00000092`677df760 00007ff6`0c0c94f5 mongod!mongo::shutdown+0x1b3 [e:\workspace\mongo\src\mongo\util\exit.cpp @ 118]
00000092`677df870 00007ff6`0ec6e967 mongod!mongo::exitCleanly+0x25 [e:\workspace\mongo\src\mongo\util\exit.h @ 77]
00000092`677df8a0 00007ff6`0ec6ea38 mongod!mongoDbMain+0x287 [e:\workspace\mongo\src\mongo\db\db.cpp @ 1068]
00000092`677dfa40 00007ff6`0ecb5fa4 mongod!wmain+0x78 [e:\workspace\mongo\src\mongo\db\db.cpp @ 801]
00000092`677dfab0 00007ff6`0ecb5e3e mongod!invoke_main+0x34 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 80]
00000092`677dfaf0 00007ff6`0ecb5cfe mongod!__scrt_common_main_seh+0x12e [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 255]
00000092`677dfb50 00007ff6`0ecb5fc9 mongod!__scrt_common_main+0xe [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 300]
00000092`677dfb80 00007ffc`e4008102 mongod!wmainCRTStartup+0x9 [f:\dd\vctools\crt\vcstartup\src\startup\exe_wmain.cpp @ 17]
00000092`677dfbb0 00007ffc`e41bc5b4 kernel32!BaseThreadInitThunk+0x22
00000092`677dfbe0 00000000`00000000 ntdll!RtlUserThreadStart+0x34

        1. all_tests.tar.gz
          9.27 MB
        2. shard_aware_init.log.gz
          38 kB

            Assignee:
            backlog-server-repl [DO NOT USE] Backlog - Replication Team
            Reporter:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: