Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-7068

Windows access violation in NamespaceDetails::__stdAlloc() during replication

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.2.0
    • Component/s: Internal Code
    • Labels:
      None
    • Environment:
      Windows 64
      Server 2008 R2 Ent.
      7 GB Ram
      Windows Azure?
    • ALL

      We have at least one and maybe two reports of an access violation in Windows v2.2.0 following a log message of "~~ Assertion - cur out of range in _alloc()", which comes from NamespaceDetails::__stdAlloc(); line 335 in src/mongo/db/namespace_details.cpp. One report includes a stack trace.

      The value following "alloc()" is supposed to be a DiskLoc found in deletedList bucket 18. Other samples of this assertion refer to buckets 5 and 9, all with chain:1.

      Tue Sep 18 15:43:52 [repl writer worker 2]  xxx.xxx ~~ Assertion - cur out of range in _alloc() 3c9d9bc:3c9d6b4 a:63560124 b:18 chain:1
                                                 mongod.exe    ...\src\mongo\db\namespace_details.cpp(337)                      mongo::NamespaceDetails::__stdAlloc+0x344
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1466)                                mongo::DataFileMgr::insert+0x693
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1217)                                mongo::DataFileMgr::insertWithObjMod+0x48
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(458)                             mongo::_updateObjects+0x200b
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(487)                             mongo::updateObjects+0x288
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\oplog.cpp(779)                                  mongo::applyOperation_inlock+0x4e9
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\repl\rs_sync.cpp(84)                            mongo::replset::SyncTail::syncApply+0x245
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\db\repl\rs_sync.cpp(116)                           mongo::replset::multiSyncApply+0x50
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\mongo\util\concurrency\thread_pool.cpp(66)               mongo::threadpool::Worker::loop+0x94
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180)  boost::`anonymous namespace'::thread_start_function+0x21
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x17
      Tue Sep 18 15:43:52 [repl writer worker 2] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292)      _threadstartex+0x7f
      Tue Sep 18 15:43:52 [repl writer worker 2] kernel32.dll                                                                   BaseThreadInitThunk+0xd
      Tue Sep 18 15:43:52 [repl writer worker 2] *** unhandled exception (access violation) at 0x000000013F44FE39, terminating
      Tue Sep 18 15:43:52 [repl writer worker 2] *** access violation was a write to 0x000000491AC9D90C
      Tue Sep 18 15:43:52 [repl writer worker 2] *** stack trace for unhandled exception:
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\namespace_details.cpp(339)                      mongo::NamespaceDetails::__stdAlloc+0x359
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1466)                                mongo::DataFileMgr::insert+0x693
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1217)                                mongo::DataFileMgr::insertWithObjMod+0x48
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(458)                             mongo::_updateObjects+0x200b
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(487)                             mongo::updateObjects+0x288
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\oplog.cpp(779)                                  mongo::applyOperation_inlock+0x4e9
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\repl\rs_sync.cpp(84)                            mongo::replset::SyncTail::syncApply+0x245
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\repl\rs_sync.cpp(116)                           mongo::replset::multiSyncApply+0x50
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\util\concurrency\thread_pool.cpp(66)               mongo::threadpool::Worker::loop+0x94
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180)  boost::`anonymous namespace'::thread_start_function+0x21
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x17
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292)      _threadstartex+0x7f
      Tue Sep 18 15:43:53 [repl writer worker 2] kernel32.dll                                                                   BaseThreadInitThunk+0xd
      Tue Sep 18 15:43:53 [repl writer worker 2] writing minidump diagnostic file mongo.dmp
      Tue Sep 18 15:43:53 dbexit: unhandled exception
      Tue Sep 18 15:43:53 [repl writer worker 2] shutdown: going to close listening sockets...
      Tue Sep 18 15:43:53 [repl writer worker 2] closing listening socket: 524
      Tue Sep 18 15:43:53 [repl writer worker 2] shutdown: going to flush diaglog...
      Tue Sep 18 15:43:53 [repl writer worker 2] shutdown: going to close sockets...
      Tue Sep 18 15:43:53 [repl writer worker 2] shutdown: waiting for fs preallocator...
      Tue Sep 18 15:43:53 [repl writer worker 2] shutdown: lock for final commit...
      Tue Sep 18 15:43:53 [repl writer worker 2]  xxx.xxx Assertion failure threadState() == 0 src\mongo\db\d_concurrency.cpp 145
      Tue Sep 18 15:43:53 [rsBackgroundSync] Socket recv() errno:10053 An established connection was aborted by the software in your host machine. 10.28.212.72:27017
      Tue Sep 18 15:43:53 [rsBackgroundSync] SocketException: remote: 10.28.212.72:27017 error: 9001 socket exception [1] server [10.28.212.72:27017] 
      Tue Sep 18 15:43:53 [rsBackgroundSync] replSet db exception in producer: 10278 dbclient error communicating with server: 10.28.212.72:27017
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\util\stacktrace.cpp(161)       mongo::printStackTrace+0x3e
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\util\assert_util.cpp(109)      mongo::verifyFailed+0xdc
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\d_concurrency.cpp(145)      mongo::WrapperForQLock::lock_R_try+0x60
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\d_concurrency.cpp(479)      mongo::Lock::GlobalRead::GlobalRead+0x95
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\d_concurrency.cpp(796)      mongo::readlocktry::readlocktry+0x47
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\instance.cpp(977)           mongo::shutdownServer+0x1b4
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\instance.cpp(1060)          mongo::dbexit+0x264
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\db.cpp(1393)                mongo::exceptionFilter+0x1e4
      Tue Sep 18 15:43:53 [repl writer worker 2] kernel32.dll                                               UnhandledExceptionFilter+0x160
      Tue Sep 18 15:43:53 [repl writer worker 2] ntdll.dll                                                  RtlUserThreadStart+0xae
      Tue Sep 18 15:43:53 [repl writer worker 2] ntdll.dll                                                  _C_specific_handler+0x14c
      Tue Sep 18 15:43:53 [repl writer worker 2] ntdll.dll                                                  _chkstk+0x9d
      Tue Sep 18 15:43:53 [repl writer worker 2] ntdll.dll                                                  RtlDestroyMemoryBlockLookaside+0x36b
      Tue Sep 18 15:43:53 [repl writer worker 2] ntdll.dll                                                  KiUserExceptionDispatcher+0x2e
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\namespace_details.cpp(339)  mongo::NamespaceDetails::__stdAlloc+0x359
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1466)            mongo::DataFileMgr::insert+0x693
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\pdfile.cpp(1217)            mongo::DataFileMgr::insertWithObjMod+0x48
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(458)         mongo::_updateObjects+0x200b
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\ops\update.cpp(487)         mongo::updateObjects+0x288
      Tue Sep 18 15:43:53 [repl writer worker 2] mongod.exe    ...\src\mongo\db\oplog.cpp(779)              mongo::applyOperation_inlock+0x4e9
      Tue Sep 18 15:43:53 shutdown failed with exception
      Tue Sep 18 15:43:53 dbexit: really exiting now
      

        1. log.zip
          45.43 MB

            Assignee:
            tad Tad Marshall
            Reporter:
            tad Tad Marshall
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: