Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-48606

Unhandled exception in WTCheckpointThread

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 4.2.7
    • Component/s: WiredTiger
    • Labels:
      None
    • Environment:
      Windows Server 2019 Standard

      Dell Server PER540

      Processor: Intel(R) Xeon(R) Silver 4212 CPU @ 2.20GHz (2 processors)

      RAM: 128 GB

      Hard Disk: Dell PERC H740P Apd SCSI
    • ALL

      After upgrading our nodes to 4.2.7 we've been getting unhandled exceptions sporadically. The mongod.exe ends up terminating and this is affecting terribly our live environment. 

      The exception from the mongodb.log is as follows:

      2020-06-04T06:29:59.276-0500 F CONTROL [WTCheckpointThread] *** unhandled exception (access violation) at 0x00007FF649BBB29A, terminating
      2020-06-04T06:29:59.276-0500 F CONTROL [WTCheckpointThread] *** access violation was a read from 0x2667ffeee
      2020-06-04T06:29:59.276-0500 F CONTROL [WTCheckpointThread] *** stack trace for unhandled exception:
      2020-06-04T06:30:00.611-0500 I - [WTCheckpointThread] mongod.exe ...\src\third_party\gperftools-2.7\dist\src\tcmalloc.cc(1972) tc_calloc+0x9a
      mongod.exe ...\src\third_party\wiredtiger\src\os_common\os_alloc.c(50) __wt_calloc+0x5a
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_session.c(34) __block_ext_alloc+0x57
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_ext.c(233) __block_off_insert+0x2a
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_ext.c(1005) __block_merge+0x1b0
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_ext.c(894) __wt_block_extlist_merge+0x30a
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_ckpt.c(654) __ckpt_process+0x3b0
      mongod.exe ...\src\third_party\wiredtiger\src\block\block_ckpt.c(255) __wt_block_checkpoint+0xa4
      mongod.exe ...\src\third_party\wiredtiger\src\btree\bt_io.c(346) __wt_bt_write+0x3cc
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(2303) __rec_write_wrapup+0x5f1
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(209) __reconcile+0x2e2
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(102) __wt_reconcile+0x176
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(516) __rec_root_write+0x20b
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(278) __reconcile+0x5ad
      mongod.exe ...\src\third_party\wiredtiger\src\reconcile\rec_write.c(102) __wt_reconcile+0x176
      mongod.exe ...\src\third_party\wiredtiger\src\btree\bt_sync.c(311) __wt_sync_file+0x620
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(1578) __checkpoint_tree+0x169
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(1686) __checkpoint_tree_helper+0x55
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(199) __checkpoint_apply+0x56
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(848) __txn_checkpoint+0x50d
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(1043) __txn_checkpoint_wrapper+0x189
      mongod.exe ...\src\third_party\wiredtiger\src\txn\txn_ckpt.c(1101) __wt_txn_checkpoint+0x112
      mongod.exe ...\src\third_party\wiredtiger\src\session\session_api.c(1962) __session_checkpoint+0x3bc
      mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp(375) mongo::WiredTigerKVEngine::WiredTigerCheckpointThread::run+0x78a
      mongod.exe ...\src\mongo\util\background.cpp(154) mongo::BackgroundJob::jobBody+0x145
      mongod.exe c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread(230) std::_LaunchPad<std::unique_ptr<std::tuple<<lambda_e94772a372767cbd7bebbcdd24044cf3> >,std::default_delete<std::tuple<<lambda_e94772a372767cbd7bebbcdd24044cf3> > > > >::_Go+0x6d
      mongod.exe c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread(209) std::_Pad::_Call_func+0x9
      ucrtbase.dll o_exp+0x5a
      KERNEL32.DLL BaseThreadInitThunk+0x14
      2020-06-04T06:30:00.612-0500 I CONTROL [WTCheckpointThread] writing minidump diagnostic file C:\MMSAutomation\versions\mongodb-win32-x86_64-4.2.7-ent\bin\mongod.2020-06-04T11-30-00.mdmp
      2020-06-04T06:30:01.395-0500 I NETWORK [listener] connection accepted from 169.254.1.2:64568 #6240 (97 connections now open)
      {{2020-06-04T06:30:01.395-0500 I NETWORK [conn6240] received client metadata from 169.254.1.2:64568 conn6240: { driver:

      { name: "mongo-go-driver", version: "v1.1.4" }

      , os: { type: "windows", architecture: "amd64" }, platform: "go1.14.2" }}}
      2020-06-04T06:30:01.401-0500 F CONTROL [WTCheckpointThread] *** immediate exit due to unhandled exception

            Assignee:
            Unassigned Unassigned
            Reporter:
            diana@orionswave.com Diana Zhivkova
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: