[SERVER-70380] immediate exit due to unhandled exception Created: 09/Oct/22  Updated: 05/Dec/22  Resolved: 27/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sameer Kattel Assignee: Backlog - Triage Team
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File mongod.2022-09-27T10-11-41.mdmp     Text File primary-crash-mongo.log    
Issue Links:
Duplicate
duplicates SERVER-54786 How to avoid this error “terminate() ... Closed
Related
is related to SERVER-54786 How to avoid this error “terminate() ... Closed
Assigned Teams:
Server Triage
Operating System: ALL
Steps To Reproduce:

n/a

Participants:

 Description   

OS: windows 2019  datacenter x64
Mongo version: 4.4.0

Mongo crashed because of unhandled exception and the reason is not clear from the logs.
Also don't see much info from windbg

Microsoft (R) Windows Debugger Version 10.0.25200.1003 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
 
 
Loading Dump File [C:\Users\Sameer\Downloads\mongod.2022-09-27T10-11-41.mdmp]
User Mini Dump File: Only registers, stack and portions of memory are available
 
Symbol search path is: srv*
Executable search path is: 
Windows 10 Version 17763 MP (16 procs) Free x64
Product: Server, suite: TerminalServer DataCenter
Edition build lab: 17763.1.amd64fre.rs5_release.180914-1434
Machine Name:
Debug session time: Tue Sep 27 15:56:42.000 2022 (UTC + 5:45)
System Uptime: not available
Process Uptime: 0 days 5:36:54.000
................................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1530.10dc): Unknown exception - code e0000001 (first/second chance not available)
For analysis of this file, run !analyze -v
ntdll!NtGetContextThread+0x14:
00007ff8`132c1764 c3              ret
0:023> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************
 
*** WARNING: Unable to verify timestamp for mongod.exe
 
KEY_VALUES_STRING: 1
 
    Key  : Analysis.CPU.mSec
    Value: 390
 
    Key  : Analysis.DebugAnalysisManager
    Value: Create
 
    Key  : Analysis.Elapsed.mSec
    Value: 7497
 
    Key  : Analysis.IO.Other.Mb
    Value: 9
 
    Key  : Analysis.IO.Read.Mb
    Value: 0
 
    Key  : Analysis.IO.Write.Mb
    Value: 16
 
    Key  : Analysis.Init.CPU.mSec
    Value: 108
 
    Key  : Analysis.Init.Elapsed.mSec
    Value: 24977
 
    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 76
 
    Key  : Timeline.Process.Start.DeltaSec
    Value: 20214
 
    Key  : WER.OS.Branch
    Value: rs5_release
 
    Key  : WER.OS.Timestamp
    Value: 2018-09-14T14:34:00Z
 
    Key  : WER.OS.Version
    Value: 10.0.17763.1
 
    Key  : WER.Process.Version
    Value: 4.4.0.0
 
 
FILE_IN_CAB:  mongod.2022-09-27T10-11-41.mdmp
 
NTGLOBALFLAG:  0
 
CONTEXT:  (.ecxr)
rax=000002111dab6140 rbx=0000000000000001 rcx=000002111dab6150
rdx=000002111dab6150 rsi=00000037616fe810 rdi=00000000ffffffff
rip=00007ff80f929319 rsp=00000037616fdca0 rbp=00000037616fe020
 r8=00007ff81323803d  r9=00007ff7ee5d4160 r10=00007ff80f55b5c6
r11=0000000000000000 r12=0000000000000000 r13=00000037616fe9d0
r14=00000037616fe1d0 r15=00000037616fe200
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
KERNELBASE!RaiseException+0x69:
00007ff8`0f929319 0f1f440000      nop     dword ptr [rax+rax]
Resetting default scope
 
EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ff80f929319 (KERNELBASE!RaiseException+0x0000000000000069)
   ExceptionCode: e0000001
  ExceptionFlags: 00000001
NumberParameters: 0
 
PROCESS_NAME:  mongod.exe
 
ERROR_CODE: (NTSTATUS) 0xe0000001 - <Unable to get error code text>
 
EXCEPTION_CODE_STR:  e0000001
 
STACK_TEXT:  
00000037`616fdca0 00007ff7`edccc169     : 00000000`00000000 00000000`00000000 00000000`00000000 00007ff7`edf6cdfa : KERNELBASE!RaiseException+0x69
00000037`616fdd80 00000000`00000000     : 00000000`00000000 00000000`00000000 00007ff7`edf6cdfa 00000000`e0000001 : mongod+0x141c169
 
 
SYMBOL_NAME:  mongod+141c169
 
MODULE_NAME: mongod
 
IMAGE_NAME:  mongod.exe
 
STACK_COMMAND:  ~23s; .ecxr ; kb
 
FAILURE_BUCKET_ID:  APPLICATION_FAULT_e0000001_mongod.exe!Unknown
 
OS_VERSION:  10.0.17763.1
 
BUILDLAB_STR:  rs5_release
 
OSPLATFORM_TYPE:  x64
 
OSNAME:  Windows 10
 
IMAGE_VERSION:  4.4.0.0
 
FAILURE_ID_HASH:  {87ea1f7d-1c2e-882d-7844-2eb57a54811c}
 
Followup:     MachineOwner
---------
 
0:023> .ecxr
rax=000002111dab6140 rbx=0000000000000001 rcx=000002111dab6150
rdx=000002111dab6150 rsi=00000037616fe810 rdi=00000000ffffffff
rip=00007ff80f929319 rsp=00000037616fdca0 rbp=00000037616fe020
 r8=00007ff81323803d  r9=00007ff7ee5d4160 r10=00007ff80f55b5c6
r11=0000000000000000 r12=0000000000000000 r13=00000037616fe9d0
r14=00000037616fe1d0 r15=00000037616fe200
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
KERNELBASE!RaiseException+0x69:
00007ff8`0f929319 0f1f440000      nop     dword ptr [rax+rax]
0:023> lmvm mongod
Browse full module list
start             end                 module name
00007ff7`ec8b0000 00007ff7`eefe9000   mongod   T (no symbols)           
    Loaded symbol image file: mongod.exe
    Image path: E:\Program Files\MongoDB\Server\4.4\bin\mongod.exe
    Image name: mongod.exe
    Browse all global symbols  functions  data
    Timestamp:        Sat Jul 25 21:14:03 2020 (5F1C4FBF)
    CheckSum:         00000000
    ImageSize:        02739000
    File version:     4.4.0.0
    Product version:  4.4.0.0
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        1.0 App
    File date:        00000000.00000000
    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4
    Information from resource tables:
0:023> .exr -1
ExceptionAddress: 00007ff80f929319 (KERNELBASE!RaiseException+0x0000000000000069)
   ExceptionCode: e0000001
  ExceptionFlags: 00000001
NumberParameters: 0

 



 Comments   
Comment by Chris Kelly [ 27/Oct/22 ]

Hi samcot@gmail.com,

This looks like an error renaming files in the --dbpath directory. I would suggest investigating other tasks on the machine running that may be interfering with the mongod's permissions. Exception code e0000001 has happened in these cases before. I've linked some similar tickets to review as well.

For further questions about this particular issue, I'd recommend asking our community for help by posting on the MongoDB Developer Community Forums.

If the discussion there leads you to suspect a bug in the MongoDB server, then we'd want to investigate it as a possible bug here in the SERVER project.

 

Regards,

Christopher

 

Comment by Sameer Kattel [ 14/Oct/22 ]

Hi Chris,

Sorry, don't have any data related to this as all the resources with the mongod instances have been deleted.
monod log was huge: 5GB so only extracted the part that indicates failure and have attached here.

Regards,
Sameer

Comment by Chris Kelly [ 13/Oct/22 ]

Hi Sameer,

In order to begin investigating this, it would be helpful to get more context as to what was happening leading up to this issue. If possible the data stored in $dbpath/diagnostic.data would also be helpful here (if it still exists from the incident).

For each node in the replica set spanning a time period that includes the incident, would you please archive (tar or zip) and upload to the ticket:

  • the mongod logs
  • the $dbpath/diagnostic.data directory (the contents are described here)

Regards,
Christopher

Generated at Thu Feb 08 06:16:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.