[SERVER-24625] Monitor Silent Process Exit on Windows hosts Created: 16/Jun/16  Updated: 02/Sep/16  Resolved: 16/Aug/16

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: 3.3.8
Fix Version/s: 3.3.12

Type: Improvement Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Robert Guo (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-25079 Bypass Windows CRT when exiting Closed
related to SERVER-25726 Archive *.dmp files in Evergreen Closed
is related to SERVER-25797 remove silent process monitoring code. Closed
is related to SERVER-24624 Integrate Windows Event Logs into EVG... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG 2016-08-29
Participants:
Linked BF Score: 0

 Description   

We should configure the Windows hosts to monitor silent process exits, which could catch processes that exit abnormally.

This requires some changes to the registry.
https://msdn.microsoft.com/en-us/library/windows/hardware/jj602791(v=vs.85).aspx

Additionally, we could configure the silent process exit monitor to generate minidumps, which might resolve an issue where no minidump is generated on a segfault.



 Comments   
Comment by Githook User [ 16/Aug/16 ]

Author:

{u'username': u'guoyr', u'name': u'Robert Guo', u'email': u'robertguo@me.com'}

Message: SERVER-24625 monitor silent process exits on windows
Branch: master
https://github.com/mongodb/mongo/commit/f5c94de1eeaa2da2c66749dc80558228ae6ad314

Comment by Mark Benvenuto [ 22/Jul/16 ]

Here are the correct registry keys to enable silent process exit.

REG ADD "HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Image File Execution Options\\mongod.exe"  /v GlobalFlag /t REG_DWORD /d 0x200   /f || true
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v LocalDumpFolder /t REG_SZ /d $(cygpath -w `pwd`)   /f || true
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v DumpType    /t REG_DWORD /d 0x121   /f || true
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v MaxNumberOfDumpFiles /t REG_DWORD /d 0x2710 /f || true
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v ReportingMode    /t REG_DWORD /d 0x6 /f || true

Note: The files end with ".dmp".

I have used this to successfully catch a deadlock in a mongod that was being killed by mongo shell (SERVER-25210).

Comment by Eric Milkie [ 16/Jun/16 ]

"Ignore Self Exits" would mean we would only log processes that were externally killed, and the only time we do that is when the kill_helper runs in the shell. And that's usually due to a slow or deadlocked process anyway.

Comment by Robert Guo (Inactive) [ 16/Jun/16 ]

I was thinking about initially setting "Ignore Self Exits" to "true", which should cut down on some noise.

But it doesn't seem to work all the time for me, even though it should, according to that docs page...

Comment by Eric Milkie [ 16/Jun/16 ]

Also, I take "exit on the main thread" to mean, exactly, that we return from main() or wmain() prior to calling exit() (or friends) on any other thread.

Comment by Mark Benvenuto [ 16/Jun/16 ]

If you wanted to enable it, here is an example of evergreen changes I experimented with:

REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v LocalDumpFolder /t REG_SZ /d $(cygpath -w `pwd`)   /f
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v DumpType    /t REG_DWORD /d 0x121   /f
REG ADD "HKLM\\SOFTWARE\\MICROSOFT\\Windows NT\\CurrentVersion\\SilentProcessExit\\mongod.exe"  /v ReportingMode    /t REG_DWORD /d 0x2 /f

Comment by Mark Benvenuto [ 16/Jun/16 ]

Since we do not exit on the main thread, yes, we are silently exiting.

Comment by Eric Milkie [ 16/Jun/16 ]

I'm not sure. Isn't normal server exit going to be treated as "silent", according to the criteria on that docs page?

Generated at Thu Feb 08 04:06:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.