[SERVER-32071] Powercycle - use internal crash Created: 22/Nov/17  Updated: 30/Oct/23  Resolved: 04/Dec/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.6.3, 3.7.1

Type: Task Priority: Major - P3
Reporter: Jonathan Abrahams Assignee: Jonathan Abrahams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6
Sprint: TIG 2017-12-04, TIG 2017-12-18
Participants:
Linked BF Score: 0

 Description   

Use the internal crash mechanism for the powercycle tasks.



 Comments   
Comment by Githook User [ 02/Feb/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'name': 'Jonathan Abrahams', 'username': 'hptabster'}

Message: SERVER-32071 Fix download on Windows of NotMyFault

(cherry picked from commit 46be56ad329324ed0e3b0f277d59b6e008519da1)
Branch: v3.6
https://github.com/mongodb/mongo/commit/dbe6a700ab81ca464328c354f715b6d787879334

Comment by Githook User [ 02/Feb/18 ]

Author:

{'email': 'jonathan@mongodb.com', 'name': 'Jonathan Abrahams', 'username': 'hptabster'}

Message: SERVER-32071 Powercycle - use internal crash nstead of AWS EC2 force-stop, which does not
immediately terminate an instance. Internal crash methods are:

  • Linux, use sysrq-trigger
  • Windows, use NotMyFault

(cherry picked from commit 956b5ba1730d8772f272c3af4e2e4f1b666d9506)
Branch: v3.6
https://github.com/mongodb/mongo/commit/69917c380e2c5d187543e1e3bed190be2a083dd1

Comment by Githook User [ 04/Dec/17 ]

Author:

{'username': 'hptabster', 'email': 'jonathan@mongodb.com', 'name': 'Jonathan Abrahams'}

Message: SERVER-32071 Fix download on Windows of NotMyFault
Branch: master
https://github.com/mongodb/mongo/commit/46be56ad329324ed0e3b0f277d59b6e008519da1

Comment by Jonathan Abrahams [ 04/Dec/17 ]

Fix Windows download of NotMyFault

Comment by Githook User [ 01/Dec/17 ]

Author:

{'name': 'Jonathan Abrahams', 'username': 'hptabster', 'email': 'jonathan@mongodb.com'}

Message: SERVER-32071 Powercycle - use internal crash nstead of AWS EC2 force-stop, which does not
immediately terminate an instance. Internal crash methods are:

Comment by Jonathan Abrahams [ 28/Nov/17 ]

We can use notmyfault to crash the Windows system.

Example:

./notmyfaultc.exe crash 1
 
Sysinternals NotMyfault v4.01 - Driver Bug Test Program
Copyright (C) 2002-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

Comment by Jonathan Abrahams [ 24/Nov/17 ]

External crashing is preferable, however using AWS force-stop may not be sufficient and using an internal crash should guarantee that no processes can terminate from the OS.

It has been observed in the mongod log, running on Windows, that a termination is being performed, from a force-stop:

2017-11-23T15:52:50.135+0000 I STORAGE  [serviceShutdown] shutdown: removing fs lock...
2017-11-23T15:52:50.185+0000 I CONTROL  [serviceShutdown] now exiting

Similarly in the mongod log, running on Amazon Linux:

2017-11-23T15:56:19.907+0000 I STORAGE  [signalProcessingThread] WiredTigerKVEngine shutting
 down
2017-11-23T15:56:21.393+0000 I STORAGE  [signalProcessingThread] shutdown: removing fs lock.
..
2017-11-23T15:56:21.393+0000 I CONTROL  [signalProcessingThread] now exiting
2017-11-23T15:56:21.393+0000 I CONTROL  [signalProcessingThread] shutting down with code:0

This is still under investigation why this is occurring.

Comment by Max Hirschhorn [ 23/Nov/17 ]

jonathan.abrahams, is this just to have coverage of the internal crash mechanism, or is there some other reason we don't want to force-stop Linux AWS instances?

Generated at Thu Feb 08 04:29:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.