Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17820

Windows service stop can lead to mongod abrupt termination due to long shutdown time

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.3, 3.1.1
    • Component/s: Admin
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:

      Description

      Issue Status as of Apr 29, 2015

      ISSUE SUMMARY
      On Windows, and when MongoDB is run as a service, if MongoDB takes more than 60 seconds to stop while responding to a service stop request, Windows will forcibly terminate the process, leading to a possibly unclean termination. This change will allow MongoDB to take more than 60 seconds for a graceful stop.

      WORKAROUNDS
      Upgrade to 3.0.3.

      AFFECTED VERSIONS
      Versions of MongoDB before 3.0.3.

      FIX VERSION
      The fix is included in the 3.0.3 production release.

      Original description

      2015-03-23T20:41:02.102Z I CONTROL  [serviceShutdown] got SERVICE_CONTROL_STOP request from Windows Service Control Manager, will terminate after current cmd ends
      2015-03-23T20:41:02.106Z I STORAGE  [conn398] got request after shutdown()
      ...
      2015-03-23T20:41:59.157Z I STORAGE  [conn223] got request after shutdown()
      2015-03-23T20:42:12.065Z I CONTROL  ***** SERVER RESTARTED *****
      2015-03-23T20:42:12.071Z I CONTROL  Trying to start Windows service 'MongoDB'
      2015-03-23T20:42:12.071Z I STORAGE  Service running
      2015-03-23T20:42:12.072Z W -        [initandlisten] Detected unclean shutdown - D:\Mongo\data\db\mongod.lock is not empty.
      2015-03-23T20:42:12.072Z W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
      2015-03-23T20:42:12.072Z I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=63G,session_max=20000,eviction=(threads_max=4),statistics=(fast),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2015-03-23T20:42:12.094Z I -        [initandlisten] Fatal assertion 28548 NoSuchKey Unable to find metadata for table:survey/collection-9-5755622992935745294
      2015-03-23T20:42:12.094Z I -        [initandlisten] 
       
      ***aborting after fassert() failure
      
      

      • mongod received SERVICE_CONTROL_STOP request
      • it takes a while to shut down (maybe doing a lengthy checkpoint?)
      • Windows gets impatient after 60 seconds and abruptly terminates mongod
      • recovery fails on subsequent startup

      Recovery should succeed (SERVER-17818 tracks that issue) but it would seem desirable to avoid the abrupt termination as well. I believe there is a way to tell Windows that you got the message and are still in the process of shutting down.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: