Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17820

Windows service stop can lead to mongod abrupt termination due to long shutdown time

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 3.0.3, 3.1.1
    • Admin
    • None
    • Fully Compatible
    • ALL

    Description

      Issue Status as of Apr 29, 2015

      ISSUE SUMMARY
      On Windows, and when MongoDB is run as a service, if MongoDB takes more than 60 seconds to stop while responding to a service stop request, Windows will forcibly terminate the process, leading to a possibly unclean termination. This change will allow MongoDB to take more than 60 seconds for a graceful stop.

      WORKAROUNDS
      Upgrade to 3.0.3.

      AFFECTED VERSIONS
      Versions of MongoDB before 3.0.3.

      FIX VERSION
      The fix is included in the 3.0.3 production release.

      Original description

      2015-03-23T20:41:02.102Z I CONTROL  [serviceShutdown] got SERVICE_CONTROL_STOP request from Windows Service Control Manager, will terminate after current cmd ends
      2015-03-23T20:41:02.106Z I STORAGE  [conn398] got request after shutdown()
      ...
      2015-03-23T20:41:59.157Z I STORAGE  [conn223] got request after shutdown()
      2015-03-23T20:42:12.065Z I CONTROL  ***** SERVER RESTARTED *****
      2015-03-23T20:42:12.071Z I CONTROL  Trying to start Windows service 'MongoDB'
      2015-03-23T20:42:12.071Z I STORAGE  Service running
      2015-03-23T20:42:12.072Z W -        [initandlisten] Detected unclean shutdown - D:\Mongo\data\db\mongod.lock is not empty.
      2015-03-23T20:42:12.072Z W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
      2015-03-23T20:42:12.072Z I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=63G,session_max=20000,eviction=(threads_max=4),statistics=(fast),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2015-03-23T20:42:12.094Z I -        [initandlisten] Fatal assertion 28548 NoSuchKey Unable to find metadata for table:survey/collection-9-5755622992935745294
      2015-03-23T20:42:12.094Z I -        [initandlisten] 
       
      ***aborting after fassert() failure
      
      

      • mongod received SERVICE_CONTROL_STOP request
      • it takes a while to shut down (maybe doing a lengthy checkpoint?)
      • Windows gets impatient after 60 seconds and abruptly terminates mongod
      • recovery fails on subsequent startup

      Recovery should succeed (SERVER-17818 tracks that issue) but it would seem desirable to avoid the abrupt termination as well. I believe there is a way to tell Windows that you got the message and are still in the process of shutting down.

      Attachments

        Issue Links

          Activity

            People

              mark.benvenuto@mongodb.com Mark Benvenuto
              bruce.lucas@mongodb.com Bruce Lucas
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: