Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-9552

when replica set member has full disk, step down to (sec|rec)?

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major - P3 Major - P3
    • None
    • None
    • Diagnostics, Replication
    • 64-bit Linux, server 2.4.x, replica set
    • Replication

    Description

      When a replica has disk space issues it does not cause it to shut down, nor in the case of a primary, to step down. Instead the server periodically tests to see if enough space has been freed up to continue.

      As a result, the only indication of a problem is additional writes cause user asserts on the primary, and there's likely some introduction of repl lag on secondaries. But, from the perspective of rs.status() and db.serverStatus() everything looks fine (except for any introduced asserts/lag).

      Some options mentioned in discussion with server team:

      • Have replset member step down (if primary)
      • Have replset member enter maintenance status (until disk space is avail)
      • Add warning message to [startup]warning log

      Bonus: would be great if there was an explicit state/status change that could be picked up and reported by MMS. The last option should work for that.

      Attachments

        Activity

          People

            backlog-server-repl Backlog - Replication Team
            john.morales@mongodb.com John Morales
            Votes:
            10 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

              Created:
              Updated: