Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39580

[4.0] Skip repairing the FCV document if the major version is too low

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.0.8
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • v4.0
    • Storage NYC 2019-03-25
    • 3

      When repair with 4.0.3+ runs against older versions of MongoDB, we first repair all databases, and then check if we can restore the FCV document by checking if all collections have UUIDs. If they do not, repair exits without completing, but cleanly, downgrading the WiredTiger log files.

      ** IMPORTANT: UPGRADE PROBLEM: The data files need to be fully upgraded to version 3.6 before attempting an upgrade to 4.0; see http://dochub.mongodb.org/core/4.0-upgrade-fcv for more details.
      Downgrading WiredTiger datafiles.
      

      This causes a few problems. The first is that it appears like an error, despite having repaired all databases.

      The other is that it does not remove the _repair_incomplete file. This causes MongoDB 4.0 to fassert on the following startup:

      An incomplete repair has been detected! This is likely because a repair operation unexpectedly failed before completing. MongoDB will not start up again without --repair.
      Fatal Assertion 50922 at src/mongo/db/storage/storage_engine_init.cpp 86
      

      As a result of the fassert, the WiredTiger log file version is not downgraded, something we only do on clean shutdown. The has the following effects:

      • log file versions are unintentionally upgraded, preventing the original binary from ever starting again
      • if the FCV is old enough (more than 1 major release), the new binary also fails to start

      The only options for recovering then become:

      • Delete the journal files, which can lead to data inconsistencies
      • Retry repair from a backup without introducing an unclean shutdown

            Assignee:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Reporter:
            louis.williams@mongodb.com Louis Williams
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: