Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-15111

partially written journal last section causes recovery to fail

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 2.4.12, 2.6.5, 2.7.7
    • Affects Version/s: None
    • Component/s: Storage
    • None
    • Fully Compatible
    • ALL

      Issue Status as of Sep 18, 2014

      ISSUE SUMMARY
      On occasion, the last journal section may be partially written in the case of a crash or a filesystem snapshot backup. This condition is expected and should be tolerated during recovery on a subsequent mongod startup, as the partially written last journal section can be safely ignored. However when this situation occurs it is not ignored, causing recovery to fail with the following log entries:

      Assertion: 15874:couldn't uncompress journal section

      USER IMPACT
      When journal recovery encounters this situation, mongod refuses to start.

      WORKAROUNDS
      Affected users can run a 2.6.5 or 2.4.12 mongod to recover from this situation. This issue affects recovery only – database files after crashes and filesystem snapshot backups made under prior versons of mongod are healthy and uncorrupted, and are trivially recoverable by 2.6.5 and 2.4.12 mongod.

      AFFECTED VERSIONS
      MongoDB production releases up to 2.6.4 and 2.4.11 are affected by this issue.

      FIX VERSION
      The fix is included in the 2.6.5 and 2.4.12 production releases.

      RESOLUTION DETAILS
      Do not treat an incomplete last section of the journal as an error.

      Original description

      The last journal section may be partially written in the case of a crash or a filesystem snapshot backup. This condition is expected and should be tolerated during recovery on subsequent mongod startup, as the partially written last journal section can be safely ignored. However when it occurs it causes recovery to fail with the following log entries:

      Thu Aug 28 19:27:53.590 [initandlisten] couldn't uncompress journal section
Thu Aug 28 19:27:53.590 [initandlisten] Assertion: 15874:couldn't uncompress journal section
0xde8c31 0xdaa3fb 0x936021 0x936674 0x9377ca 0x937c02 0x93852c 0x938792 0x923a2f 0x6d79cc 0x6d81fd 0x6def10 0x6e0cb9 0x7fa1261f176d 0x6cf789 
 mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xde8c31]
 mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0xdaa3fb]
 mongod(_ZN5mongo3dur11RecoveryJob14processSectionEPKNS0_11JSectHeaderEPKvjPKNS0_11JSectFooterE+0x561) [0x936021]
 mongod(_ZN5mongo3dur11RecoveryJob17processFileBufferEPKvj+0x134) [0x936674]
 mongod(_ZN5mongo3dur11RecoveryJob11processFileEN5boost11filesystem34pathE+0xda) [0x9377ca]
 mongod(_ZN5mongo3dur11RecoveryJob2goERSt6vectorIN5boost11filesystem34pathESaIS5_EE+0x122) [0x937c02]
 mongod(_ZN5mongo3dur8_recoverEv+0x1dc) [0x93852c]
 mongod(_ZN5mongo3dur7recoverEv+0x22) [0x938792]
 mongod(_ZN5mongo3dur7startupEv+0x7f) [0x923a2f]
 mongod(_ZN5mongo14_initAndListenEi+0x3ec) [0x6d79cc]
 mongod(_ZN5mongo13initAndListenEi+0x1d) [0x6d81fd]
 mongod() [0x6def10]
 mongod(main+0x9) [0x6e0cb9]
 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa1261f176d]
 mongod() [0x6cf789]
Thu Aug 28 19:27:53.637 [initandlisten] dbexception during recovery: 15874 couldn't uncompress journal section
Thu Aug 28 19:27:53.638 [initandlisten] exception in initAndListen: 15874 couldn't uncompress journal section, terminating

            Assignee:
            mathias@mongodb.com Mathias Stearn
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated:
              Resolved: