Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-9235

Better messaging/logging when disks are inaccessible

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.2.3
    • Component/s: None
    • Labels:
      None
    • Environment:
      Ubuntu/AWS/EBS

      We ran into a bad problem with EBS volumes becoming intermittently inaccessible. The following stacktrace occurs and MongoDB shuts down. Better messaging in the logs would help users diagnose this problem faster.

      I believe this can be recreated by unmounting a drive while MongoDB is running:

      dbexception in groupCommit causing immediate shutdown: 0 assertion src/mongo/util/alignedbuilder.cpp:91
      
      Assertion failure a <= 512*1024*1024 src/mongo/util/alignedbuilder.cpp 91
      0xb07561 0xacdb5d 0xacc36d 0x73dda5 0x73e0b4 0x731024 0x73157a 0x72ef8c 0x72f09c 0x9ac179 0x9ab9ef 0xadab5d 0xb4d3d9 0x7f99bbfa6d8c 0x7f99bb347fdd
      /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
      /usr/bin/mongod(_ZN5mongo12verifyFailedEPKcS1_j+0xfd) [0xacdb5d]
      /usr/bin/mongod(_ZN5mongo14AlignedBuilder14growReallocateEj+0x7d) [0xacc36d]
      /usr/bin/mongod() [0x73dda5]
      /usr/bin/mongod(_ZN5mongo3dur13PREPLOGBUFFERERNS0_11JSectHeaderERNS_14AlignedBuilderE+0x214) [0x73e0b4]
      /usr/bin/mongod() [0x731024]
      /usr/bin/mongod(_ZN5mongo3dur11DurableImpl9commitNowEv+0x1a) [0x73157a]
      /usr/bin/mongod(_ZN5mongo3dur11DurableImpl16_aCommitIsNeededEv+0x14c) [0x72ef8c]
      /usr/bin/mongod(_ZN5mongo3dur11DurableImpl14commitIfNeededEb+0x4c) [0x72f09c]
      /usr/bin/mongod(_ZN5mongo7replset8SyncTail9syncApplyERKNS_7BSONObjEb+0x2a9) [0x9ac179]
      /usr/bin/mongod(_ZN5mongo7replset14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x4f) [0x9ab9ef]
      /usr/bin/mongod(_ZN5mongo10threadpool6Worker4loopEv+0x26d) [0xadab5d]
      /usr/bin/mongod() [0xb4d3d9]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c)
      

            Assignee:
            Unassigned Unassigned
            Reporter:
            jason.zucchetto Jason Zucchetto
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: