Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23715

Failure of mmap() call should terminate mongod

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 2.6.12, 3.0.11, 3.2.5
    • Fix Version/s: 3.5.1
    • Component/s: MMAPv1
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Sprint:
      Storage 2016-12-12

      Description

      The mmap() call may fail with ENOMEM, for example due to the ulimit -v process virtual memory limit. Current behavior is to continue operating but fail all operations that require the file that couldn't be mapped. This should be treated as a fatal error because the node is running in a severely degraded state and is unlikely to recover. Under 2.6 this can also cause incorrect behavior (SERVER-23714).

      repro script:

      db=/ssd/db
       
      # for ulimit -v on primary
      LIMIT=$((480*1000))
       
      killall -9 -w mongod
      rm -rf $db
      mkdir -p $db/{r0,r1}
              
      common="--smallfiles --fork --logappend"
      #common="$common --storageEngine mmapv1" # for 3.2
      (ulimit -v $LIMIT; mongod --dbpath $db/r0 --logpath $db/r0.log --port 27017 $common)
       
      mongo --eval "
       
          // insert stuff until failure due to ulimit -v
          every = 100
          var size = 1024
          s = ''
          for (var i=0; i<1024; i++)
              s += 's'
          for (var i=0; ; ) {
              var bulk = db.c.initializeUnorderedBulkOp();
              for (var j=0; j<every; j++, i++)
                  bulk.insert({x:i, s:s})
              print(i)
              bulk.execute();
          }
      "
      

      Depending on which version of mongod is used and what value is chosen for LIMIT, one of two errors may result, depending on which mmap() call failed:

      			"errmsg" : "can't map file memory",
      			"errmsg" : "file /ssd/db/r0/test.2 open/create failed in createPrivateMap (look in log for more information)",
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              daniel.gottlieb Daniel Gottlieb
              Reporter:
              bruce.lucas Bruce Lucas
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: