Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23715

Failure of mmap() call should terminate mongod

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.5.1
    • Affects Version/s: 2.6.12, 3.0.11, 3.2.5
    • Component/s: MMAPv1
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Storage 2016-12-12

      The mmap() call may fail with ENOMEM, for example due to the ulimit -v process virtual memory limit. Current behavior is to continue operating but fail all operations that require the file that couldn't be mapped. This should be treated as a fatal error because the node is running in a severely degraded state and is unlikely to recover. Under 2.6 this can also cause incorrect behavior (SERVER-23714).

      repro script:

      db=/ssd/db
      
      # for ulimit -v on primary
      LIMIT=$((480*1000))
      
      killall -9 -w mongod
      rm -rf $db
      mkdir -p $db/{r0,r1}
              
      common="--smallfiles --fork --logappend"
      #common="$common --storageEngine mmapv1" # for 3.2
      (ulimit -v $LIMIT; mongod --dbpath $db/r0 --logpath $db/r0.log --port 27017 $common)
      
      mongo --eval "
      
          // insert stuff until failure due to ulimit -v
          every = 100
          var size = 1024
          s = ''
          for (var i=0; i<1024; i++)
              s += 's'
          for (var i=0; ; ) {
              var bulk = db.c.initializeUnorderedBulkOp();
              for (var j=0; j<every; j++, i++)
                  bulk.insert({x:i, s:s})
              print(i)
              bulk.execute();
          }
      "
      

      Depending on which version of mongod is used and what value is chosen for LIMIT, one of two errors may result, depending on which mmap() call failed:

      			"errmsg" : "can't map file memory",
      			"errmsg" : "file /ssd/db/r0/test.2 open/create failed in createPrivateMap (look in log for more information)",
      

            Assignee:
            daniel.gottlieb@mongodb.com Daniel Gottlieb (Inactive)
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: