The mmap() call may fail with ENOMEM, for example due to the ulimit -v process virtual memory limit. Current behavior is to continue operating but fail all operations that require the file that couldn't be mapped. This should be treated as a fatal error because the node is running in a severely degraded state and is unlikely to recover. Under 2.6 this can also cause incorrect behavior (SERVER-23714).
repro script:
db=/ssd/db # for ulimit -v on primary LIMIT=$((480*1000)) killall -9 -w mongod rm -rf $db mkdir -p $db/{r0,r1} common="--smallfiles --fork --logappend" #common="$common --storageEngine mmapv1" # for 3.2 (ulimit -v $LIMIT; mongod --dbpath $db/r0 --logpath $db/r0.log --port 27017 $common) mongo --eval " // insert stuff until failure due to ulimit -v every = 100 var size = 1024 s = '' for (var i=0; i<1024; i++) s += 's' for (var i=0; ; ) { var bulk = db.c.initializeUnorderedBulkOp(); for (var j=0; j<every; j++, i++) bulk.insert({x:i, s:s}) print(i) bulk.execute(); } "
Depending on which version of mongod is used and what value is chosen for LIMIT, one of two errors may result, depending on which mmap() call failed:
"errmsg" : "can't map file memory", "errmsg" : "file /ssd/db/r0/test.2 open/create failed in createPrivateMap (look in log for more information)",
- is related to
-
SERVER-23714 Operation failure due to mmap() failure on primary causes inconsistency on primary and crashes of secondaries
- Closed