[SERVER-642] "mmap failed with out of memory" on 64-bit host Created: 17/Feb/10  Updated: 02/Mar/10  Resolved: 17/Feb/10

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: 1.2.2, 1.2.3, 1.3.2, 1.3.4
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Menno Smits Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Suse Linux Enterprise 11 - 64-bit

uname -a output:
Linux myhost 2.6.27.39-0.3-default #1 SMP 2009-11-23 12:57:38 +0100 x86_64 x86_64 x86_64 GNU/Linux

glibc version is 2.9

8GB of physical memory. 2GB of swap.


Attachments: File break-stuff.py    
Participants:

 Description   

I'm currently evaluating MongoDB for storing large datasets. After inserting a significant amount of data (perhaps 4GB based on the mongod process size?) inserts start failing with messages like the following:

Tue Feb 16 10:41:18 insert test_database.archive2 exception userassert:can't map file memory 0ms
mmap() failed for /home/msmits/mongo-data/test_database.7 len:2146435072 errno:12
mmap failed with out of memory, if you're using 32-bits, then you probably need to upgrade to 64
Tue Feb 16 10:41:18 test_database.archive2 Caught Assertion insert, continuing

I was under the impression that this should only happen on 32-bit systems. At the point of failure there is still several GB of physical memory available.

I am able to reproduce this problem with all the 64-bit builds on the downloads page (1.2.2, 1.2 latest, 1.3.2 and 1.3 latest).

See the attachment for a sample Python client script that reproduces the problem. On my machine it takes approx 15 mins for the failure to occur.



 Comments   
Comment by Menno Smits [ 19/Feb/10 ]

Thanks for your help. That's exactly what the problem was.

For the benefit of others, the default address space limit on SLE 11 is 8GB. The limit can be changed/removed at the command line.

When using bash:
ulimit -a # check limits
ulimit -v 0 # remove address space limit

When using zsh:
limit # check limits
limit addressspace unlimited # remove address space limit

You need to run these commands in the shell before starting mongod.

To change the limit globally at boot time see /etc/sysconfig/ulimit

Comment by Eliot Horowitz (Inactive) [ 17/Feb/10 ]

this is a suse limit problem i believe.
do a limit -a (or ulimit based on shell)
i think suse ships with a setting like
max virtual memory 4gb
you need to change that to unlimited before starting mongod

Comment by Menno Smits [ 17/Feb/10 ]

Forgot to mention. I'm happy to test patches or try other things to help debug this.

Generated at Thu Feb 08 02:54:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.