[SERVER-1121] Crashes reported on OpenVZ Created: 14/May/10 Updated: 10/Dec/14 Resolved: 23/Sep/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Richard Kreuter (Inactive) | Assignee: | Richard Kreuter (Inactive) |
| Resolution: | Won't Fix | Votes: | 28 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Participants: | |||||||||||||
| Description |
|
Folks report crashes running MongoDB on OpenVZ. See downthread here: http://groups.google.com/group/mongodb-user/browse_thread/thread/82a09dd33f3d377f |
| Comments |
| Comment by Ian Whalen (Inactive) [ 23/Sep/13 ] |
|
Resolving as "Won't Fix" because this is a known issue in older versions of RHEL (RHEL 5 and previous). |
| Comment by W L [ 25/May/12 ] |
|
I have a try on an openVZ shared server yesterday. CentOS release 5.8 (Final) Linux xxx 2.6.18-028stab091.2 #1 SMP Fri Jun 3 00:02:40 MSD 2011 x86_64 x86_64 x86_64 GNU/Linux. The server have 7GB memory with only 10GB storage. |
| Comment by Andrew Luzin [ 08/Feb/12 ] |
|
First, make sure that the privvmpages parameter is set to unlimited or a huge value. The memory consumed by the container must be limited by physpages. As Rich says, do repeated document inserts and monitor the amount of memory allocated by MongoDB. It is going to grow until it occupies all free memory. This is normal. When it happens, the operating system will sync data to a disk and free some memory. If the solution works, /proc/user_beancounters failcount for privvmpages should be 0. |
| Comment by Rich Thornett [ 07/Feb/12 ] |
|
As you do repeated document inserts, the MongoDB memory footprint grows. In a healthy environment, when that footprint approaches some max threshold that a system can handle (based on total memory and other processes running), memory allocation will eventually flatline as the operating reclaims memory to keep MongoDB in check. On OpenVZ, memory is never reclaimed and the footprint grows unchecked until the machine becomes unstable. (On my server, this manifests with "cannot allocate memory" errors when you attempt to access or do anything on the box.) So create a few indexes, start adding a lot of data and watch your memory grow. That should give you an answer. |
| Comment by Richard Kreuter (Inactive) [ 07/Feb/12 ] |
|
Not really exactly, but if you're seeing MongoDB running stably on OpenVZ for more than a few minutes with any non-zero write volume, that might suggest the problem has been resolved in recent versions of OpenVZ. |
| Comment by Jorge Costa [ 04/Feb/12 ] |
|
I'm evaluating mongodb 2.0.2 on Centos 6.2 with OpenVZ and NFS. |
| Comment by Andrew Luzin [ 08/Oct/11 ] |
|
I made little investigation with this bug, and discussed it with OpenVZ developers. The reason is OpenVZ memory limitation mechanism. The problem is already fixed in OpenVZ in RHEL6 Kernel and some branches from RHEL6 kernel. In OpenVZ version earlier then RHEL6 this problem can't be fixed in generally. Solutions: Details: Links: |
| Comment by kevinlau [ 26/May/11 ] |
|
After studying many dbs, we decide to use mongoDB for our deployment but we can't continue our deployment without this fix. Any news or quick fix would be appreciated. What are the pitfalls of this quick fix? |
| Comment by Juri Kuehn [ 02/Apr/11 ] |
|
Could please someone post the link to the openvz bugtracker related to this issue? There should be one, but I could not find it. |
| Comment by Eliot Horowitz (Inactive) [ 11/Mar/11 ] |
|
Durability was implemented with mmap, so has no impact on this issue. |
| Comment by Rich Thornett [ 11/Mar/11 ] |
|
Looks like contact was made with the openvz team, so I wanted to check and see if there had been any progress on this. My site is hosted at Media Temple where all of their server offerings run on openvz (Virtuozzo). So presumably I (and the rest of their customers) are unable to run MongoDB without a fix. To follow up on Richard Kreuter's comment above: "This problem may go away if mongodb stops relying on VM, probably with server-980. Moving to 1.7" The issue referenced, durability: http://jira.mongodb.org/browse/SERVER-980 has been implemented and released in v1.7.5. However, I still see the memory allocation errors with 1.8rc-1. Having said that, would running the server with durability turned on (I have not to this point) change the memory management implementation such that this problem might go away? Or did implementing durability not require a move away from VM? Thanks, folks. Great product, I hope I can use it! |
| Comment by Stuart Johnson [ 23/Feb/11 ] |
|
LXC seems to be the new OpenVZ. It's part of the linux kernel, and major distros are switching to it. http://lxc.sourceforge.net Does this problem exist on LXC? |
| Comment by Eliot Horowitz (Inactive) [ 22/Nov/10 ] |
|
Working with openvz team. |
| Comment by ZedroS [ 06/Sep/10 ] |
|
as seen there http://groups.google.com/group/mongodb-user/browse_thread/thread/4b756c95ded37297/ce08b362b65e79fc openvz allows trigger some count/iterators errors. best |
| Comment by Jesse Proudman [ 06/Jul/10 ] |
|
OpenVZ essentially throws the concept of virtual memory in the guest VM away, so your diagnosis seems accurate. Please let us know what else we can do to assist. |
| Comment by Richard Kreuter (Inactive) [ 06/Jul/10 ] |
|
By inspection, OpenVZ seems to have a hard time with how mongod leans on virtual memory. Inserting a billion documents with integer ids eventually caused OpenVZ to run out of memory for new processes (i.e., fork() started to fail) and also for existing processes (i.e., existing shells started to die from malloc() errors). So we think OpenVZ's virtual memory isn't up to snuff. This problem may go away if mongodb stops relying on VM, probably with server-980. Moving to 1.7 |
| Comment by Jesse Proudman [ 06/Jul/10 ] |
|
Hi Richard, Just sent you an email back with access to our cloud tools. Will continue to communicate via private email. Thanks
|
| Comment by Richard Kreuter (Inactive) [ 06/Jul/10 ] |
|
The thing that would be most helpful immediately would be access to an already running OpenVZ instance, especially one on which you've run into troubles with MongoDB. Please contact me at richard at 10gen dot com if you've some instances available. Thanks! |
| Comment by Jesse Proudman [ 06/Jul/10 ] |
|
We're a hosting provider who primarily uses OpenVZ. We'd love to see resolution to this issue and would be happy to donate server space for testing. Please get in touch to let me know how we can be of assistance. Thanks
|