[SERVER-4478] Mongod should allow a setting to periodically flush memory on windows Created: 13/Dec/11 Updated: 15/Aug/12 Resolved: 04/Mar/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Stability |
| Affects Version/s: | 2.0.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Andre' Hazelwood | Assignee: | Tad Marshall |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows Server 2008 R2 x64. |
||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | Windows | ||||||||
| Participants: | |||||||||
| Description |
|
We are currently testing mongodb for a document store, and in the process of inserting more than 3 million documents, mongodb used up so much memory on the machine that the box froze and had to be rebooted. There is currently only the _id index on the collection, and no other collections or databases currently defined. Because the size of the index is so small (way less than the memory assigned to the machine, I would expect it to continue to operate up to the point where the indexes no longer fit in memory (then slow way down). It would be nice if there was an option to tell mongodb to flush/release memory it is using for the memory mapped files and only keep the indexes in memory (even at the expense of some performance). After reboot, we are able to continue adding files to the collection since there is now more than an adequate amount of free memory for mongod to use. Thanks. mongod log file:
Collection details:
|
| Comments |
| Comment by Tad Marshall [ 04/Mar/12 ] |
|
Collapsing closely related tickets into one. |
| Comment by Tad Marshall [ 26/Jan/12 ] |
|
Thanks. The VMMap suggestion is helpful from a couple of perspectives: seeing what VMMap says gives us information about where the memory issues are, and seeing if using VMMap's Empty Working Set command helps indicates how much control we could gain by doing the same thing ourselves. |
| Comment by Andre' Hazelwood [ 26/Jan/12 ] |
|
I've been on paternity leave for the past few weeks. I should be able to look at this in the next week or so. |
| Comment by Tad Marshall [ 23/Dec/11 ] |
|
Can you try running VMMap ( http://technet.microsoft.com/en-us/sysinternals/dd535533 ) and see what it says about mongod.exe's memory usage? If you use VMMap's Empty Working Set command from the View menu, does this drop the Working Set dramatically (or at least significantly)? Can you also issue a serverStatus command when mongod.exe is in a high memory usage state and post the result? Thanks! |
| Comment by Tad Marshall [ 13/Dec/11 ] |
|
We are investigating memory usage in Windows and have open Jira tickets to improve things. See https://jira.mongodb.org/browse/SERVER-538 for one possible solution. One comment-er found that the Windows System Resource Manager was helpful ... see http://www.captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/ and http://technet.microsoft.com/en-us/library/cc732553.aspx for details. Others have reported that manually emptying the working set of the mongod.exe process relieves the problem until the working set grows overlarge again. The VMMap utility from the Sysinternals suites has an Empty Working Set command on its View menu, for example. See http://technet.microsoft.com/en-us/sysinternals/dd535533 to download VMMap. Microsoft has given us instructions for gathering more information about these issues. See https://jira.mongodb.org/browse/SERVER-4301?focusedCommentId=70050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-70050 for their suggestion. These suggestions are not the final answer, we're still working on that. |