[SERVER-21837] MongoD memory usage higher than wiredTigerCacheSizeGB for primary in replica set Created: 10/Dec/15 Updated: 25/Jan/16 Resolved: 25/Jan/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Nick Judson | Assignee: | Unassigned |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Steps To Reproduce: | 3 node replica set on Windows (2 replicas and one arb). |
||||||||
| Participants: | |||||||||
| Description |
|
I've been watching mongod memory fairly closely ( Testing a replica-set with the same workload, the memory appears to push past this limit significantly: using a 4GB argument, actual memory usage is at 6.5 GB fairly quickly after kicking off the run. I haven't done a full run yet, but will attempt to do so to see if memory keeps climbing or if it levels off. |
| Comments |
| Comment by Ramon Fernandez Marina [ 25/Jan/16 ] |
|
Closing as a duplicate of |
| Comment by Nick Judson [ 16/Dec/15 ] |
|
Great - thanks for the analysis. |
| Comment by Bruce Lucas (Inactive) [ 16/Dec/15 ] |
|
Thanks Nick. Here's what we see in the stats:
Zooming in on an early part of the run:
This I believe is a sign of memory fragmentation: as memory is freed to evict data from the cache in order to make room to read new data in, the blocks that are freed aren't usable for the data being read in, so they remain on the free list and new memory is obtained. This appears to be to be an instance of Bruce |
| Comment by Nick Judson [ 16/Dec/15 ] |
|
Here is the result of half of my test run. WT configured for 4GB, mongod > 6. |
| Comment by Bruce Lucas (Inactive) [ 10/Dec/15 ] |
|
Hi Nick, Can you please tar and attach to this ticket the $dbpath/diagnostic.data directory from the instance that showed the issue? This will contain data that will help us understand the reason for the excess memory usage. Thanks, |
| Comment by Nick Judson [ 10/Dec/15 ] |
|
Hi Wan, Imagine I'm provisioning a system which contains my software and Mongo (using WT). Let's say this system has 8G RAM and I need to leave 1GB for the OS, and 4GB for my application. This means I need to cap WT to around 3G. Running mongo as a stand-alone, I can do this, but running as part of a replica set, it appears I need to account for some extra usage. Is there a way to reasonably calculate this extra usage? I'm not saying the --wiredTigerCacheSizeGB should account for everything (clearly it is WT only), but it would be nice to have a guide/docs for this scenario.
Nick. |
| Comment by Wan Bachtiar [ 10/Dec/15 ] |
|
Hi Nick, As noted in MongoDB documentation, the --wiredTigerCacheSizeGB only limits the size of the WiredTiger cache, not the total amount of memory used by mongod. The WiredTiger cache is only one component of the RAM used by MongoDB. To better understand your test:
Kind regards, |