[SERVER-25318] Limit total memory utilization for bulk index builds Created: 28/Jul/16 Updated: 10/May/18 Resolved: 07/Nov/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance, Storage |
| Affects Version/s: | None |
| Fix Version/s: | 3.2.13, 3.4.0-rc3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | Eric Milkie |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Requested: |
v3.2
|
||||||||||||||||||||
| Sprint: | Storage 2016-11-21 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Linked BF Score: | 0 | ||||||||||||||||||||
| Description |
|
A bulk index build of multiple indexes on a collection, for example using createIndexes, or during an initial sync, will use up to 100 MB of memory per index, or up to 6400 MB of memory total (given the limit of 64 indexes per collection). This can lead to out-of-memory on smaller machines. We should limit the total memory used on smaller machines by reducing the amount of memory used per index build, and/or limiting the number of simultaneous index builds. |
| Comments |
| Comment by Githook User [ 02/Feb/17 ] |
|
Author: {u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}Message: (cherry picked from commit b20282c5fec7cdedc9ba0d1aaf8439bfcd226753) Conflicts: |
| Comment by Eric Milkie [ 07/Nov/16 ] |
|
New server parameter: maxIndexBuildMemoryUsageMegabytes Note that this limits the amount of memory that simultaneous foreground index builds for one collection will temporarily consume. In 3.4, initial sync clones one collection at a time, so no initial sync index build phase should consume more memory than the limit. |
| Comment by Githook User [ 07/Nov/16 ] |
|
Author: {u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}Message: |