[DOCS-9454] Docs for SERVER-25318: Limit total memory utilization for bulk index builds Created: 02/Dec/16 Updated: 11/Sep/18 Resolved: 25/Oct/17 |
|
| Status: | Closed |
| Project: | Documentation |
| Component/s: | manual |
| Affects Version/s: | 3.4.0 |
| Fix Version/s: | 3.4.0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Emily Hall | Assignee: | Steve Renaker (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Participants: | |||||||||||||||||
| Days since reply: | 6 years, 16 weeks ago | ||||||||||||||||
| Description |
|
Engineering Ticket Description: A bulk index build of multiple indexes on a collection, for example using createIndexes, or during an initial sync, will use up to 100 MB of memory per index, or up to 6400 MB of memory total (given the limit of 64 indexes per collection). This can lead to out-of-memory on smaller machines. We should limit the total memory used on smaller machines by reducing the amount of memory used per index build, and/or limiting the number of simultaneous index builds. |
| Comments |
| Comment by Githook User [ 25/Oct/17 ] |
|
Author: {'email': 'steve.renaker@mongodb.com', 'name': 'Steve Renaker', 'username': 'steveren'}Message: |
| Comment by Githook User [ 25/Oct/17 ] |
|
Author: {'email': 'steve.renaker@mongodb.com', 'name': 'Steve Renaker', 'username': 'steveren'}Message: |
| Comment by Eric Milkie [ 19/Oct/17 ] |
|
Yes please! That sounds good to me. |
| Comment by Steve Renaker (Inactive) [ 19/Oct/17 ] |
|
milkie The parameter maxIndexBuildMemoryUsageMegabytes isn't listed on the server parameters docs page, and there's no ticket to add it. I'll add it there as part of this ticket unless anyone objects. |
| Comment by Eric Milkie [ 18/Oct/17 ] |
|
We're starting to get more questions about this one, so it would be good to put this somewhere. We should definitely put this in the Limits and Thresholds page, and probably also on this page: https://docs.mongodb.com/manual/core/index-creation/ |
| Comment by Daniel Pasette (Inactive) [ 02/Feb/17 ] |
|
This was backported to 3.2.13 This is the relevant documentation information from Note that this limits the amount of memory that simultaneous foreground index builds for one collection will temporarily consume. In 3.4, initial sync clones one collection at a time, so no initial sync index build phase should consume more memory than the limit. |