[SERVER-31155] High memory & resources usage + very long boot time with many DBs Created: 19/Sep/17 Updated: 17/Oct/17 Resolved: 20/Sep/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.4.6 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | assaf-oren | Assignee: | Kelsey Schubert |
| Resolution: | Duplicate | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Ubuntu 16.04 |
||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
When using a lot of DBs (few thousands) mongod process take a lot of time to boot (minutes) and takes a lot of system resources afterwards. It open all storage files (one per every collection and per index in every DB) + consumes a lot of memory. For 2000 DBs with 10 collections and 5 indexes per collection, there will be 100000 open file handlers. On our use-case, we keep the DBs but access them rarely, usually only few DBs are relevant at any given moment. I would expect mongo to lazy load the DBs when first connection is made. Assaf |
| Comments |
| Comment by Kelsey Schubert [ 17/Oct/17 ] | |||||
|
Hi assaf-oren, Sorry for the slow reply. Yes, it will close indexes or collections that have not been accessed recently. Kind regards, | |||||
| Comment by assaf-oren [ 21/Sep/17 ] | |||||
|
Thanks, Will it release all related resources for the collection, DB and indexes? | |||||
| Comment by Kelsey Schubert [ 20/Sep/17 ] | |||||
|
Hi assaf-oren, Thanks for the report. This appears to be a duplicate of the improvement request tracked in By default, MongoDB will close inactive WiredTiger tables after 25 hours, however this can be tuned. From your description, you may see benefit using the following config, which reduces the idle time to 30 seconds:
Kind regards, |