[SERVER-43096] Pre-allocate smaller buffer for "distinct" commands Created: 30/Aug/19 Updated: 29/Oct/23 Resolved: 08/Oct/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.2.1, 4.3.1 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | James Wahlin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Backport Requested: |
v4.2, v4.0
|
||||||||
| Sprint: | Query 2019-10-07, Query 2019-10-21 | ||||||||
| Participants: | |||||||||
| Case: | (copied to CRM) | ||||||||
| Description |
|
We allocate a 16 MB buffer to hold the results of every "distinct" command, no matter how big or small the result is. This can create a performance problem: every buffer is allocated directly from and returned directly to the tcmalloc pageheap, which is moderately expensive, and is a serial bottleneck. Also, tcmalloc decommits memory (returns it to the o/s) at a rate more-or-less proportional to the rate at which memory is returned to the pageheap, so this high rate of memory returned to the pageheap can create a high rate of decommitting and the re-commiting memory, which is a very expensive operation. We could consider pre-allocating a much smaller buffer here and allowing it to grow as needed. |
| Comments |
| Comment by Githook User [ 05/Nov/19 ] |
|
Author: {'name': 'James Wahlin', 'username': 'jameswahlin', 'email': 'james.wahlin@mongodb.com'}Message: (cherry picked from commit 22256673a9db8f9cf44bdb149fe20a599ecb7e9a) |
| Comment by Githook User [ 15/Oct/19 ] |
|
Author: {'name': 'James Wahlin', 'username': 'jameswahlin', 'email': 'james.wahlin@mongodb.com'}Message: (cherry picked from commit 22256673a9db8f9cf44bdb149fe20a599ecb7e9a) |
| Comment by Githook User [ 08/Oct/19 ] |
|
Author: {'username': 'jameswahlin', 'email': 'james.wahlin@mongodb.com', 'name': 'James Wahlin'}Message: |