[SERVER-62094] Incorrect memory usage calculation in $top/$bottom/$topN/$bottomN window functions due to Document field caching Created: 16/Dec/21 Updated: 29/Oct/23 Resolved: 18/Dec/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 5.2.0-rc1 |
| Fix Version/s: | 5.3.0, 5.2.0-rc3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Mickey Winters | Assignee: | Mickey Winters |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Requested: |
v5.2
|
||||||||||||||||
| Steps To Reproduce: | use one of these accumulators in $setWindowFields over a collection with documents containing large strings. The accumulator spec should use the large string field as part of the sortBy expression. This will cause the memory usage to eventually be negative |
||||||||||||||||
| Sprint: | QE 2021-12-27 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 150 | ||||||||||||||||
| Description |
|
The Document class has a cache to speed up accessing fields on large documents that can increase the value of getApproximateSize as fields are accessed. The outer window function code greedily calls fillCache() on input documents so that their field caches are stable and remove() calls subtracts the same amount of memory as the corresponding add() call removed. However these accumulators still run into the issue since they generate a new Document Value when they calculate sortKeys and it starts out with an empty cache. remove() searches through the internal map structure to remove the value provided, so by the time we do the memory calculation the sortKey's field cache is already fully populated, thus causing remove() calls to subtract more memory than add() added. |
| Comments |
| Comment by Githook User [ 18/Dec/21 ] |
|
Author: {'name': 'Mickey. J Winters', 'email': 'mickey.winters@mongodb.com', 'username': 'mjrb'}Message: (cherry-picked from commit 8376b0cbfcffb3d6737eaa1b42d0879df562fe0a) |
| Comment by Githook User [ 18/Dec/21 ] |
|
Author: {'name': 'Mickey. J Winters', 'email': 'mickey.winters@mongodb.com', 'username': 'mjrb'}Message: |