[SERVER-62094] Incorrect memory usage calculation in $top/$bottom/$topN/$bottomN window functions due to Document field caching Created: 16/Dec/21  Updated: 29/Oct/23  Resolved: 18/Dec/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.2.0-rc1
Fix Version/s: 5.3.0, 5.2.0-rc3

Type: Bug Priority: Major - P3
Reporter: Mickey Winters Assignee: Mickey Winters
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-61281 Fix underflow when accounting for Doc... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.2
Steps To Reproduce:

use one of these accumulators in $setWindowFields over a collection with documents containing large strings. The accumulator spec should use the large string field as part of the sortBy expression. This will cause the memory usage to eventually be negative

Sprint: QE 2021-12-27
Participants:
Linked BF Score: 150

 Description   

The Document class has a cache to speed up accessing fields on large documents that can increase the value of getApproximateSize as fields are accessed. The outer window function code greedily calls fillCache() on input documents so that their field caches are stable and remove() calls subtracts the same amount of memory as the corresponding add() call removed. However these accumulators still run into the issue since they generate a new Document Value when they calculate sortKeys and it starts out with an empty cache. remove() searches through the internal map structure to remove the value provided, so by the time we do the memory calculation the sortKey's field cache is already fully populated, thus causing remove() calls to subtract more memory than add() added.



 Comments   
Comment by Githook User [ 18/Dec/21 ]

Author:

{'name': 'Mickey. J Winters', 'email': 'mickey.winters@mongodb.com', 'username': 'mjrb'}

Message: SERVER-62094 fix memory tracking in $top/$bottom/$topN/$bottomN window functions to prevent underflow

(cherry-picked from commit 8376b0cbfcffb3d6737eaa1b42d0879df562fe0a)
Branch: v5.2
https://github.com/mongodb/mongo/commit/ecfa7547f6a1b1fef7efe7f8233ea06203001f40

Comment by Githook User [ 18/Dec/21 ]

Author:

{'name': 'Mickey. J Winters', 'email': 'mickey.winters@mongodb.com', 'username': 'mjrb'}

Message: SERVER-62094 fix memory tracking in $top/$bottom/$topN/$bottomN window functions to prevent underflow
Branch: master
https://github.com/mongodb/mongo/commit/8376b0cbfcffb3d6737eaa1b42d0879df562fe0a

Generated at Thu Feb 08 05:54:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.