[SERVER-68297] Document::memUsageForSorter returns an unstable value Created: 26/Jul/22  Updated: 19/Oct/22  Resolved: 19/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Dan Larkin-York Assignee: Rushan Chen
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-68196 Memory usage for BoundedSorter is inv... Closed
is related to SERVER-61281 Fix underflow when accounting for Doc... Closed
Operating System: ALL
Sprint: QE 2022-10-17, QE 2022-10-31
Participants:

 Comments   
Comment by Dan Larkin-York [ 28/Sep/22 ]

nikita.lapkov@mongodb.com For most sorters, you're right that we fill/seal/drain. For the BoundedSorter used for a partially-streaming sort for time series collections, we have a more interleaved operation order. (It's based on a heap, and we add elements until we reach a point in the input sequence where we know it's safe to extract the minimum element from the heap and emit it, then fill again as necessary, etc.)

I should have been more clear in my original post that while the memory sharing is apparent, I haven't confirmed a specific location where it's happening, so treat that as the "most plausible conjecture" I could come up with to explain what I was seeing*. I can tell you the bug was happening seemingly exclusively with uncompressed buckets, so BucketUnpackerV1::getNext seems like a good place to dig in.

*Specifically, a document that was inserted into the sorter would have a higher memUsageForSorter() value when it came out of the sorter than when it went in. My understanding is that shouldn't happen unless new fields of the doc are accessed in between, and I don't know how that would be the case if the doc is stashed in the sorter, unless its memory is somehow shared with e.g. another document being processed by another pipeline stage.

Comment by Kyle Suarez [ 20/Sep/22 ]

nikita.lapkov@mongodb.com, is this a duplicate of your work in SERVER-61281 / SERVER-69793?

Generated at Thu Feb 08 06:10:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.