[SERVER-81129] Detect high memory usage in stream processor and prevent it from OOMing. Created: 17/Sep/23  Updated: 08/Dec/23  Resolved: 30/Nov/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Sandeep Dhoot Assignee: Aadesh Patel (Inactive)
Resolution: Fixed Votes: 0
Labels: big, init-337-m3
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-84005 Streams: prevent SP process from bein... Closed
Assigned Teams:
Atlas Streams
Backwards Compatibility: Fully Compatible
Sprint: Sprint 32, Sprint 33, Sprint 34, Sprint 35, Sprint 36, Sprint 37
Participants:

 Description   

$tumblingWindow, $group, $sort, $lookup can have arbitrarily high memory usage. We need to detect all such high memory usage scenarios and throw necessary exceptions to prevent the pod from OOMing.

Filing this ticket to for tracking purposes. We should break up this ticket into multiple tickets eventually. Given how important it is to get this right, we would need a design doc for this.

Our behavior here also needs to be clearly documented for our users.



 Comments   
Comment by Githook User [ 07/Nov/23 ]

Author:

{'name': 'Aadesh Patel', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81129: [streams] expose active memory usage in sorter stats in sort executor
Branch: master
https://github.com/mongodb/mongo/commit/321a983475fc488280caa292337fdac854fa9438

Generated at Thu Feb 08 06:45:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.