[SERVER-76026] Limit memory usage for bulkWrite (mongod) Created: 12/Apr/23  Updated: 29/Oct/23  Resolved: 14/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Task Priority: Major - P3
Reporter: Lingzhi Deng Assignee: Frederic Vitzikam
Resolution: Fixed Votes: 0
Labels: milestone-1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-72602 Support cursor response for bulkWrite... Closed
is related to SERVER-76027 Limit memory usage for bulkWrite (mon... Closed
Assigned Teams:
Replication
Backwards Compatibility: Fully Compatible
Sprint: Repl 2023-05-29, Repl 2023-06-12, Repl 2023-09-04, Repl 2023-09-18
Participants:

 Description   

On mongod, bulkWrite uses WorkingSet and QueuedDataStage to queue data and construct cursor response. Memory usage for WorkingSet and QueuedDataStage is unbounded. And they currently do not support spilling to disk. Additionally, we currently build the entire response in memory first without a limit.

This ticket is to limit the memory usage (e.g. capped at 100MB) for bulkWrite by putting a cap for the size of responses queued.

The end goal is to limit the memory usage when we build or cache responses for bulkWrite ops. Once this limit is hit, we should stop executing the remaining operations in this bulkWrite.



 Comments   
Comment by Githook User [ 13/Sep/23 ]

Author:

{'name': 'Frederic Vitzikam', 'email': 'frederic.vitzikam@mongodb.com', 'username': 'fredvitz'}

Message: SERVER-76026 Limit memory usage for bulkWrite (mongod)
Branch: master
https://github.com/mongodb/mongo/commit/48df3da0d995c3467631529d2bdfaf86c015575f

Generated at Thu Feb 08 06:31:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.