[SERVER-44869] Add query knob to control memory limit for $push and $addToSet Created: 27/Nov/19  Updated: 29/Oct/23  Resolved: 03/Dec/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 3.6.17, 4.2.3, 4.0.14, 4.3.3

Type: New Feature Priority: Major - P3
Reporter: Charlie Swanson Assignee: Charlie Swanson
Resolution: Fixed Votes: 0
Labels: qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-73315 Guardrail to prevent an accumulator f... Closed
is related to SERVER-44174 $push and $addToSet should restrict m... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.2, v4.0, v3.6, v3.4
Sprint: Query 2019-12-02, Query 2019-12-16
Participants:
Case:

 Description   

InĀ SERVER-44174 we added a fixed 100MB limit for these accumulators. But when thinking about backporting we got nervous that this may break some applications. We should add a knob to configure this just in case this happens, but still default the limit to 100MB.



 Comments   
Comment by Danny Hatcher (Inactive) [ 15/Jan/20 ]

This patch adds two new server parameters: internalQueryMaxPushBytes and internalQueryMaxAddToSetBytes. Both default to 100MB (100 * 1024 * 1024) and can be used to control the amount of memory we're willing to accumulate in a single array when using $push or $addToSet in aggregation.

Comment by Githook User [ 10/Dec/19 ]

Author:

{'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson', 'username': 'cswanson310'}

Message: SERVER-44869 Add query knobs for $push and $addToSet memory limits

(cherry picked from commit f7d0d2f3d6fe9e4c472bd1d8893c8b6dc96881b6)
(cherry picked from commit f69a356120ae8edd6fec5ab6efaebccc1427dd28)
(cherry picked from commit 9cb1bd53f83c84e1e4e0b19a8572d09a0d5c137f)
Branch: v3.6
https://github.com/mongodb/mongo/commit/a33058d14dbd2c9325d4875b53a5b9664e0e9845

Comment by Githook User [ 09/Dec/19 ]

Author:

{'name': 'Charlie Swanson', 'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com'}

Message: SERVER-44869 Add query knobs for $push and $addToSet memory limits

(cherry picked from commit f7d0d2f3d6fe9e4c472bd1d8893c8b6dc96881b6)
Branch: v4.2
https://github.com/mongodb/mongo/commit/81ddc335f9129868bc703831719dddc5bb1c29c2

Comment by Githook User [ 09/Dec/19 ]

Author:

{'name': 'Charlie Swanson', 'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com'}

Message: SERVER-44869 Add query knobs for $push and $addToSet memory limits

(cherry picked from commit f7d0d2f3d6fe9e4c472bd1d8893c8b6dc96881b6)
(cherry picked from commit f69a356120ae8edd6fec5ab6efaebccc1427dd28)
Branch: v4.0
https://github.com/mongodb/mongo/commit/9cb1bd53f83c84e1e4e0b19a8572d09a0d5c137f

Comment by Githook User [ 03/Dec/19 ]

Author:

{'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson', 'username': 'cswanson310'}

Message: SERVER-44869 Add query knobs for $push and $addToSet memory limits
Branch: master
https://github.com/mongodb/mongo/commit/89ed89b70cd57a504d0b337013d01d1e1a1e3ad2

Generated at Thu Feb 08 05:07:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.