[SERVER-58717] Should window-function $push / $addToSet restrict memory usage? Created: 20/Jul/21  Updated: 28/Aug/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: David Percy Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Query Optimization
Sprint: QO 2021-10-18, QO 2021-11-01, QO 2021-11-15, QO 2021-11-29, QO 2021-12-13, QO 2021-12-27, QO 2022-01-10, QO 2022-01-24, QO 2022-02-07, QO 2022-02-21, QO 2022-03-07, QO 2022-03-21, QO 2022-04-04, QO 2022-04-18, QO 2022-05-02, QO 2022-05-16, QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QO 2022-07-11
Participants:

 Description   

In $group, the $push and $addToSet accumulators have their own memory limits (as of SERVER-44174). Should we do something similar when they are used in $setWindowFields?

AccumulatorPush::processInternal does the check here:
https://github.com/mongodb/mongo/blob/a0198789a90fb099a252a441849ac0678d3862a7/src/mongo/db/pipeline/accumulator_push.cpp#L54

But WindowFunctionPush::add() does not: https://github.com/mongodb/mongo/blob/a0198789a90fb099a252a441849ac0678d3862a7/src/mongo/db/pipeline/window_function/window_function_push.h#L51

$group already tracks the total memory usage, to decide when to spill to disk. I think this extra limit was added because $push and $addToSet don't really benefit from spilling (if an intermediate result is too big, so will the final result).

See also SERVER-43944, which asks for a more general solution, for all operators.



 Comments   
Comment by Githook User [ 29/Sep/21 ]

Author:

{'name': 'Matthew Russotto', 'email': 'matthew.russotto@mongodb.com', 'username': 'mtrussotto'}

Message: SERVER-58717 ignore me
Branch: matthew.russotto/SERVER-57817
https://github.com/10gen/mongo-enterprise-modules/commit/c2c0ac558ad0c60a2e1daba081a9599d04b9bdf4

Generated at Thu Feb 08 05:45:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.