[SERVER-49539] $bucketAuto does not account the memory used while running accumulator Created: 16/Jul/20  Updated: 29/Oct/23  Resolved: 14/Sep/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.8.0

Type: Bug Priority: Major - P3
Reporter: Arun Banala Assignee: Arun Banala
Resolution: Fixed Votes: 0
Labels: qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

coll.drop();
for (let i = 0; i < 500; ++i) {
    for (let j = 0; j < 10; ++j) {
        assert.commandWorked(coll.insert({groupBy: i}));
    }
}
 
const res = coll.aggregate([
              {
                  $bucketAuto: {
                      groupBy: "$groupBy",
                      buckets: 500,
                     // This should create an array of 10MB size for each group, resulting in 5GB of memory usage overall.
                      output: {arr: {$push: "a".repeat(1000000)}} 
                  }
              },
              {$count: "c"}
          ])
          .toArray();

Sprint: Query 2020-08-24, Query 2020-09-07, Query 2020-09-21
Participants:

 Description   

We account the memory used while running the sorter but not when running the accumulator. This could result in server consuming unlimited amount of memory while executing an aggregate command.



 Comments   
Comment by Githook User [ 14/Sep/20 ]

Author:

{'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}

Message: SERVER-49539 Populate a bucket in $bucketAuto only when required
Branch: master
https://github.com/mongodb/mongo/commit/296feba1b776611cdab7719f6a8c784de964fd85

Generated at Thu Feb 08 05:20:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.