[SERVER-73404] commit queue is running out of memory during compile_unittests Created: 27/Jan/23  Updated: 29/Oct/23  Resolved: 07/Feb/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: Daniel Moody Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

Two cases we have seen the commit queue running out memory. The unittest source files are rather memory intensive, in the 4-5 GB range. With a j36 build, its entirely possible to OOM from compiles.

 

slack thread 1
task 1

slack thread 2
task 2

 

We may need to reduce j for this build, or create a jlink like option for unittest compile jobs. Another consideration is turning off GDB_INDEX which increase the link jobs by several GBs, although in the cases mentioned it seems unittest compile jobs are the main culprit.



 Comments   
Comment by Githook User [ 07/Feb/23 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-73404 added task limiter tool to limit concurrency
Branch: master
https://github.com/mongodb/mongo/commit/ef3ad37a33f38df9b20510c1c33510b0ceee8438

Comment by Alex Neben [ 27/Jan/23 ]

If it is just a few unit test files we can also drive down the memory usage for those files?

Generated at Thu Feb 08 06:24:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.