[SERVER-67046] Create system memory monitor Created: 06/Jun/22  Updated: 29/Oct/23  Resolved: 07/Jul/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Daniel Moody Assignee: Richard Samuels (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Gantt Dependency
has to be done before SERVER-67059 build metric visualization graph Closed
has to be done after SERVER-67044 Create build metrics CLI interface an... Closed
has to be done after SERVER-67109 create build metrics json validator Closed
Problem/Incident
Backwards Compatibility: Fully Compatible
Sprint: Dev Platform 2022-06-27, Dev Platform 2022-07-11
Participants:
Linked BF Score: 168

 Description   

System memory will be monitored by constantly checking used memory via psutil. The interval for checking will be 0.1 seconds. This should get a good approximation of peak system memory usage without too much overhead. All the memory values over time will be stored, and can be used to diagnose system state concurrent with actions taking place at given time intervals or generating memory over time visuals. From this data we can also calculate the peak system memory, min system memory, and average memory. Because each system may be different in its ambient memory use, we use the difference of the peak memory vs the starting memory or min memory to find a baseline value from which to compare other build variants with.

 

 

Json format:
{
 system_memory: 
	{
	 mem_over_time:{[
		timestamp: <datetime timestamp>(3),
           memory: <long>(3)
	 ]}
 min: <long>(2),
	 max: <long>(2),
	 arithmetic_mean: <long>(2),
	 start_mem: <long>(2)
}
}
 

 



 Comments   
Comment by Githook User [ 07/Jul/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-67046 Fix windows typing import
Branch: master
https://github.com/mongodb/mongo/commit/59518e4f9c55ff4d897bc1ac1e7412b9243392a4

Comment by Githook User [ 07/Jul/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-67046 Create system memory monitor
Branch: master
https://github.com/mongodb/mongo/commit/9c471282110ba73ba40d3782fd858135e16b5cac

Comment by Daniel Moody [ 08/Jun/22 ]

richard.samuels@mongodb.com I originally designed this as a separate script which was launched before the scons build command, mostly because then we can see all memory data points from the start of scons. However This means an extra manual step which must be performed. Another idea is if we hook the start of the script early enough in the process, can it be integrated into the scons build command? I am thinking early in the buildscripts/scons.py script which can be done before any scons modules are loaded, so we would miss the start of the main python process, but that may be negligible in terms of memory.

Generated at Thu Feb 08 06:07:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.