[SERVER-67052] Create per action CPU metrics for linux Created: 06/Jun/22  Updated: 29/Oct/23  Resolved: 08/Jul/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Daniel Moody Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Gantt Dependency
has to be done before SERVER-67053 Create per action memory metric for w... Closed
has to be done before SERVER-67054 Create per action memory metric for m... Closed
has to be done before SERVER-67059 build metric visualization graph Closed
has to be done after SERVER-67051 Create per action memory and duration... Closed
Backwards Compatibility: Fully Compatible
Sprint: Dev Platform 2022-06-27, Dev Platform 2022-07-11
Participants:

 Description   

For Command line actions, we can take over the SPAWN construction variable, which will call our own spawn function and then we can tailor the command and process output before returning back to scons.

For function actions we can duck-type the FunctionAction calls to execute from our own function.

CPU

Windows makes it a bit difficult to get this info, however chromium already has a good solution:
https://chromium.googlesource.com/native_client/src/native_client/+/refs/heads/main/tools/test_lib.py

That solution is cross platform as well.

For function actions there is https://docs.python.org/3/library/time.html#time.thread_time
Duration

From python, we can measure the wall clock time of the subprocess. I plan on using https://docs.python.org/3/library/timeit.html#timeit.default_timer

Memory and duration metrics were completed in SERVER-67051

Json Format:
 
Note: for the action field, either the function name or command line will be used.
{
 build_tasks: [{
	 array_index: <int>(2),
	 outputs: [<str>](2),
	 inputs: [<str>](2),
	 action: <str>(2),
 cpu_time: <float>(2),
 mem_usage: <long>(2),
	 start_time: <datetime timestamp>(2),
	 end_time: <datetime timestamp>(2)
}]
}



 Comments   
Comment by Githook User [ 08/Jul/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-67051 SERVER-67052 SERVER-67053 SERVER-67054 add per action metrics
Branch: master
https://github.com/mongodb/mongo/commit/847e884a5360a295921edd841e1de261ffdfeea4

Generated at Thu Feb 08 06:07:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.