[SERVER-67060] Create metrics evergreen task Created: 06/Jun/22  Updated: 29/Oct/23  Resolved: 15/Sep/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.2.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Daniel Moody Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Gantt Dependency
has to be done before SERVER-67062 tweak build metrics frequency Closed
has to be done after SERVER-67053 Create per action memory metric for w... Closed
has to be done after SERVER-67054 Create per action memory metric for m... Closed
has to be done after SERVER-67055 Create scons builtin metrics Closed
has to be done after SERVER-67056 Create artifact metrics Closed
has to be done after SERVER-67058 create libdeps metrics Closed
has to be done after SERVER-67059 build metric visualization graph Closed
Problem/Incident
Backwards Compatibility: Fully Compatible
Sprint: Dev Platform 2022-07-25
Participants:
Linked BF Score: 35

 Description   

Evergreen task steps and procedure

  1. Assuming normal required evergreen build steps like checking out the source code, etc are in place, first a script will launch to monitor system memory usage while the build is running. Primarily we are interested in peak memory usage.
  2. The next step will be a clean build with metrics gathering turned on, but with the cache disabled. The build will use the targets install-all-meta and generate-libdeps-graph. We will turn off random build order, to create consistent builds to compare against. Since this will be building everything in one build it is expected to take several hours.
  3. Next the build will be rerun in its completed state, with push to cache turned on, so that the cache will be populated. The build will pus to a local cache to see relative performance of our caching mechanism such as the ratio between a clean build, pushing everything to cache, and pulling everything from cache.
  4. Next the build will be cleaned, and a build will be completed with a hot cache to see the ratio of pulling from the cache. Note that some things which do not go into the cache will be rebuilt, so the performance impacts of those files will be accounted for here.
  5. The libdeps graph analyzer will run and extract basic metrics about the dependency graph.
  6. Finally, the artifacts of the build will be analyzed. This will include detailed data for binary artifacts (executable and compiled objects), and basic data for all other artifacts generated to the build directory.
  7. All steps will generate some json data which will be uploaded to evergreen performance analysis infrastructure.
  8. Finally a build visualization graph will be generated from the json data.


 Comments   
Comment by Githook User [ 15/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-67060 update static build metrics targets to be correct
Branch: master
https://github.com/mongodb/mongo/commit/ba5ad7837a0c11ddd620b5aad9249cd26f48bf89

Comment by Githook User [ 03/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-67060 fix build metrics lint mistake.
Branch: master
https://github.com/mongodb/mongo/commit/993d08489112ac4527764528f378bbad89af12f7

Comment by Githook User [ 03/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-67060 add build metrics evergreen tasks and cedar reporting
Branch: master
https://github.com/mongodb/mongo/commit/026f76cae138e21e6ab51cf2f91cfff2ac3b5ec5

Comment by Daniel Moody [ 25/Aug/22 ]

yeah I will check in the script and I will run it and output the top N info to the logs and a separate attached file

Comment by Alex Neben [ 25/Aug/22 ]

As part of this will we be generating the metrics you displayed on this epic? https://jira.mongodb.org/browse/PM-1855

Generated at Thu Feb 08 06:07:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.