[SERVER-80309] Design a mechanism to collect performance metrics Created: 22/Aug/23  Updated: 23/Jan/24

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Steve Gross Assignee: Daniel Moody
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2023-10-17-12-26-43-396.png    
Issue Links:
Related
is related to SERVER-82263 add generated build metrics tasks whi... Open
is related to SERVER-82265 add query to track bazel build times ... Open
is related to SERVER-82266 create script to extract engflow buil... Open
is related to SERVER-82262 start running the existing scons buil... Closed
Assigned Teams:
Build
Sprint: Build and Correctness OnDeck
Participants:

 Description   

We need to collect performance metrics for Bazel, including:

  • How long do builds take?
  • How does Bazel performance compare to SCons?

(daniel.moody updated info)
SCons has built in metrics (limited) and we extended those via our own build metrics tool. The build metrics tool will track time and memory usage of any given build action, so scons will call to bazel in a single invocation to build the bazel things, and scons will track the time and memory of bazel subprocess as a whole.

One thing to note is that scons build metrics do not track remote icecream execution, because the metrics were originally designed for use in evergreen context (no icecream), and icecream does not have much in the way of metrics reporting options in any case.



 Comments   
Comment by Daniel Moody [ 17/Oct/23 ]

There needs to be some clarification and categorization of scope and goals here. For example any kind of comparison of scons and bazel has to be done in a controlled way. Like you need to build the same thing in the same environment twice (one for scons and one for bazel) to compare. This comparison does not translate to developer use case. Sure we can track average developer build time, but there are so many underlying factors there that you don't what the root cause is for any given change.

Also I believe there will be some overhead in build time while we have a "multi build system" build (similar to extra upfront cost for scons + ninja). There will be a cost for scons and bazel to communicate and work together, that will not be there if it was just scons or just bazel.

Comment by Alex Neben [ 22/Aug/23 ]

https://envoy.cluster.engflow.com/?createdAfter=2023-03-16T23%3A00%3A00.000Z&createdBefore=2023-03-17T23%3A00%3A00.000Z

Generated at Thu Feb 08 06:43:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.