[SERVER-46125] system_perf.yml and perf.yml cleanups Created: 13/Feb/20  Updated: 16/Sep/20  Resolved: 05/Mar/20

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Henrik Ingo (Inactive) Assignee: Ryan Timmons
Resolution: Duplicate Votes: 0
Labels: tig-evgconfig
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-46464 Unify interface between DSI and everg... Closed
Related
related to SERVER-46082 Use bin/analysis.py for performance p... Closed
Sprint: STM 2020-03-23
Participants:
Story Points: 5

 Description   

Following from SERVER-46082 etc/perf.yml and etc/system_perf.yml can now be cleaned up a bit.

  • source dsienv.sh can be removed
  • setup-dsi-env.sh can be removed
  • some of the shell.exec blocks can be merged, at least for perf.yml
  • perf.yml has diverged between master and 4.2, for example mongod.log ends up in different directory. This should be reconciled to stable branches.
  • In Microbenchmarks there are two mongod.log checks that are always green because they check files from dsi unittest-files. Pending future work on dsi libanalysis code, suggested solution is to just `rm -r dsi/bin/tests`
  • Ryan: Prefer that also perf.yml uses run-dsi.
  • I added a couple `set -o verbose` when troubleshooting bin/analysis.py. This could be removed I think.
  • I think it's possibly related to perf.yml that analysis.py / mongod.log check isn't picking up the test start and end times from perf.json. As a workaround, I disabled the election related checks in analysis.common.yml.
  • I bet boostrap.production isn't actually needed in perf.yml?
  • Addition from team discussion:
    • Consolidate setupcluster into one DSI command 
    • Read from expansions.yml instead of what sysperf currently does to write runtimesecret.yml, etc.

For system_perf.yml, consider also renaming the yaml file expansions to match what is used in dsi:

  • cluster -> infrastructure_provisioning
  • setup -> mongodb_setup
  • test -> test_control


 Comments   
Comment by Ryan Timmons [ 05/Mar/20 ]

lol nice

Comment by Henrik Ingo (Inactive) [ 05/Mar/20 ]

Great.

It's not a contest, but I want to state for the record I had this idea 3 years ago

Comment by Ryan Timmons [ 05/Mar/20 ]

This is a great ticket and I think DSI and sys-perf will be much clearer once this is implemented. Henrik, you and I had the idea at roughly the same time and I created SERVER-46464 to capture largely the same ideas. Purely because I love the ticket number (it's a 2 number palindrome!) I'm going to close this in favor of the other one, but I copeid the text there.

Generated at Thu Feb 08 05:10:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.