[SERVER-50277] Performance Yaml Cleanups pt 1 Created: 12/Aug/20  Updated: 29/Oct/23  Resolved: 20/Aug/20

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: Backlog
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: Ryan Timmons Assignee: Ryan Timmons
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: STM 2020-08-24
Participants:
Linked BF Score: 0
Story Points: 5

 Description   

A handful of things that will make iteration a bit easier. This is all really hard to do in separate tickets or in a staged way. Smaller PRs first wherever possible, but minimize times we backport. Best to just rip off the bandaid.

Changes to sys-perf yamls (master and 4.4):

  1. Single f_run_dsi_workload Evergreen function
    1. Mostly the same logic that's currently in the handful of existing functions, but in a single function.
    2. Update the param names to match the files e.g. cluster to infrastructure_provisioning
    3. Use conventional module locations where possible
    4. Use conventional report output locations where possible
    5. Remove cruft like dsienv.sh; run-dsi invocations are single-line scripts
  2. Change the order of tasks/functions to keep compile and dsi stuff more separated.
  3. Add a genny task that is not a generated task.

Changes to microbenchmarks yamls (master and 4.4):

  1. Single f_run_microbenchmarks_workload evergreen function for non-genny workloads
  2. Single f_run_genny_workload evergreen function for genny workloads
  3. Both functions to use conventional module locations where possible
  4. Remove extraneous genny invocation--I think just need the call to lamp without venv nonsense
  5. Tidy the weird pkill logic
  6. Conventional locations for genny, DSI, and signal-processing modules
  7. Change the order of tasks/functions to keep compile, dsi, and non-dsi-based-workloads more separated.

Changes to DSI:

  1. Update for conventional paths above
  2. Make evergreen-dsitest.yml a representative snapshot of what's in system_perf.yml and, if possible, something similar for perf.yml
  3. Change documentation for how to patch-build without compile

Changes to Genny:

  1. Kill "legacy" task-gen logic


 Comments   
Comment by Githook User [ 21/Aug/20 ]

Author:

{'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}

Message: SERVER-50277 missing 'start server' invocation on insert microbenchmark
Branch: v4.4
https://github.com/mongodb/mongo/commit/6894573909861cd11da06afad429373affe76bd0

Comment by Githook User [ 21/Aug/20 ]

Author:

{'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}

Message: SERVER-50277 missing 'start server' invocation on insert microbenchmark
Branch: master
https://github.com/mongodb/mongo/commit/2da98e01960de6d24174df19b0b926752b9dbddf

Comment by Githook User [ 21/Aug/20 ]

Author:

{'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}

Message: SERVER-50277 Performance Yaml Cleanups pt 1
Branch: v4.4
https://github.com/mongodb/mongo/commit/5f50db39c481baf780cd83c288518bdcbe299ca0

Comment by Githook User [ 20/Aug/20 ]

Author:

{'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}

Message: SERVER-50277 Performance Yaml Cleanups pt 1
Branch: master
https://github.com/mongodb/mongo/commit/267ac7c98b0c4eb08a02b7a9cf1fc38be907d27b

Comment by Githook User [ 20/Aug/20 ]

Author:

{'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}

Message: SERVER-50277 Bring 4.4 perf evergreen yamls up to latest master
Branch: v4.4
https://github.com/mongodb/mongo/commit/d24788671a62e7297fb9d6e24ecedaa5443e383f

Generated at Thu Feb 08 05:22:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.