[SERVER-49786] Freeze DSI and Genny for non-master perf projects Created: 21/Jul/20 Updated: 29/Oct/23 Resolved: 12/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.2.10, 4.0.21 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Ryan Timmons | Assignee: | Ryan Timmons |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Sprint: | STM 2020-08-10, STM 2020-08-24 | ||||||||
| Participants: | |||||||||
| Story Points: | 1 | ||||||||
| Description |
|
PM-1822 will change the interface between DSI and sys-perf evergreen yamls. To minimize the risk of breaking non-master branches and the overhead of multiple backports during PM-1822, we wish to "freeze" the version of DSI in use on the following projects: To do this we will create a "legacy" branch of DSI and modify non-master evergreen yamls to use this. This means that any changes to DSI will not be usable by old perf projects unless those changes are backported. At the end of PM-1822 we could selectively bring the required projects back up to DSI master or leave these projects "forever frozen". |
| Comments |
| Comment by Githook User [ 10/Aug/20 ] |
|
Author: {'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}Message: |
| Comment by Githook User [ 10/Aug/20 ] |
|
Author: {'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}Message: |
| Comment by Githook User [ 10/Aug/20 ] |
|
Author: {'name': 'Ryan Timmons', 'email': 'ryan.timmons@10gen.com', 'username': 'rtimmons'}Message: |
| Comment by Ryan Timmons [ 27/Jul/20 ] |
|
Thanks all. I'm going to proceed with this unless any objections by COB.
Yes. I'll update the DSI PR template indicating that new workloads should be "backported" to the "legacy" branch. I'll also call it out in #performance slack.
I'm less optimistic about this. It's very hard to know what's a breaking change--half the reason we're doing this project. I think we'll consider it on a case-by-case basis depending on what the changes are, but I don't want us to guess wrong and accidentally cause everything to turn purple. This project isn't huge, and we'll kill 'legacy' at the end of it, so this is all temporary. |
| Comment by David Daly [ 27/Jul/20 ] |
|
A few quick thoughts as I'm generally fine with this plan
|
| Comment by Eric Milkie [ 25/Jul/20 ] |
|
Thanks Ryan – I like this plan. |
| Comment by Cristopher Stauffer [ 25/Jul/20 ] |
|
I don't think I have used this recently enough to meaningfully weigh in on it. |
| Comment by Ryan Timmons [ 24/Jul/20 ] |
|
Yup your summary is exactly right, milkie. Thanks for your input.
It has in the past for sure. The problem is the lack of an "API" between DSI and the server repo branches. This has meant we had to patch-test all server branches since each branch did things differently. With the API between DSI and evergreen perf yamls established (and tested) we can freely change the impl without fear of breaking pre-master branches of the server. This eliminates the need for branching DSI with every server release. I suspect we'll bring v4.0+ up to the latest API once it's established in master. If we no longer need 3.6 at that point we can kill the "legacy" branch and continue like things are now.
I was planning to add a note to the DSI PR template mentioning the distinct branches. I think that's sufficient for most DSI asks, but I could outline something more formal if it feels too non-obvious. |
| Comment by Eric Milkie [ 24/Jul/20 ] |
|
It took me a while to understand what this proposal is. Let me summarize it and please tell me if I'm correct or off base: I think this plan is fine, but I have some questions: |
| Comment by Robert Guo (Inactive) [ 24/Jul/20 ] |
|
Some historical data: for 4.2, we added the ValidateCmd Genny workload ( for 4.0, we added tpc-c ( |
| Comment by Ian Whalen (Inactive) [ 24/Jul/20 ] |
|
I don't know enough to give a helpful answer either way. |
| Comment by Ryan Timmons [ 24/Jul/20 ] |
|
Adding milkie, ian.whalen, and cristopher.stauffer - do any of you have any concerns with freezing DSI for non-master perf projects like this ticket proposes? |
| Comment by Ryan Timmons [ 24/Jul/20 ] |
|
david.daly, kelsey.schubert, and david.bradford - do any of you have any concerns with freezing DSI for non-master perf projects like this ticket proposes? |
| Comment by Brooke Miller [ 23/Jul/20 ] |
|
We discussed in triaging that we should come up with a one-pager explaining the value that could be provided by disabling DSI. |