[SERVER-59296] Reduce friction between evergreen task names and their resmoke invocations Created: 11/Aug/21 Updated: 02/Nov/23 Resolved: 02/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Daniel Gottlieb (Inactive) | Assignee: | [DO NOT ASSIGN] Backlog - DevProd Correctness |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | neweng, tig-resmoke, zt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Assigned Teams: |
Correctness
|
| Participants: | |
| Story Points: | 3 |
| Description |
|
When a task fails in evergreen, a common first step for a developer to triage is to run the failing test locally. Today, this is mostly seamless as task names map to suites 1-1 (with no special configuration required: looking at you --storageEngine=mmapv1 being the default on 4.0 tests) This isn't entirely defined as what's problematic, but the most annoying one for me is the core/jsCore difference The other class of issues are tasks that also specify additional arguments such as _compatibility tasks:
or the "concurrent" variety of tasks that use --numClientsPerFixture, or tasks that specify something other than wiredTiger, such as the watchdog_* tasks. Additionally, when a variant adds flags, that's an extra layer of denormalization a developer has to do. Solving the first issue could be as simple as doing a pass over what's inconsistent and renaming the task or the suite (ideally with some alias thing that wouldn't break existing users that have committed to memorizing the differences?) Not sure what the options are for the second problem. It's probably best tackled with a combination of changes that can alleviate specific pain points. |
| Comments |
| Comment by Alex Neben [ 07/Jul/22 ] |
|
Based on your comments I would like to backlog this for now. I think matching up a local invocation to what is run remotely would be better than what we have today. I think that is a bigger project though and what we have now is "good enough". If you change your minds or something new comes up please feel free to raise it again. |
| Comment by Robert Guo (Inactive) [ 01/Jul/22 ] |
|
Putting it back on SDP's radar as a valuable improvement for the local build/test experience. |