[SERVER-54002] Display documentation for resmoke.py tasks in Evergreen where available Created: 24/Jan/21 Updated: 29/Oct/23 Resolved: 17/Mar/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.9.0 |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Robert Guo (Inactive) | Assignee: | Mikhail Shchatko |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Sprint: | STM 2021-03-22 | ||||||||
| Participants: | |||||||||
| Story Points: | 2 | ||||||||
| Description |
|
Add a YAML configuration to resmokeconfig with evergreen task name: documentation key value pairs. Resmoke will print out the documentation for its task_name when a task_name is passed in through the command line and when documentation is available. Since generated tasks have arbitrary suffixes, we should remove any pure numerical suffixes in the task name after the last underscore. (e.g. "_123" should be removed, but "_abc" should not) |
| Comments |
| Comment by Githook User [ 16/Mar/21 ] |
|
Author: {'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}Message: |
| Comment by Robert Guo (Inactive) [ 11/Mar/21 ] |
|
Yeah, the wall of text is indeed a problem; We should be able to do something clever specifically for BFs to avoid duplicate or misleading doc. It's not going to be perfect, but we easily do something like: if we see dupe BFGs across different tests more than 3 times for a suite or variant , add the docs for that suite or variant. If new variants and/or suites continue to be added and there are more than 2 variants or suites, we add a warning comment saying that the suite and variant docs should be used with caution given the failures are possibly more widespread.
This would mean we get a max of 2 variants + 2 suites + 1 warning = 5 comments for documentation on a given BF ticket.
What do you think charlie.swanson?
|
| Comment by Brooke Miller [ 04/Mar/21 ] |
|
robert.guo will follow-up to your last question, Charlie, before Misha begins working on this. |
| Comment by Charlie Swanson [ 26/Jan/21 ] |
|
I'm not sure I follow your last point... Is it that any given BF can affect multiple variants? I hear that and I don't think the explanation would be as useful there. But isn't it also true that a BF can be multiple suites? e.g. something like an invariant. Seems like a worthwhile problem to think about in either case. We obviously don't want to overwhelm people with a wall of text, but I think having this available in cases where it's a single variant or single suite would be very valuable. |
| Comment by Robert Guo (Inactive) [ 25/Jan/21 ] |
|
Both are great ideas! I'll file a DAG ticket for the BFG/BF addition. For the build-level doc, it might be misleading to add the description to the ticket as the BF is arbitrarily chosen and could be on any build variant. Given the relatively few number of build variants compared with tasks, maybe we can create a separate document with all the "unique" build variants? |
| Comment by Charlie Swanson [ 25/Jan/21 ] |
|
robert.guo I'm curious about this ticket and idea. Do you have any more details or can you point me to where the request came from? |