[SERVER-54002] Display documentation for resmoke.py tasks in Evergreen where available Created: 24/Jan/21  Updated: 29/Oct/23  Resolved: 17/Mar/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.9.0

Type: New Feature Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Mikhail Shchatko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-58613 Add documentation on which jstest tag... Closed
Backwards Compatibility: Fully Compatible
Sprint: STM 2021-03-22
Participants:
Story Points: 2

 Description   

Add a YAML configuration to resmokeconfig with evergreen task name: documentation key value pairs. Resmoke will print out the documentation for its task_name when a task_name is passed in through the command line and when documentation is available.

Since generated tasks have arbitrary suffixes, we should remove any pure numerical suffixes in the task name after the last underscore. (e.g. "_123" should be removed, but "_abc" should not)



 Comments   
Comment by Githook User [ 16/Mar/21 ]

Author:

{'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}

Message: SERVER-54002 Evergreen task documentation
Branch: master
https://github.com/mongodb/mongo/commit/8c303d8fa59213f27567da4582be899c71111400

Comment by Robert Guo (Inactive) [ 11/Mar/21 ]

Yeah, the wall of text is indeed a problem; We should be able to do something clever specifically for BFs to avoid duplicate or misleading doc. It's not going to be perfect, but we easily do something like: if we see dupe BFGs across different tests more than 3 times for a suite or variant , add the docs for that suite or variant. If new variants and/or suites continue to be added and there are more than 2 variants or suites, we add a warning comment saying that the suite and variant docs should be used with caution given the failures are possibly more widespread.

 

This would mean we get a max of 2 variants + 2 suites + 1 warning = 5 comments for documentation on a given BF ticket.

 

What do you think charlie.swanson?

 

Comment by Brooke Miller [ 04/Mar/21 ]

robert.guo will follow-up to your last question, Charlie, before Misha begins working on this.

Comment by Charlie Swanson [ 26/Jan/21 ]

I'm not sure I follow your last point... Is it that any given BF can affect multiple variants? I hear that and I don't think the explanation would be as useful there. But isn't it also true that a BF can be multiple suites? e.g. something like an invariant. Seems like a worthwhile problem to think about in either case.

We obviously don't want to overwhelm people with a wall of text, but I think having this available in cases where it's a single variant or single suite would be very valuable.

Comment by Robert Guo (Inactive) [ 25/Jan/21 ]

Both are great ideas! I'll file a DAG ticket for the BFG/BF addition.

For the build-level doc, it might be misleading to add the description to the ticket as the BF is arbitrarily chosen and could be on any build variant. Given the relatively few number of build variants compared with tasks, maybe we can create a separate document with all the "unique" build variants?

Comment by Charlie Swanson [ 25/Jan/21 ]

robert.guo I'm curious about this ticket and idea. Do you have any more details or can you point me to where the request came from?

Generated at Thu Feb 08 05:32:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.