[SERVER-58613] Add documentation on which jstest tags have which passthrough effects Created: 24/Mar/21  Updated: 29/Oct/23  Resolved: 22/Jul/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Kyle Suarez Assignee: Mikhail Shchatko
Resolution: Fixed Votes: 2
Labels: dp-qp-stakeholder-request-2021-04, dp-qp-stakeholder-request-2021-07, tig-qwin-eligible
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-54002 Display documentation for resmoke.py ... Closed
Backwards Compatibility: Fully Compatible
Sprint: STM 2021-07-26
Participants:
Story Points: 3

 Description   

For engineers on Query it is a pain point for us to add new tests to jscore. With the increasing number of passthroughs and jstest tags, it's not easy to tell at-a-glance what tags are available and/or might be relevant for someone writing a new test.

This is a request to add some sort of documentation about the test tags. It would be cool for this to be done automatically somehow by parsing our YAML definitions, so that newly-added tags are automatically documented somewhere.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Ethan Zhang (Inactive) [ 26/Jul/21 ]

Thanks mikhail.shchatko, this sounds good.

Comment by Mikhail Shchatko [ 22/Jul/21 ]

With the commit above a new resmoke list-tags subcommand is available:

# print out all tags from all suites
$buildscripts/resmoke.py list-tags
# print out tags from listed suites
$buildscripts/resmoke.py list-tags --suites=[SUITE1,SUITE2]

ethan.zhang are you OK with using grep for the tags in buildscripts/resmokeconfig? E.g.

grep assumes_write_concern_unchanged buildscripts/resmokeconfig/**/*.yml

Comment by Githook User [ 22/Jul/21 ]

Author:

{'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}

Message: SERVER-58613 Add list-tags subcommand to resmoke
Branch: master
https://github.com/mongodb/mongo/commit/73b4bcdabaddc1e2b5b59a19e0d7bae969a85669

Comment by Kyle Suarez [ 30/Mar/21 ]

I've added this to the bottom of the stakeholder spreadsheet. Thanks to the STM team for considering this.

In terms of priority, this is definitely not a blocker. But the potential impact is high, in my opinion, since reducing testing friction helps to speed along the rate at which engineers can complete tasks. The Server and Query teams are also rapidly expanding, so better tooling around controlling jstest tags will also help with the onboarding experience for new engineers in general.

Comment by Robert Guo (Inactive) [ 30/Mar/21 ]

Linking SERVER-54002 for related work on task/suite/evg documentation. There's active discussion on this and we hope to make good progress in the next few months

Comment by Brooke Miller [ 30/Mar/21 ]

Thanks for the request kyle.suarez! Would you mind including this in the next stakeholder spreadsheet (for April)? It'd be helpful to understand the priority and impact of this, alongside other quick-win requests for next quarter that your team would like us to consider doing.

Comment by Ethan Zhang (Inactive) [ 25/Mar/21 ]

Currently, the resmoke.py tool has a `find-suites` that can list the suites where a particular jstest is run. But I think it would be even nicer if we can have some utilities to show if we add/remove a tag, what are the affected suites.

Generated at Thu Feb 08 05:44:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.