[SERVER-60684] Exclude Tests and Tasks tagged with "serverless" from running on last-lts Created: 14/Oct/21 Updated: 16/Oct/23 |
|
| Status: | Open |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Robert Guo (Inactive) | Assignee: | Backlog - Service Architecture |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | branch-6.0 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Service Arch
|
||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v6.1, v6.0
|
||||||||||||||||||||||||||||||||||||
| Sprint: | STM 2021-12-13, STM 2022-03-07, DAG 2022-06-13, Server Serverless 2022-12-26, Server Serverless 2023-01-09, Server Serverless 2023-01-23, Server Serverless 2023-02-06, Server Serverless 2023-02-20, Server Serverless 2023-05-01 | ||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 35 | ||||||||||||||||||||||||||||||||||||
| Story Points: | 1 | ||||||||||||||||||||||||||||||||||||
| Description |
|
Build a mechanism to prevent Evergreen tasks and JS tests tagged with "serverless-only" from running on last-lts branches. The implementation should allow for easy expansion to enable arbitrary features to not run on arbitrary branches. Impl. sketch: |
| Comments |
| Comment by Esha Maharishi (Inactive) [ 21/Dec/22 ] |
|
We decided Server Serverless will manually delete the tests on 6.0. |
| Comment by Steven Vannelli [ 07/Dec/22 ] |
|
Ah fantastic, thanks juan.gu@mongodb.com! |
| Comment by Juan Gu [ 07/Dec/22 ] |
|
steven.vannelli@mongodb.com , It has been scheduled and should be fixed by end of this week or early next week at the latest. |
| Comment by Steven Vannelli [ 07/Dec/22 ] |
|
alexander.neben@mongodb.com juan.gu@mongodb.com just checking in on this one since the associated hot BF (BF-26914) is still open. Do we have a plan for being able to close that out? |
| Comment by Alex Neben [ 28/Nov/22 ] |
|
1. Create another ticket to replace this one for 6.0 |
| Comment by Esha Maharishi (Inactive) [ 09/Nov/22 ] |
|
Bringing this back into Needs Scheduling for SDP: Now that cloud is running serverless on v6.1, we'd like to stop running serverless tests on the v6.0 branch. We had filed this ticket last year when we similarly wanted to stop running serverless tests on v5.0. As a stop-gap we manually deleted the serverless tests from v5.0. I wanted to check in if SDP could automate this process, since otherwise we will have to take this manual step every year. We've tagged the tests that should stop being run with a "serverless" test tag (example). |
| Comment by Robert Guo (Inactive) [ 13/Apr/22 ] |
|
(Update) This is a different request. Reopening.
|
| Comment by Githook User [ 31/Jan/22 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: SERVER-60684 Delete Serverless tests from 5.0 branch |
| Comment by Githook User [ 28/Jan/22 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: SERVER-60684 Tag serverless tests, 2 |
| Comment by Githook User [ 28/Jan/22 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: SERVER-60684 Tag serverless tests |
| Comment by A. Jesse Jiryu Davis [ 28/Jan/22 ] |
|
We have enough info in multiversionconstants.py to determine if the current branch is latest, last-continuous, and/or last-lts. Plan:
The last rule only really matters for last-lts, since we quickly stop testing entirely on other branches older than last-continuous. We could rename the "serverless" tag to something like "skip-if-older-than-last-continuous" if we find non-Serverless uses for it. |
| Comment by A. Jesse Jiryu Davis [ 27/Jan/22 ] |
|
Esha and Louisa tell me we can disable serverless tests for 5.0 and 5.1. |
| Comment by Robert Guo (Inactive) [ 14/Dec/21 ] |
|
Thinking about this a bit more, I realized we don't currently have a way for an lts branch to know that it's LTS. Doing so would likely require a manual change to the release branch, and may not be meaningfully better than manually removing the tests during branching using the steps described in this ticket description since it is only once a year. I'm going to put this ticket under the 6.0 branching work with a label and sync up with the build barons to collaborate on this change going forward. Reducing redundant test coverage for continuous releases is being initially discussed in PROG-5936. Most of the work there may be on the cloud side; we would still have to manually enable or disable an Evergreen project. |
| Comment by Brooke Miller [ 19/Oct/21 ] |
|
robert.guo is going to file a separate ticket in order to allow us to enforce tags for modified tests and suites |
| Comment by Robert Guo (Inactive) [ 16/Oct/21 ] |
|
Yeah that would be nice. We can update the ticket to have tests be tagged based on features and add a new linter rule to check added/moved tests have one or more feature tag. |
| Comment by Judah Schvimer [ 14/Oct/21 ] |
|
Should this be more general, to exclude serverless-only tests from any branch besides last-continuous? Also, should we tag tests with features and tag features with serverless-only so we can remove serverless-only easily from features in the future if desirable? |