[SERVER-79168] Improper test excludes should be caught before they are merged Created: 20/Jul/23 Updated: 29/Oct/23 Resolved: 26/Aug/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.1.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Memento Slack Bot | Assignee: | Steve Gross |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Server Development Platform
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Participants: | |||||||||
| Description |
|
A new test suite was added into the mongo repo. However, this suite definition included excludes for tests that are not a part of the community mongo repo itself. This causes issues when this suite is attempted to be run on variants that are community based. There should be some mechanism in place that checks for cases like this and prevents them from occurring in the future. This might be done as part of the commit queue or version generation. |
| Comments |
| Comment by Githook User [ 25/Aug/23 ] |
|
Author: {'name': 'Steve Gross', 'email': 'steve.gross@mongodb.com', 'username': 'stevegrossmongodb'}Message: |
| Comment by Steve Gross [ 23/Aug/23 ] |
|
Question: The description claims "new test suite was added into the mongo repo". Is it this change? |
| Comment by Steve Gross [ 23/Aug/23 ] |
|
Note: Assigned to me (since I'm working on https://jira.mongodb.org/browse/SERVER-80232) |
| Comment by Steve Gross [ 22/Aug/23 ] |
|
Note: Per discussion w/ jeff.zambory@mongodb.com I've moved this to SDP's responsibility. |
| Comment by Jeffrey Zambory [ 21/Jul/23 ] |
|
Thanks for the clarification Max. Regarding burn_in, I guess I was hoping for there to have been a new or touched jstest to have been a part of the new suite and for that to have caused burn_in to trigger for it but that doesn't seem to have been the case. |
| Comment by Jeffrey Zambory [ 21/Jul/23 ] |
|
I don't think it should have but I'm not certain. max.hirschhorn@mongodb.com or adi.zaimi@mongodb.com can you comment on this, do you know if this all passed in the patch build run before this suite was added, including in the burn_in variants? |
| Comment by Alex Neben [ 21/Jul/23 ] |
|
But how did this pass a required patch build? |
| Comment by Jeffrey Zambory [ 21/Jul/23 ] |
|
The issue, as I understand it, is that a test suite was added to the mongo repo. This suite included excludes in it which referenced tests that only exist in the enterprise module. So, when the suite was attempted to be run on variants that are community only, these tests and paths did not exist, causing resmoke to error out. Burn_in was the mechanism by which the suite was run on a community variant. There's previous history in regards to adding configuration checks to the commit queue:
I think the check here would be similar in that resmoke would need to check if the suite configuration was valid in some way. This was an error surfaced by resmoke so resmoke should have the capability to check the config in some manner. |
| Comment by Alex Neben [ 21/Jul/23 ] |
|
I'm still a little confused on the problem. To me it seems that the bug is that this fails in burn in but not elsewhere? |