[SERVER-72606] Filter tests which are incompatible with feature flags provided on the command line Created: 06/Jan/23 Updated: 05/Feb/24 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Matt Broadstone | Assignee: | Zack Winter |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Correctness
|
||||||||
| Participants: | |||||||||
| Description |
|
Earlier I was trying to debug a failure in the tenant_migration_donor_retry.js test which is marked as incompatible_with_shard_merge. I didn't realize that I was accidentally providing --additionalFeatureFlags featureFlagShardMerge in my local resmoke invocation (a habit because I had been running a lot of tests which require that feature flag recently), so when I ran the test locally I was getting a very different error than in my patch build. This feels like a class of error we should be able to catch programmatically. |
| Comments |
| Comment by Tausif Rahman (Inactive) [ 07/Apr/23 ] |
|
From yuhong.zhang@mongodb.com via slack: |
| Comment by Tausif Rahman (Inactive) [ 04/Mar/23 ] |
|
We have I don't have all the details of this issue, but if you want to exclude incompatible_with_shard_merge, you would need to pass that in as --excludeWithAnyTags=incompatible_with_shard_merge – which is probably what the evergreen task you were trying to reproduce was using. For resmoke to know that if featureFlagShardMerge is on, we should not run incompatible_with_shard_merge is going to be challenging (let me know if I am missing something here). For your case, I think we need to improve the local repro experience so that you have all of the same flags locally as the evergreen task runs with in CI. I believe that would have helped you in this situation. I've opened |