[SERVER-60813] Coverage gap between multiversion burn in and dedicate multiversion suite Created: 19/Oct/21  Updated: 09/Nov/21  Resolved: 09/Nov/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Robert Guo (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Sprint: STM 2021-11-15
Participants:

 Description   

We should add a sanity check for implicit multiversion suites so new behavior that causes breakages in unmodified multiversion tests.

Problem Example:

  • if someone changes behavior (i.e. protocol for initialization order) on master and modified a test and that suite doesn't run in any multiversion suites, then they won't get any multiversion test coverage.


 Comments   
Comment by Robert Guo (Inactive) [ 09/Nov/21 ]

Not enough info around the original context to proceed here. My understanding had been that we need to make burn_in run more multiversion tests based on some other criteria, but thinking about it more, there should be at least one JS test that gets modified, so running that JS test using existing burn in logic should be sufficient. The change would be adding a new multiversion test suite, not burn_in. Closing as won't fix.

Feel free to re-open if there's any case I may have missed.

Comment by Brooke Miller [ 26/Oct/21 ]

robert.guo is going to come back to this to better understand whether the problem exists (with an example).

Generated at Thu Feb 08 05:50:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.