Introduce multiversion testing against minor versions on LTS branches

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Won't Do
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Following DEVPROD-12400, there will no longer be multiversion query fuzzer testing on LTS branches as this doesn't provide a lot of useful test coverage (given that these branches aren't expected to change very much at all). 

      An alternative approach to multiversion fuzz testing LTS branches is to test the latest minor version for a given branch against the previous minor version. As an example, if 7.0.15 is the latest 7.0 version, the multiversion fuzzer on 7.0 would run 7.0.15 against 7.0.14. 

      This minor version multiversion fuzz testing would be useful in that any behavioral changes between minor versions would likely be picked up and could (possibly) indicate that a bug was introduced to an LTS version. 

      Some considerations:

      • We would need to be careful in the case that we are actually backporting a behavioral change (i.e. to suppress false positive errors). One thought is that if a known behavioral change is backported, we can pause minor version fuzz testing until another patch release goes out.
      • We would need to figure out what to do when an LTS version comes out (i.e. 9.0.0 can't have any minor version testing until 9.0.1 is released).

      Note that this might be larger than one ticket; this is simply a placeholder to capture the idea.

            Assignee:
            Philip Stoev
            Reporter:
            Mihai Andrei
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: