Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-74940

Renamed tests are missed by generate_multiversion_exclude_tags.py

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Storage Execution
    • ALL

      Tests that need to be ignored on multiversion suites until a ticket is backported have to be added to backports_required_for_multiversion_tests.yml, the generate_multiversion_exclude_tags.py script will then process this file and compare it with the `old-lts` version of the same file, to check if the test has been backported or not. 

      If a test gets renamed only on the newer branch these files will get out of sync, and generate_multiversion_exclude_tags.py will always think the ticket was not backported, so the test will not be run in the multiversion suites, losing test coverage.

      This is what happened when we renamed many tests in SERVER-64473, for instance this line from master and this line from v6.0 don't match, so the test will not be run.

      New backports could also face the same problem, one workaround is to add both filenames to the old branch like this, but this is easy to miss. Another option is to backport SERVER-64473, so that the directory structure is the same on all branches.

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            daniel.gomezferro@mongodb.com Daniel Gomez Ferro
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: