[SERVER-29643] Add Python unit tests for buildscript/update_test_lifecycle.py Created: 14/Jun/17  Updated: 30/Oct/23  Resolved: 17/Jul/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.5.11

Type: Task Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Max Hirschhorn
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-29644 Add support for resmoke.py to run Pyt... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG 2017-07-10, TIG 2017-07-31
Participants:

 Description   

We should add tests for the buildscripts/update_test_lifecycle.py script from SERVER-28786 to ensure it correctly handles updating the etc/test_lifecycle.yml file given an initial configuration and a set of Evergreen test results.

  • (test, task, variant, distro) combination becomes unreliable
  • (test, task, variant, distro) combination becomes reliable
  • (test, task, variant, distro) combination doesn't run frequently enough in the configured time span to be analyzed


 Comments   
Comment by Githook User [ 17/Jul/17 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-29642 SERVER-29643 Add Python tests for test lifecycle scripts.

For test_failures.py:

  • Replaces HistoryReport with a TestHistory class that has
    get_history_by_revision() and get_history_by_date() methods. They
    both return a list of ReportEntry tuples that can be used to
    construct a Report instance.
  • Adds Python unit test cases for the Report and ReportEntry classes.
  • Creates Wildcard class as separate concept from Missing class.
  • Enables --sinceDate and --untilDate with a warning that the script
    may not return a complete result set.
  • Adds support for running the script with Python 3.

For update_test_lifecycle.py:

  • Introduces Config namedtuple to represent the test lifecycle model.
  • Adds Python unit tests cases for the update_tags() function.
  • Takes advantage of the partial grouping so that computing summaries
    for (test, task, variant), (test, task), and (test,) combinations do
    not require re-processing the entire result set.
    Branch: master
    https://github.com/mongodb/mongo/commit/58a3909a3f678dec7bd94bfb38f96756c970113e
Generated at Thu Feb 08 04:21:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.