[SERVER-27683] burn_in_tests.py script doesn't handle when a new build variant is added to Evergreen Created: 13/Jan/17  Updated: 05/Apr/17  Resolved: 15/Mar/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.4.3, 3.5.5

Type: Bug Priority: Minor - P4
Reporter: Max Hirschhorn Assignee: Max Hirschhorn
Resolution: Done Votes: 1
Labels: tig-burnin
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
is duplicated by SERVER-28286 burn_in_tests.py can break compile wh... Closed
is duplicated by SERVER-28064 buildscripts/burn_in_tests.py fails f... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4
Steps To Reproduce:

python -c 'from buildscripts import burn_in_tests; burn_in_tests.find_changed_tests("master", None, 1, "variant-that-does-not-exist", True)'

The following patch can be applied to make an error be returned sooner by changing the number of revisions to check.

diff --git a/buildscripts/burn_in_tests.py b/buildscripts/burn_in_tests.py
index b0c362824c..56fb3e6894 100644
--- a/buildscripts/burn_in_tests.py
+++ b/buildscripts/burn_in_tests.py
@@ -199,7 +199,7 @@ def find_changed_tests(branch_name, base_commit, max_revisions, buildvariant, ch
         # previous commit when trying to find the most recent preceding commit that has been
         # activated.
         revs_to_check = callo(["git", "rev-list", base_commit,
-                               "--max-count=200", "--skip=1"]).splitlines()
+                               "--max-count=1", "--skip=1"]).splitlines()
         last_activated = find_last_activated_task(revs_to_check, buildvariant, branch_name)
         print "Comparing current branch against", last_activated
         revisions = callo(["git", "rev-list", base_commit + "..." + last_activated]).splitlines()

Sprint: TIG 2017-03-27
Participants:
Linked BF Score: 0

 Description   

The find_last_activated_task() function script returns None when a build variant has never previously run before. This led to a transient failure when the "Enterprise Ubuntu 16.04 (Clang 3.7/libc++)" builder was added to Evergreen as part of SERVER-27434.

[2016/12/29 20:34:56.709] # Capture a list of new and modified tests.
[2016/12/29 20:34:56.709] python buildscripts/burn_in_tests.py --branch=master --buildVariant=enterprise-ubuntu1604-clang-3.7-libcxx --testListOutfile=jstests/new_tests.json --noExec $burn_in_args
[2016/12/29 20:35:21.843] Traceback (most recent call last):
[2016/12/29 20:35:21.843] Comparing current branch against None
[2016/12/29 20:35:21.843]   File "buildscripts/burn_in_tests.py", line 425, in <module>
[2016/12/29 20:35:21.843]     main()
[2016/12/29 20:35:21.843]   File "buildscripts/burn_in_tests.py", line 391, in main
[2016/12/29 20:35:21.843]     values.check_evergreen)
[2016/12/29 20:35:21.843]   File "buildscripts/burn_in_tests.py", line 205, in find_changed_tests
[2016/12/29 20:35:21.843]     revisions = callo(["git", "rev-list", base_commit + "..." + last_activated]).splitlines()
[2016/12/29 20:35:21.843] TypeError: cannot concatenate 'str' and 'NoneType' objects

I suspect the find_changed_tests() function should just be changed to default last_activated to "HEAD" when the find_last_activated_task() function returns None.

def find_changed_tests(branch_name, base_commit, max_revisions, buildvariant, check_evergreen):
    """
    Use git to find which files have changed in this patch.
    TODO: This should be expanded to search for enterprise modules.
    """
    changed_tests = []
 
    if base_commit is None:
        base_commit = callo(["git", "merge-base", branch_name + "@{upstream}", "HEAD"]).rstrip()
    if check_evergreen:
        # We're going to check up to 200 commits in Evergreen for the last scheduled one.
        # The current commit will be activated in Evergreen; we use --skip to start at the
        # previous commit when trying to find the most recent preceding commit that has been
        # activated.
        revs_to_check = callo(["git", "rev-list", base_commit,
                               "--max-count=200", "--skip=1"]).splitlines()
        last_activated = find_last_activated_task(revs_to_check, buildvariant, branch_name)
        print "Comparing current branch against", last_activated
        revisions = callo(["git", "rev-list", base_commit + "..." + last_activated]).splitlines()
        base_commit = last_activated
    ...



 Comments   
Comment by Githook User [ 16/Mar/17 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-27683 Handle first time build variant runs in burn_in_tests.py.

Changes 'last_activated' to default to HEAD if we fail to find a commit
that's been activated in Evergreen.

(cherry picked from commit 10c9d67de63a38249fca0234208660af8b45f12b)
Branch: v3.4
https://github.com/mongodb/mongo/commit/a237bb9697baa1b168c790b1d22fe1bb05c2538d

Comment by Githook User [ 15/Mar/17 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-27683 Handle first time build variant runs in burn_in_tests.py.

Changes 'last_activated' to default to HEAD if we fail to find a commit
that's been activated in Evergreen.
Branch: master
https://github.com/mongodb/mongo/commit/10c9d67de63a38249fca0234208660af8b45f12b

Generated at Thu Feb 08 04:15:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.