Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30565

burn_in_tests.py shouldn't cause compile task to fail

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.11, 3.5.12
    • Component/s: Testing Infrastructure
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v3.4
    • Sprint:
      TIG 2017-08-21
    • Linked BF Score:
      0

      Description

      We occasionally experience network connectivity issues with the Evergreen API server while running tasks in Evergreen. This causes burn_in_tests.py to exit with a non-zero return code because of an uncaught connection error.

      [2017/08/02 15:25:46.431] # Capture a list of new and modified tests.
      [2017/08/02 15:25:46.431] /opt/mongodbtoolchain/v2/bin/python2 buildscripts/burn_in_tests.py --branch=master --buildVariant=suse12 --testListOutfile=jstests/new_tests.json --noExec $burn_in_args
      [2017/08/02 15:25:53.510] Traceback (most recent call last):
      [2017/08/02 15:25:53.510]   File "buildscripts/burn_in_tests.py", line 416, in <module>
      [2017/08/02 15:25:53.510]     main()
      [2017/08/02 15:25:53.510]   File "buildscripts/burn_in_tests.py", line 376, in main
      [2017/08/02 15:25:53.510]     values.check_evergreen)
      [2017/08/02 15:25:53.510]   File "buildscripts/burn_in_tests.py", line 158, in find_changed_tests
      [2017/08/02 15:25:53.510]     last_activated = find_last_activated_task(revs_to_check, buildvariant, branch_name)
      [2017/08/02 15:25:53.510]   File "buildscripts/burn_in_tests.py", line 124, in find_last_activated_task
      [2017/08/02 15:25:53.510]     response = requests.get(api_prefix + "projects/" + project + "/revisions/" + githash)
      [2017/08/02 15:25:53.510]   File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/api.py", line 70, in get
      [2017/08/02 15:25:53.511]     return request('get', url, params=params, **kwargs)
      [2017/08/02 15:25:53.511]   File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/api.py", line 56, in request
      [2017/08/02 15:25:53.511]     return session.request(method=method, url=url, **kwargs)
      [2017/08/02 15:25:53.511]   File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
      [2017/08/02 15:25:53.616]     resp = self.send(prep, **send_kwargs)
      [2017/08/02 15:25:53.616]   File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
      [2017/08/02 15:25:53.616]     r = adapter.send(request, **kwargs)
      [2017/08/02 15:25:53.616]   File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
      [2017/08/02 15:25:53.756]     raise ConnectionError(e, request=request)
      [2017/08/02 15:25:53.756] requests.exceptions.ConnectionError: HTTPConnectionPool(host='evergreen-api.mongodb.com', port=8080): Max retries exceeded with url: /rest/v1/projects/mongodb-mongo-master/revisions/fd01541d77ca1455c603c411f395d6ce34fab6f1 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc1f2eb5c90>: Failed to establish a new connection: [Errno 111] Connection refused',))
      

      We should remove the invocation of burn_in_tests.py the "compile" task and instead change the "burn_in_tests" task to run the git.get_project command to clone the mongodb/mongo repository and apply the user's patch if necessary. If we continue use the --noExec invocation of burn_in_tests.py, then we can make this logic in the "compile" task a command of type="system" so that failing to communicate with the Evergreen API server causes the task to turn purple.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: