[DRIVERS-2012] Mark "max retries exceeded" as atlas failure Created: 20/Dec/21  Updated: 28/Oct/23  Resolved: 22/Dec/21

Status: Closed
Project: Drivers
Component/s: Atlas Testing
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Oleg Pudeyev (Inactive) Assignee: Oleg Pudeyev (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: Astrolabe Testing Improvements
Driver Changes: Not Needed
Quarter: FY22Q4

 Description   

This exception is an intermittent atlas problem and should be marked as such:

1/12/17 05:14:49.612] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 643.2 sec
 [2021/12/17 05:14:50.649] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 644.3 sec
 [2021/12/17 05:14:51.691] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 645.3 sec
 [2021/12/17 05:14:52.729] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 646.3 sec
 [2021/12/17 05:14:53.774] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 647.4 sec
 [2021/12/17 05:14:54.840] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 648.5 sec
 [2021/12/17 05:14:55.881] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 649.5 sec
 [2021/12/17 05:14:56.921] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 650.5 sec
 [2021/12/17 05:14:57.967] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 651.6 sec
 [2021/12/17 05:14:59.015] INFO:astrolabe.runner:Cluster 96146eba85: current state: creating; wanted state: idle; waited for 652.6 sec
 [2021/12/17 05:15:02.889] Traceback (most recent call last):
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn
 [2021/12/17 05:15:02.889]     (self._dns_host, self.port), self.timeout, **extra_kw
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/util/connection.py", line 96, in create_connection
 [2021/12/17 05:15:02.889]     raise err
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/util/connection.py", line 86, in create_connection
 [2021/12/17 05:15:02.889]     sock.connect(sa)
 [2021/12/17 05:15:02.889] OSError: [Errno 101] Network is unreachable
 [2021/12/17 05:15:02.889] During handling of the above exception, another exception occurred:
 [2021/12/17 05:15:02.889] Traceback (most recent call last):
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
 [2021/12/17 05:15:02.889]     chunked=chunked,
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
 [2021/12/17 05:15:02.889]     self._validate_conn(conn)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
 [2021/12/17 05:15:02.889]     conn.connect()
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connection.py", line 358, in connect
 [2021/12/17 05:15:02.889]     conn = self._new_conn()
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn
 [2021/12/17 05:15:02.889]     self, "Failed to establish a new connection: %s" % e
 [2021/12/17 05:15:02.889] urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fd827221eb8>: Failed to establish a new connection: [Errno 101] Network is unreachable
 [2021/12/17 05:15:02.889] During handling of the above exception, another exception occurred:
 [2021/12/17 05:15:02.889] Traceback (most recent call last):
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
 [2021/12/17 05:15:02.889]     timeout=timeout
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
 [2021/12/17 05:15:02.889]     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment
 [2021/12/17 05:15:02.889]     raise MaxRetryError(_pool, url, error or ResponseError(cause))
 [2021/12/17 05:15:02.889] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cloud-qa.mongodb.com', port=443): Max retries exceeded with url: /api/atlas/v1.0/groups/605cd3edefdd5e52ab52bd92/clusters/96146eba85 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fd827221eb8>: Failed to establish a new connection: [Errno 101] Network is unreachable',))
 [2021/12/17 05:15:02.889] During handling of the above exception, another exception occurred:
 [2021/12/17 05:15:02.889] Traceback (most recent call last):
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/atlasclient/client.py", line 213, in request
 [2021/12/17 05:15:02.889]     response = requests.request(method, url, **request_kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/requests/api.py", line 61, in request
 [2021/12/17 05:15:02.889]     return session.request(method=method, url=url, **kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
 [2021/12/17 05:15:02.889]     resp = self.send(prep, **send_kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
 [2021/12/17 05:15:02.889]     r = adapter.send(request, **kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
 [2021/12/17 05:15:02.889]     raise ConnectionError(e, request=request)
 [2021/12/17 05:15:02.889] requests.exceptions.ConnectionError: HTTPSConnectionPool(host='cloud-qa.mongodb.com', port=443): Max retries exceeded with url: /api/atlas/v1.0/groups/605cd3edefdd5e52ab52bd92/clusters/96146eba85 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fd827221eb8>: Failed to establish a new connection: [Errno 101] Network is unreachable',))
 [2021/12/17 05:15:02.889] During handling of the above exception, another exception occurred:
 [2021/12/17 05:15:02.889] Traceback (most recent call last):
 [2021/12/17 05:15:02.889]   File "astrolabevenv/bin/astrolabe", line 33, in <module>
 [2021/12/17 05:15:02.889]     sys.exit(load_entry_point('astrolabe', 'console_scripts', 'astrolabe')())
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 829, in __call__
 [2021/12/17 05:15:02.889]     return self.main(*args, **kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 782, in main
 [2021/12/17 05:15:02.889]     rv = self.invoke(ctx)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
 [2021/12/17 05:15:02.889]     return _process_result(sub_ctx.command.invoke(sub_ctx))
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
 [2021/12/17 05:15:02.889]     return _process_result(sub_ctx.command.invoke(sub_ctx))
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
 [2021/12/17 05:15:02.889]     return ctx.invoke(self.callback, **ctx.params)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/core.py", line 610, in invoke
 [2021/12/17 05:15:02.889]     return callback(*args, **kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabevenv/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
 [2021/12/17 05:15:02.889]     return f(get_current_context(), *args, **kwargs)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabe/cli.py", line 453, in run_single_test
 [2021/12/17 05:15:02.889]     failed = runner.run()
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabe/runner.py", line 408, in run
 [2021/12/17 05:15:02.889]     return self.do_run()
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabe/runner.py", line 428, in do_run
 [2021/12/17 05:15:02.889]     active_case.wait_for_idle()
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/astrolabe/runner.py", line 315, in wait_for_idle
 [2021/12/17 05:15:02.889]     cluster_info = self.cluster_url.get().data
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/atlasclient/client.py", line 56, in get
 [2021/12/17 05:15:02.889]     return self._client.request('GET', self._path, **params)
 [2021/12/17 05:15:02.889]   File "/data/mci/8fd35e75b2490c44d971227a8cca21df/astrolabe-src/atlasclient/client.py", line 218, in request
 [2021/12/17 05:15:02.889]     request_method=method
 [2021/12/17 05:15:02.889] atlasclient.exceptions.AtlasClientError: HTTPSConnectionPool(host='cloud-qa.mongodb.com', port=443): Max retries exceeded with url: /api/atlas/v1.0/groups/605cd3edefdd5e52ab52bd92/clusters/96146eba85 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fd827221eb8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)) (GET https://cloud-qa.mongodb.com/api/atlas/v1.0/groups/605cd3edefdd5e52ab52bd92/clusters/96146eba85)
 [2021/12/17 05:15:02.890] Finished 'subprocess.exec' in "run test" in 11m2.487427182s

HTTP error could also be a genuine astrolabe problem, therefore only the above message should be matched and only in that case should the build be marked purple.



 Comments   
Comment by Githook User [ 22/Dec/21 ]

Author:

{'name': 'Oleg Pudeyev', 'email': 'code@olegp.name', 'username': 'p'}

Message: DRIVERS-2012 Mark "max retries exceeded" as atlas failure
Branch: master
https://github.com/mongodb-labs/drivers-atlas-testing/commit/a55be51ba1625c0ab0385885077d6c2940ae5bd2

Generated at Thu Feb 08 08:24:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.