[SERVER-27627] Use requests library to manage HTTP posts in resmoke.py buildlogger support Created: 10/Jan/17  Updated: 05/Apr/17  Resolved: 02/Mar/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.5.4

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Robert Guo (Inactive)
Resolution: Done Votes: 0
Labels: tig-resmoke
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-28125 Remove Compatibility Code for Python ... Backlog
is related to SERVER-22150 multiversion download script should u... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4, v3.2
Sprint: TIG 2017-03-06
Participants:
Linked BF Score: 0

 Description   

It appears that urllib2 does no connection pooling, so every POST of log data to buildlogger requires a new HTTP connection. Moving to the requests (or some other more modern HTTP library) should allow pooling.



 Comments   
Comment by Githook User [ 01/Mar/17 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-27627 Tolerate much older versions of Python's requests package.
Branch: master
https://github.com/mongodb/mongo/commit/5cd4ba841e76a2c38ad3c7f9d5d9c2ab6c4ed293

Comment by Githook User [ 28/Feb/17 ]

Author:

{u'username': u'guoyr', u'name': u'Robert Guo', u'email': u'robert.guo@10gen.com'}

Message: SERVER-27627 use requests instead of urllib2 in resmoke.py
Branch: master
https://github.com/mongodb/mongo/commit/c203a3be8076c4939011d03e958bc010422ac86d

Comment by Sam Kleinman (Inactive) [ 24/Feb/17 ]

Insofar as I'm entitled to an opinion, I would say:

  • use toolchain python, which should be new enough, if/when that's available.
  • verify false with tickets to remove when toolchain python is available
Comment by Max Hirschhorn [ 24/Feb/17 ]

Until Python 2.7.9, the urllib2 module didn't verify the certificate presented for HTTPS connections, so we could also choose to pass verify=False to requests.post() and suppress the warning from urllib3.

diff --git a/buildscripts/resmokelib/logging/handlers.py b/buildscripts/resmokelib/logging/handlers.py
index 0a9f3db075..5afd572d19 100644
--- a/buildscripts/resmokelib/logging/handlers.py
+++ b/buildscripts/resmokelib/logging/handlers.py
@@ -8,9 +8,11 @@ from __future__ import absolute_import
 import json
 import logging
 import threading
+import warnings
 
 import requests
 import requests.auth
+import requests.exceptions
 
 from .. import utils
 from ..utils import timer
@@ -174,8 +176,14 @@ class HTTPHandler(object):
 
         url = self._make_url(endpoint)
 
-        response = requests.post(url, data=data, headers=headers, timeout=timeout_secs,
-                                 auth=self.auth_handler)
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore", requests.exceptions.InsecureRequestWarning)
+            response = requests.post(url,
+                                     data=data,
+                                     headers=headers,
+                                     timeout=timeout_secs,
+                                     auth=self.auth_handler,
+                                     verify=False)
 
         response.raise_for_status()

Comment by Max Hirschhorn [ 24/Feb/17 ]

I asked robert.guo to revert this change because logs aren't being sent to logkeeper as a result of the SSLError seen below. IIUC, the version of Python we use on many build variants doesn't support SNI, which causes the load balancer to present the certificate for .com rather than .org and for certificate validation to fail. CC ernie.hershey, sam.kleinman

Edit: See also http://docs.python-requests.org/en/master/community/faq/#what-are-hostname-doesn-t-match-errors

Python3 and Python 2.7.9+ include native support for SNI in their SSL modules. For information on using SNI with Requests on Python < 2.7.9 refer to this Stack Overflow answer.

Example task output

https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_aggregation_WT_14f16f384a2ace3b5ccb45dcbfbb66f3f57e945a_17_02_24_15_54_33

[2017/02/24 16:37:54.580] /usr/lib/python2.6/site-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
[2017/02/24 16:37:54.580]   SNIMissingWarning
[2017/02/24 16:37:54.580] /usr/lib/python2.6/site-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
[2017/02/24 16:37:54.580]   InsecurePlatformWarning
[2017/02/24 16:37:54.617] [buildlogger:fallback] Encountered an error.
[2017/02/24 16:37:54.617] Traceback (most recent call last):
[2017/02/24 16:37:54.617]   File "/data/mci/26ec212d536166ac3f0289cb3cb05ae1/src/buildscripts/resmokelib/logging/buildlogger.py", line 37, in wrapper
[2017/02/24 16:37:54.617]     return func(*args, **kwargs)
[2017/02/24 16:37:54.617]   File "/data/mci/26ec212d536166ac3f0289cb3cb05ae1/src/buildscripts/resmokelib/logging/buildlogger.py", line 88, in new_build_id
[2017/02/24 16:37:54.617]     "task_id": _config.TASK_ID,
[2017/02/24 16:37:54.617]   File "/data/mci/26ec212d536166ac3f0289cb3cb05ae1/src/buildscripts/resmokelib/logging/handlers.py", line 178, in post
[2017/02/24 16:37:54.617]     auth=self.auth_handler)
[2017/02/24 16:37:54.617]   File "/usr/lib/python2.6/site-packages/requests/api.py", line 110, in post
[2017/02/24 16:37:54.617]     return request('post', url, data=data, json=json, **kwargs)
[2017/02/24 16:37:54.617]   File "/usr/lib/python2.6/site-packages/requests/api.py", line 56, in request
[2017/02/24 16:37:54.617]     return session.request(method=method, url=url, **kwargs)
[2017/02/24 16:37:54.617]   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 488, in request
[2017/02/24 16:37:54.617]     resp = self.send(prep, **send_kwargs)
[2017/02/24 16:37:54.617]   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 609, in send
[2017/02/24 16:37:54.617]     r = adapter.send(request, **kwargs)
[2017/02/24 16:37:54.617]   File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 497, in send
[2017/02/24 16:37:54.617]     raise SSLError(e, request=request)
[2017/02/24 16:37:54.617] SSLError: hostname 'logkeeper.mongodb.org' doesn't match either of '*.mongodb.com', 'mongodb.com'
[2017/02/24 16:37:54.618] [executor:js_test] 2017-02-24T16:37:54.617+0000 Encountered an error configuring buildlogger for job #0, falling back to stderr.
[2017/02/24 16:37:54.621] /usr/lib/python2.6/site-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
[2017/02/24 16:37:54.621]   InsecurePlatformWarning

Comment by Githook User [ 24/Feb/17 ]

Author:

{u'username': u'guoyr', u'name': u'Robert Guo', u'email': u'robert.guo@10gen.com'}

Message: Revert "SERVER-27627 use requests instead of urllib2 in resmoke.py"

This reverts commit 14f16f384a2ace3b5ccb45dcbfbb66f3f57e945a.
Branch: master
https://github.com/mongodb/mongo/commit/973b8b9da39db84073e98d4979ec3a8d6179b217

Comment by Githook User [ 24/Feb/17 ]

Author:

{u'username': u'guoyr', u'name': u'Robert Guo', u'email': u'robert.guo@10gen.com'}

Message: SERVER-27627 use requests instead of urllib2 in resmoke.py
Branch: master
https://github.com/mongodb/mongo/commit/14f16f384a2ace3b5ccb45dcbfbb66f3f57e945a

Comment by Max Hirschhorn [ 24/Jan/17 ]

We should also see if it is possible to use the requests package instead of shelling out to curl in the setup_multiversion_mongodb.py. (See SERVER-22150)

Generated at Thu Feb 08 04:15:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.