[MONGOCRYPT-276] test-python PyPy 3 task failing on RHEL 6.2 undefined symbol: d2i_DHxparams Created: 01/Sep/20  Updated: 27/Oct/23  Resolved: 03/Sep/20

Status: Closed
Project: Libmongocrypt
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kevin Albertson Assignee: Unassigned
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to PYTHON-2451 test-python PyPy tasks failing on RHE... Closed

 Description   

https://evergreen.mongodb.com/task/libmongocrypt_rhel_62_64_bit_test_python_8530af06643daa28259e5830ce1dff22b6be326a_20_08_26_15_21_12

Appears to be an issue with crypto callbacks. test_data_key_creation fails with:

[2020/09/01 15:19:13.631] Traceback (most recent call last):
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/test/test_mongocrypt.py", line 337, in test_data_key_creation
[2020/09/01 15:19:13.631]     key_alt_names=key_alt_names)
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/pymongocrypt/explicit_encrypter.py", line 112, in create_data_key
[2020/09/01 15:19:13.631]     with self.mongocrypt.data_key_context(kms_provider, opts) as ctx:
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/pymongocrypt/mongocrypt.py", line 228, in data_key_context
[2020/09/01 15:19:13.631]     return DataKeyContext(self._create_context(), kms_provider, opts)
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/pymongocrypt/mongocrypt.py", line 472, in __init__
[2020/09/01 15:19:13.631]     self._raise_from_status()
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/pymongocrypt/mongocrypt.py", line 269, in _raise_from_status
[2020/09/01 15:19:13.631]     raise exc
[2020/09/01 15:19:13.631] pymongocrypt.errors.MongoCryptError: Traceback (most recent call last):
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/pymongocrypt/crypto.py", line 53, in aes_256_cbc_encrypt
[2020/09/01 15:19:13.631]     backend=default_backend())
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/.eggs/cryptography-2.9.2-py3.6-linux-x86_64.egg/cryptography/hazmat/backends/__init__.py", line 15, in default_backend
[2020/09/01 15:19:13.631]     from cryptography.hazmat.backends.openssl.backend import backend
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/.eggs/cryptography-2.9.2-py3.6-linux-x86_64.egg/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module>
[2020/09/01 15:19:13.631]     from cryptography.hazmat.backends.openssl.backend import backend
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/.eggs/cryptography-2.9.2-py3.6-linux-x86_64.egg/cryptography/hazmat/backends/openssl/backend.py", line 75, in <module>
[2020/09/01 15:19:13.631]     from cryptography.hazmat.bindings.openssl import binding
[2020/09/01 15:19:13.631]   File "/data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/.eggs/cryptography-2.9.2-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/openssl/binding.py", line 16, in <module>
[2020/09/01 15:19:13.631]     from cryptography.hazmat.bindings._openssl import ffi, lib
[2020/09/01 15:19:13.631] ImportError: /data/mci/855bd8193a913e7481d4d37155d70e3f/libmongocrypt/bindings/python/.eggs/cryptography-2.9.2-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_openssl.pypy3-71-x86_64-linux-gnu.so: undefined symbol: d2i_DHxparams



 Comments   
Comment by Shane Harvey [ 03/Sep/20 ]

I'm not sure what changed but this error has gone away: https://evergreen.mongodb.com/task/libmongocrypt_rhel_62_64_bit_test_python_59edb3caa54c53e3450b7cf60e6a689cddadf985_20_09_02_19_16_51

 [2020/09/03 21:28:35.202] running build_ext
 [2020/09/03 21:28:35.202] 3.6.1 (784b254d669919c872a505b807db8462b6140973, Apr 16 2019, 18:18:28)
 [2020/09/03 21:28:35.202] [PyPy 7.1.1-beta0 with GCC 8.2.0]
 [2020/09/03 21:28:40.871] + /opt/python/pypy3.6/bin/pypy3 setup.py test
 [2020/09/03 21:28:40.871] WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
 [2020/09/03 21:29:09.388] running test
 [2020/09/03 21:29:09.388] running egg_info
 [2020/09/03 21:29:09.388] writing pymongocrypt.egg-info/PKG-INFO
 [2020/09/03 21:29:09.388] writing dependency_links to pymongocrypt.egg-info/dependency_links.txt
 [2020/09/03 21:29:09.388] writing requirements to pymongocrypt.egg-info/requires.txt
 [2020/09/03 21:29:09.388] writing top-level names to pymongocrypt.egg-info/top_level.txt
 [2020/09/03 21:29:09.399] reading manifest file 'pymongocrypt.egg-info/SOURCES.txt'
 [2020/09/03 21:29:09.399] writing manifest file 'pymongocrypt.egg-info/SOURCES.txt'
 [2020/09/03 21:29:10.160] test_libmongocrypt_version (test.test_binding.TestBinding) ... ok
 [2020/09/03 21:29:10.160] test_mongocrypt_binary_new (test.test_binding.TestBinding) ... ok
 [2020/09/03 21:29:10.161] test_mongocrypt_new (test.test_binding.TestBinding) ... ok
 [2020/09/03 21:29:10.162] test_mongocrypt_status_new (test.test_binding.TestBinding) ... ok
 [2020/09/03 21:29:10.162] test_pymongocrypt_version (test.test_binding.TestBinding) ... ok
 [2020/09/03 21:29:10.389] test_data_key_creation (test.test_mongocrypt.TestExplicitEncryption) ... ok
 [2020/09/03 21:29:10.397] test_encrypt_decrypt (test.test_mongocrypt.TestExplicitEncryption) ... ok
 [2020/09/03 21:29:10.403] test_encrypt_decrypt_key_alt_name (test.test_mongocrypt.TestExplicitEncryption) ... ok
 [2020/09/03 21:29:10.410] test_decrypt (test.test_mongocrypt.TestMongoCrypt) ... ok
 [2020/09/03 21:29:10.419] test_encrypt (test.test_mongocrypt.TestMongoCrypt) ... ok
 [2020/09/03 21:29:10.419] test_mongocrypt (test.test_mongocrypt.TestMongoCrypt) ... ok
 [2020/09/03 21:29:10.422] test_mongocrypt_validation (test.test_mongocrypt.TestMongoCrypt) ... ok
 [2020/09/03 21:29:10.423] test_mongocrypt_binary_in (test.test_mongocrypt.TestMongoCryptBinary) ... ok
 [2020/09/03 21:29:10.423] test_mongocrypt_binary_out (test.test_mongocrypt.TestMongoCryptBinary) ... ok
 [2020/09/03 21:29:10.430] test_decrypt (test.test_mongocrypt.TestMongoCryptCallback) ... ok
 [2020/09/03 21:29:10.437] test_encrypt (test.test_mongocrypt.TestMongoCryptCallback) ... ok
 [2020/09/03 21:29:10.438] test_mongocrypt_options (test.test_mongocrypt.TestMongoCryptOptions) ... ok
 [2020/09/03 21:29:10.466] test_mongocrypt_options_validation (test.test_mongocrypt.TestMongoCryptOptions) ... ok
 [2020/09/03 21:29:10.466] ----------------------------------------------------------------------
 [2020/09/03 21:29:10.466] Ran 18 tests in 0.280s
 [2020/09/03 21:29:10.466] OK

Comment by Shane Harvey [ 02/Sep/20 ]

According to this issue: https://github.com/pyca/cryptography/issues/4274, the "undefined symbol: d2i_DHxparams" error will occur when you "compiled cryptography against one version of OpenSSL (> 1.1.0) but linked it against an older version."

Interesting to note that this error only happens on pypy3 and not on pypy 2. It probably has something to with the fact that cryptography does not ship pre-built wheels for pypy.

Some initial theories:

  • This might be fixed by upgrading from to PyPy 7.1.1-beta0 to PyPy 7.3.1 in BUILD-11602.
  • cryptography 3.1 is the latest release but we're still using 2.9.2 because we cap our dependency. Upgrading to 3.X might solve the issue.
  • We may need to set CFLAGS and LDFLAGS when installing cryptography.
  • If none of the above options work, then we should install cryptography as a static wheel according to: https://cryptography.io/en/3.1/installation/#static-wheels
Generated at Thu Feb 08 09:08:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.