[DRIVERS-2355] Test with consistent versions of crypt_shared and server Created: 14/Jun/22  Updated: 21/Aug/23

Status: Implementing
Project: Drivers
Component/s: Client Side Encryption, Evergreen Tools
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: Jeremy Mikola Assignee: Kevin Albertson
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by PYTHON-3296 Tests should use the same crypt_share... Closed
Issue split
split to PHPC-2135 Test with consistent versions of cryp... Closed
split to CXX-2545 Test with consistent versions of cryp... Backlog
split to JAVA-4678 Test with consistent versions of cryp... Closed
split to CDRIVER-4427 Test with consistent versions of cryp... Closed
split to CSHARP-4254 Test with consistent versions of cryp... Closed
split to GODRIVER-2492 Test with consistent versions of cryp... Closed
split to MOTOR-994 Test with consistent versions of cryp... Closed
split to NODE-4422 Test with consistent versions of cryp... Closed
split to PHPLIB-911 Test with consistent versions of cryp... Closed
split to PYTHON-3355 Test with consistent versions of cryp... Closed
split to RUBY-3051 Test with consistent versions of cryp... Closed
split to RUST-1408 Test with consistent versions of cryp... Closed
Related
related to DRIVERS-1950 FLE 1.0 Shared Library Closed
related to GODRIVER-2437 Revert pin to crypt_shared "latest" r... Closed
Driver Changes: Needed
Downstream Changes Summary:

Update CSFLE tests to use the same version of crypt_shared as the server.
Update CSFLE tests to use mongocryptd when using server version < 6.

Please see https://github.com/mongodb/specifications/commit/9a0a385dc897319ecc10dd3dd19bc9d8e4c10de7 for the specification clarification.

The CRYPT_SHARED_LIB_PATH expansion is made available from run-orchestration.sh as part of https://github.com/mongodb-labs/drivers-evergreen-tools/commit/5af4bfe0f18ece58d8492436b115c24a4167f1a3

Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4427 Fixed 1.23.2
CXX-2545 Backlog
CSHARP-4254 Fixed 2.18.0
GODRIVER-2492 Fixed 1.11.0
JAVA-4678 Fixed 4.9.0
NODE-4422 Fixed 4.9.0
MOTOR-994 Duplicate
PYTHON-3355 Fixed 4.3
PHPLIB-911 Fixed 1.13.1
RUBY-3051 Works as Designed
RUST-1408 Fixed 2.4.0
SWIFT-1602 Won't Do
PHPC-2135 Fixed 1.14.1

 Description   

Summary

Drivers generally provision server versions in Evergreen using the download-mongodb.sh script, which internally keeps a map of Major.Minor server versions to specific packages for various operating systems.

The mongodl.py originally introduced in b233de2 operates very differently by downloading full.json from mongodb.com. Although it does have aliases for "latest" and "latest-stable", it's not clear if those are consistent with the aliases we have in download-mongodb.sh.

Two ideas came to mind (I'm open to others):

  • mongodl.py looks like a generally useful tool. If we can find some way to share a consistent mapping of Major.Minor aliases between it and download-mongodb.sh, that would help.
  • Alternatively, we need some way to extract the exact server version fetched by download-mongodb.sh. I looked into this briefly by writing a sed one-liner to parse mongod --version but ran into trouble finding the extracted binaries. AFAIK, they are fed to mongo-orchestration but don't end up in $PATH.

This came up in mongodb/mongo-python-driver#961 for PYTHON-3296.

On a separate note, I'm not sure if mongodl.py caches full.json or downloads it on each invocation. That file is quite large, so this does add overhead to the minimal mappings we maintain for download-mongodb.sh. Perhaps a separate tool to query full.json and prepare a mapping list similar to what we have in download-mongodb.sh would be more useful, if we decide to go the mapping route.

Motivation

Who is the affected end user?

Driver developers.

How does this affect the end user?

Difficulty ensuring consistent versions are used for the MongoDB server and crypt_shared.

Is this issue urgent?

Yes. This is relevant for crypt_shared CI testing.

Is this ticket required by a downstream team?

May be relevant for shell testing, unless they're using entirely different tooling.

Is this ticket only for tests?

Yes.



 Comments   
Comment by Githook User [ 15/Jul/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2355 check for bin directory in Windows (#214)

  • fix windows64 links

use zip, not tgz

Comment by Githook User [ 14/Jul/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2355 Clarify that driver tests < 6.0 should use mongocryptd (#1275)
Branch: master
https://github.com/mongodb/specifications/commit/9a0a385dc897319ecc10dd3dd19bc9d8e4c10de7

Comment by Githook User [ 14/Jul/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2355 add crypt_shared download function to download-mongodb.sh (#213)

  • only return shared lib if exists for version
Generated at Thu Feb 08 08:25:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.