[DRIVERS-2718] Enable use of native crypto in libmongocrypt bindings Created: 08/Sep/23  Updated: 31/Jan/24

Status: Implementing
Project: Drivers
Component/s: Client Side Encryption, Performance
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Kevin Albertson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on MONGOCRYPT-616 Add API to determine if built with na... Closed
Duplicate
is duplicated by DRIVERS-2717 Enable use of native crypto in libmon... Closed
Issue split
split to JAVA-5306 Enable use of native crypto in libmon... Backlog
split to NODE-5875 Enable use of native crypto in libmon... Needs Triage
split to RUBY-3386 Enable use of native crypto in libmon... Needs Triage
split to CSHARP-4944 Enable use of native crypto in libmon... Backlog
split to PYTHON-4181 [pymongocrypt] Enable use of native c... Backlog
split to CDRIVER-4821 Enable use of native crypto in libmon... Closed
split to CXX-2817 Enable use of native crypto in libmon... Closed
split to GODRIVER-3112 Enable use of native crypto in libmon... Closed
split to MOTOR-1242 Enable use of native crypto in libmon... Closed
split to PHPLIB-1387 Enable use of native crypto in libmon... Closed
split to RUST-1835 Enable use of native crypto in libmon... Closed
Related
is related to DRIVERS-2581 Benchmark and use `mongocrypt_binary_... Implementing
Driver Changes: Needed
Quarter: FY25Q1
Downstream Changes Summary:

Summary of necessary driver changes

Package the crypto-enabled libmongocrypt on platforms where it is safe to do so: Mac and Windows. For Linux, continue to package the crypto-disabled (nocrypto) builds due to the issues with OpenSSL versioning (see MONGOCRYPT-112).

Check if the loaded libmongocrypt supports native crypto by calling mongocrypt_is_crypto_available. Only register crypto callbacks if it returns false.

Document that on Linux, high-performance use cases should install libmongocrypt on the system. If possible, ensure this install takes precedence over the packaged crypto-disabled libmongocrypt.

Context for other referenced/linked tickets

mongocrypt_is_crypto_available is added in MONGOCRYPT-616. Until it is released, libmongocrypt binaries can be downloaded from the upload-all task for the commit.

I expect this work only applies to drivers that implement the crypto hooks (i.e. calling mongocrypt_setopt_crypto_hooks and friends). From this survey, this appears to be: C#, Java, Node.js, Python, Ruby.

Engineering Lead: Kevin Albertson Kevin Albertson
Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4821 Works as Designed
CXX-2817 Works as Designed
CSHARP-4944 Backlog
GODRIVER-3112 Works as Designed
JAVA-5306 Backlog
NODE-5875 Needs Triage
MOTOR-1242 Duplicate
PYTHON-4181 Backlog 4.7
PHPLIB-1387 Works as Designed
RUBY-3386 Needs Triage
RUST-1835 Works as Designed

 Description   

Summary

Investigate options to enable use of native crypto in libmongocrypt bindings.

I expect this will require a scope document and prototyping.

Motivation

By default, libmongocrypt implements crypto primitives (including AES, HMAC, SHA) by linking to a native crypto library (OpenSSL on Linux, CommonCrypto on macOS, Cryptography API on Windows).

Alternatively: libmongocrypt can be configured without crypto using the CMake option DISABLE_NATIVE_CRYPTO. The "no crypto" version of libmongocrypt is used by driver languages that supply crypto callbacks. On last survey: Java, Python, C#, Ruby, Node, and Rust implement the crypto callbacks.

MONGOCRYPT-112 describes original motivation of the crypto callbacks:

For certain languages like Java and C# on Linux, we need to ship a binary but this binary cannot depend on the system OpenSSL.

The mongodb-crypt JAR and pymongocrypt wheel include generic Linux binaries of libmongocrypt.

Use of crypto callbacks may result in slower throughput. HELP-27047 identified a problematic workload: decrypting documents with 1500 encrypted values.

A benchmark of bulk decryption showed the most improvement when using libmongocrypt’s native system crypto, rather than the callbacks in the Java bindings:

Baseline: +264.5%
Using Java binding improvements: +173.19%
Using bulk callbacks and Java binding improvements: +146.0%
Using native crypto: +82.79%

Some bindings provide an option to use a system install of libmongocrypt (example: PYMONGOCRYPT_LIB in pymongocrypt). It may be possible to extend this to other drivers.

Who is the affected end user?

Users of In-Use Encryption.

How does this affect the end user?

Proposed changes may improve throughput of encryption and decryption.

How likely is it that this problem or use case will occur?

Certain. This was reported in HELP-27047.

If the problem does occur, what are the consequences and how severe are they?

Performance concern.

Is this issue urgent?

No.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

No.

Acceptance Criteria

Expect improved throughput of encryption and decryption in impacted drivers.


Generated at Thu Feb 08 08:26:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.