Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2718

Enable use of native crypto in libmongocrypt bindings

XMLWordPrintableJSON

    • Needed
    • Hide

      Summary of necessary driver changes

      Package the crypto-enabled libmongocrypt on platforms where it is safe to do so: Mac and Windows. For Linux, continue to package the crypto-disabled (nocrypto) builds due to the issues with OpenSSL versioning (see MONGOCRYPT-112).

      Check if the loaded libmongocrypt supports native crypto by calling mongocrypt_is_crypto_available. Only register crypto callbacks if it returns false.

      Document that on Linux, high-performance use cases should install libmongocrypt on the system. If possible, ensure this install takes precedence over the packaged crypto-disabled libmongocrypt.

      Update (2024-04-30): Drivers adding an environment variable to specify a path to libmongocrypt are recommended to use the name LIBMONGOCRYPT_PATH (consistent with Ruby driver).

      Context for other referenced/linked tickets

      mongocrypt_is_crypto_available is added in MONGOCRYPT-616. Until it is released, libmongocrypt binaries can be downloaded from the upload-all task for the commit.

      I expect this work only applies to drivers that implement the crypto hooks (i.e. calling mongocrypt_setopt_crypto_hooks and friends). From this survey, this appears to be: C#, Java, Node.js, Python, Ruby.

      Show
      Summary of necessary driver changes Package the crypto-enabled libmongocrypt on platforms where it is safe to do so: Mac and Windows. For Linux, continue to package the crypto-disabled (nocrypto) builds due to the issues with OpenSSL versioning (see MONGOCRYPT-112 ). Check if the loaded libmongocrypt supports native crypto by calling mongocrypt_is_crypto_available . Only register crypto callbacks if it returns false. Document that on Linux, high-performance use cases should install libmongocrypt on the system. If possible, ensure this install takes precedence over the packaged crypto-disabled libmongocrypt. Update (2024-04-30): Drivers adding an environment variable to specify a path to libmongocrypt are recommended to use the name LIBMONGOCRYPT_PATH (consistent with Ruby driver). Context for other referenced/linked tickets mongocrypt_is_crypto_available is added in MONGOCRYPT-616 . Until it is released, libmongocrypt binaries can be downloaded from the upload-all task for the commit . I expect this work only applies to drivers that implement the crypto hooks (i.e. calling mongocrypt_setopt_crypto_hooks and friends). From this survey , this appears to be: C#, Java, Node.js, Python, Ruby.
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4821 Works as Designed
      CXX-2817 Works as Designed
      CSHARP-4944 Done 2.26.0
      GODRIVER-3112 Works as Designed
      JAVA-5306 Fixed 5.2.0
      NODE-5875 Fixed mongodb-client-encryption-6.1.0
      MOTOR-1242 Duplicate
      PYTHON-4181 Fixed pymongocrypt-1.9.0
      PHPLIB-1387 Works as Designed
      RUBY-3386 Backlog
      RUST-1835 Works as Designed
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4821 Works as Designed CXX-2817 Works as Designed CSHARP-4944 Done 2.26.0 GODRIVER-3112 Works as Designed JAVA-5306 Fixed 5.2.0 NODE-5875 Fixed mongodb-client-encryption-6.1.0 MOTOR-1242 Duplicate PYTHON-4181 Fixed pymongocrypt-1.9.0 PHPLIB-1387 Works as Designed RUBY-3386 Backlog RUST-1835 Works as Designed

      Summary

      Investigate options to enable use of native crypto in libmongocrypt bindings.

      I expect this will require a scope document and prototyping.

      Motivation

      By default, libmongocrypt implements crypto primitives (including AES, HMAC, SHA) by linking to a native crypto library (OpenSSL on Linux, CommonCrypto on macOS, Cryptography API on Windows).

      Alternatively: libmongocrypt can be configured without crypto using the CMake option DISABLE_NATIVE_CRYPTO. The "no crypto" version of libmongocrypt is used by driver languages that supply crypto callbacks. On last survey: Java, Python, C#, Ruby, Node, and Rust implement the crypto callbacks.

      MONGOCRYPT-112 describes original motivation of the crypto callbacks:

      For certain languages like Java and C# on Linux, we need to ship a binary but this binary cannot depend on the system OpenSSL.

      The mongodb-crypt JAR and pymongocrypt wheel include generic Linux binaries of libmongocrypt.

      Use of crypto callbacks may result in slower throughput. HELP-27047 identified a problematic workload: decrypting documents with 1500 encrypted values.

      A benchmark of bulk decryption showed the most improvement when using libmongocrypt’s native system crypto, rather than the callbacks in the Java bindings:

      Baseline: +264.5%
Using Java binding improvements: +173.19%
Using bulk callbacks and Java binding improvements: +146.0%
Using native crypto: +82.79%

      Some bindings provide an option to use a system install of libmongocrypt (example: PYMONGOCRYPT_LIB in pymongocrypt). It may be possible to extend this to other drivers.

      Who is the affected end user?

      Users of In-Use Encryption.

      How does this affect the end user?

      Proposed changes may improve throughput of encryption and decryption.

      How likely is it that this problem or use case will occur?

      Certain. This was reported in HELP-27047.

      If the problem does occur, what are the consequences and how severe are they?

      Performance concern.

      Is this issue urgent?

      No.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      No.

      Acceptance Criteria

      Expect improved throughput of encryption and decryption in impacted drivers.

            Assignee:
            Unassigned Unassigned
            Reporter:
            kevin.albertson@mongodb.com Kevin Albertson
            Kevin Albertson Kevin Albertson
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: