[DRIVERS-2731] CSFLE/QE Support for more than 1 KMS provider per type Created: 22/Sep/23  Updated: 30/Jan/24

Status: Implementing
Project: Drivers
Component/s: Client Side Encryption
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Cynthia Braund (Inactive) Assignee: Kevin Albertson
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on MONGOCRYPT-605 Support for more than 1 KMS provider ... Closed
Issue split
split to GODRIVER-3076 CSFLE/QE Support for more than 1 KMS ... Needs Triage
split to NODE-5801 CSFLE/QE Support for more than 1 KMS ... Needs Triage
split to PHPLIB-1328 CSFLE/QE Support for more than 1 KMS ... Needs Triage
split to CSHARP-4892 CSFLE/QE Support for more than 1 KMS ... Backlog
split to CXX-2800 CSFLE/QE Support for more than 1 KMS ... Backlog
split to JAVA-5275 CSFLE/QE Support for more than 1 KMS ... Backlog
split to MOTOR-1228 CSFLE/QE Support for more than 1 KMS ... Backlog
split to RUBY-3363 CSFLE/QE Support for more than 1 KMS ... Backlog
split to RUST-1813 CSFLE/QE Support for more than 1 KMS ... Backlog
split to CDRIVER-4801 CSFLE/QE Support for more than 1 KMS ... Closed
split to PYTHON-4112 CSFLE/QE Support for more than 1 KMS ... Closed
Driver Changes: Needed
Quarter: FY24Q4
Downstream Changes Summary:

Summary

Driver Changes

Some drivers may need API changes to accept an arbitrary string where a KMS provider is accepted: kmsProviders, KMSProvidersTLSOptions, ClientEncryption.createDataKey(), and RewrapManyDataKeyOpts.provider

Can current drivers accept arbitrary strings for KMS identifier? suggests Node and Rust will need API changes.

Drivers may need changes to support named KMS providers in the KMSProvidersTLSOptions map.

Test Changes

Specification tests are added. This introduces use of the encrypt and decrypt operations in the unified test format.

The Unified Test Format schema 1.18 is added to allow patternProperties in kmsProviders.

Tests refer to additional KMS providers: local:name1, aws:name1, gcp:name1, azure:name1, and kmip:name1.

The name1 KMS providers may be configured exactly as the unnamed KMS providers. I.e. aws:name1 is configured the same as aws.

To test configuring two KMS providers of the same type referring to distinct credentials, two more test KMS providers are defined: local:name2 and aws:name2.

Test credentials for aws:name2 are available in AWS Secrets Manager under drivers/csfle. The aws:name2 account credentials are in FLE_AWS_SECRET2 and FLE_AWS_KEY2. See https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets for more background on how the secrets are managed.

Prose Test 11 (KMS TLS Options Tests) is extended to test named KMS providers.

References

https://github.com/mongodb/specifications/pull/1492 includes the specification change and tests.

https://github.com/mongodb/mongo-c-driver/pull/1509 is a reference implementation in the C driver.

Engineering Lead: Kevin Albertson Kevin Albertson
Program Manager: Esha Bhargava Esha Bhargava
Start date:
Scope Cost Estimate: 0
Cost to Date: 0
Final Cost Estimate: 0
Cost Threshold %: 100
Detailed Project Statuses:

2024-01-19:

Status update:

  • libmongocrypt implementation merged
  • Specification changes in review.
  • C driver implementation in review.

2024-01-08:

Status update:

  • libmongocrypt implementation in review.
  • Specification updates in progress.

Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4801 Fixed 1.26.0
CXX-2800 Backlog
CSHARP-4892 Backlog
GODRIVER-3076 Needs Triage
JAVA-5275 Backlog
NODE-5801 Needs Triage
MOTOR-1228 Backlog 3.4
PYTHON-4112 Fixed 4.7
PHPLIB-1328 Needs Triage
RUBY-3363 Backlog
RUST-1813 Backlog

 Description   

Summary

 
Currently the KMS provider allows for only one "provider" per type - so 1 AWS KMS account, 1 GCP Cloud KMS Account, 1 Azure Key Vault account, 1 KMIP account and 1 local key.
 
That worked well until we added in key rotation/key migration, where there could be the need to rotate/migrate from 1 account to another within the same provider type. Common use cases: # App is developed in dev environment and now the app is being pushed to Prod. Customer want to use migration to move from a dev AWS KMS account to a Prod AWS KMS Account.

  1. Customer uses the "local key" provider to specify an endpoint on their system that contains a key secured however they want. There is no way to rotate that key because a new key is considered a new key provider.
  2. Company A has been sold to Company B and needs to migrate from Company A's key provider to Company B's key provider and both key providers are of the same type.

 
The question then becomes how many providers are supported per type.  In these use cases only 2 providers are needed so it should be a minimum of 2.  Depending on the complexity of adding more providers per type, it would be nice to add support for at least 3 and up to something like 10, to keep it from getting too unwieldy.
 

Cast of Characters

Engineering Lead:
Document Author:
POCers:
Product Owner:
Program Manager:
Stakeholders:

Channels & Docs

Slack Channel

[Scope Document|some.url]

[Technical Design Document|some.url]



 Comments   
Comment by Githook User [ 30/Jan/24 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2731 support named KMS providers (#1492)

  • add unified test format schema 1.18 to extend `kmsProviders`
  • add CSE unified tests to Makefile
  • use schema 1.18 in Makefile
  • add spec test: namedKMS-createDataKey
  • add spec test: namedKMS-rewrapManyDataKey
  • add legacy spec test: namedKMS

To test automatic encryption for named KMS

  • add spec test: namedKMS-explicit
  • add `Case 6` to prose test that TLS options are applied to named providers
  • remove out-dated info

AWS and local are not the only KMS providers supported. Remove sentence. Specifics of KMS providers are described in the `kmsProviders` section.

  • add terms for KMS providers, KMS provider, KMS provider type, and KMS provider name
  • replace usage of KMSProviderName with KMSProvider

KMSProviderName was previously one of the strings "aws", "azure", "gcp", "local", or "kmip". The term "KMS provider" consists of the same set of strings, but also includes an optional name suffix. Use KMSProvider to match the term and avoid ambiguity with "KMS provider name".

  • note which KMS providers support automatic credentials
  • revise terminology in `masterKey`

Note that "aws" is a `KMS provider type`, not a `KMS provider`.

  • describe `KMSProviders` interface

May help readers unfamiliar with typescript interfaces.

  • update CHANGELOG, Last Modified, and Version
  • add test `rewrap from aws:name1 to aws:name2`
  • note location of test credentials
  • note requirement for libmongocrypt 1.9.0
  • add rationale for no on-demand KMS credentials support for named KMS
Comment by Ashni Mehta [ 29/Jan/24 ]

Prioritization note: Checked in with elizabeth.roytburd@mongodb.com – customers are asking but this is not critical re: .local. As per Kenn/Liz: would be nice to preannounce if we think this will coincide with GA.

Generated at Thu Feb 08 08:26:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.