[CDRIVER-4454] Support the Azure VM-assigned Managed Identity for Automatic KMS Credentials Created: 11/Aug/22  Updated: 28/Oct/23  Resolved: 18/Oct/22

Status: Closed
Project: C Driver
Component/s: Client Side Encryption
Affects Version/s: None
Fix Version/s: 1.24.0

Type: Improvement Priority: Unknown
Reporter: PM Bot Assignee: Colby Pike
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Issue split
split from DRIVERS-2411 Support the Azure VM-assigned Managed... Closed
Related
related to CDRIVER-4698 Coverity analysis defect 133796: Remo... Closed
is related to PHPLIB-938 Support the Azure VM-assigned Managed... Closed
Quarter: FY23Q3
Upstream Changes Summary:

DRIVERS-2411:
Implementation

libmongocrypt 1.6.0 or higher is required. Binaries for 1.6.0 are available on the upload-all task.

The spec changes introduce another method of obtaining KMS credentials automatically, much like with GCP and AWS:

  • When kmsProviders contains an empty azure property, it indicates a request for automatic Azure credentials.
  • To obtain credentials, issue an HTTP request to the Azure Instance Metadata Service (IMDS).
  • IMDS will issue an accessToken that can be used to query the Azure Key Vault (if the instance has sufficient permissions).
  • Additionally, this version of auto-KMS credentials institutes a token caching requirement.

The associated spec changes are specified here: https://github.com/mongodb/specifications/commit/d6b8cce6abb3b8e1a0b8f1dc7ee737e18322cfce

The initial implementation for the C driver is here: https://github.com/mongodb/mongo-c-driver/commit/686bff81f565f93db83d99902ce1c3a6f89922c7

Mock server tests

Mock server tests specified here:
https://github.com/mongodb/specifications/commit/e780e91d708fe9c004a0b0023387baa850282881

The mock server is available here: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/csfle/fake_azure.py

Please see https://github.com/mongodb/mongo-c-driver/commit/671a15154f0dd0e4af3c8df2ac08dfe4acf01795#diff-d353a218f6d4ac77dfb35cc757a96af121a9ce1d3cf7b01535fa23e6d0c58016R98 for a reference implementation of the mock server tests in C.

Integration tests

Integration tests are specified here:
https://github.com/mongodb/specifications/commit/cf778cb8add04c0c6d8f366e6352f3d0ac9c1694

Scripts in the drivers-evergreen-tools .evergreen/csfle/azurekms directory may be used to create the temporary Azure Virtual Machine. Get credentials from DRIVERS-2411 Test Credentials.

To test, add an Evergreen task group to do the following:

  • Create an Azure VM instance in a setup_group.
  • Destroy the Azure VM instance in a teardown_group. Using a teardown_group will destroy the instance if the task fails.

Add a task in the task group to do the following:

  • Build and copy files to the remote Azure VM.
  • Install necessary dependencies on the remote Azure VM instance.
  • Run the test remotely.

Please see https://github.com/mongodb/mongo-c-driver/pull/1124 and https://github.com/mongodb/mongo-c-driver/pull/1234/ for a reference implementation of the integration tests in C.

It may be helpful to refer to driver tests for MONGODB-AWS ECS. The ECS tests perform a similar flow (copying and running a test on a remote ECS instance).


 Description   

This ticket was split from DRIVERS-2411, please see that ticket for a detailed description.



 Comments   
Comment by Githook User [ 01/Dec/22 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: CDRIVER-4454 use consistent include path for mcd-time.h (#1154)

This include was inconsistent with all others in libmongoc. The path is necessary for compilation in Visual Studio when "src/libmongoc/src/libmongoc/src" is used as an include path.

This was originally introduced in mongodb/mongo-c-driver@686bff81f565f93db83d99902ce1c3a6f89922c7
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/a60ac70449ab621ffb66df63e093ee2ab041d46b

Comment by Githook User [ 30/Nov/22 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: CDRIVER-4454: Use consistent include path for mcd-time.h

This include was inconsistent with all others in libmongoc. The path is necessary for compilation in Visual Studio when "src/libmongoc/src/libmongoc/src" is used as an include path.

This was originally introduced in mongodb/mongo-c-driver@686bff81f565f93db83d99902ce1c3a6f89922c7
Branch: cdriver-4454-fix-include-path
https://github.com/mongodb/mongo-c-driver/commit/9f260ece0a07a2c502c0ba9cf1569b8b78605a51

Comment by Githook User [ 30/Nov/22 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: CDRIVER-4454: Use consistent include path for mcd-time.h

This include was inconsistent with all others in libmongoc. The path is necessary for compilation in Visual Studio when "src/libmongoc/src/libmongoc/src" is used as an include path.

This was originally introduced in mongodb/mongo-c-driver@686bff81f565f93db83d99902ce1c3a6f89922c7 for
Branch: cdriver-4454-fix-include-path
https://github.com/mongodb/mongo-c-driver/commit/44c026db830b3ca2258a16541f1f19466171cca2

Comment by Githook User [ 03/Nov/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4454 add integration tests for Azure KMS (#1124)
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/4a6de7764e0a7a2c89a98627ae155dccf841a354

Comment by Githook User [ 18/Oct/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4454 fix error return and HTTP version check (#1123)

Comment by Githook User [ 16/Sep/22 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4454 fix VS 2013 compile (#1106)
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/605d403bea50e2a7235c867e62092b8afd48afa7

Comment by Githook User [ 14/Sep/22 ]

Author:

{'name': 'vector-of-bool', 'email': 'vectorofbool@gmail.com', 'username': 'vector-of-bool'}

Message: CDRIVER-4454 Azure auto-KMS testing (#1104)

  • Move API for IMDS HTTP, support host and port overrides
  • Live tests for Azure IMDS requests
  • Add a very simple mock IMDS server based on Bottle
  • New error types for Azure
  • A simple timer abstraction
  • HTTP fixes:
  • A timeout during a partial read is still an error.
  • Prevent a slow server from trickling data and causing an eternal wait
    (Keep track of time while reading)
  • Reject very large HTTP responses.
  • Test cases by prompting server misbehavior from the client
  • Update errors listings
Comment by Githook User [ 10/Sep/22 ]

Author:

{'name': 'vector-of-bool', 'email': 'vectorofbool@gmail.com', 'username': 'vector-of-bool'}

Message: CDRIVER-4454 Some HTTP Fixes (#1103)

Comment by Githook User [ 09/Sep/22 ]

Author:

{'name': 'vector-of-bool', 'email': 'vectorofbool@gmail.com', 'username': 'vector-of-bool'}

Message: CDRIVER-4454 Automatic Azure KMS Credentials (#1097)

Generated at Wed Feb 07 21:20:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.