Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2411

Support the Azure VM-assigned Managed Identity for Automatic KMS Credentials

XMLWordPrintableJSON

    • Needed
    • Hide

      Implementation

      libmongocrypt 1.6.0 or higher is required. Binaries for 1.6.0 are available on the upload-all task.

      The spec changes introduce another method of obtaining KMS credentials automatically, much like with GCP and AWS:

      • When kmsProviders contains an empty azure property, it indicates a request for automatic Azure credentials.
      • To obtain credentials, issue an HTTP request to the Azure Instance Metadata Service (IMDS).
      • IMDS will issue an accessToken that can be used to query the Azure Key Vault (if the instance has sufficient permissions).
      • Additionally, this version of auto-KMS credentials institutes a token caching requirement.

      The associated spec changes are specified here: https://github.com/mongodb/specifications/commit/d6b8cce6abb3b8e1a0b8f1dc7ee737e18322cfce

      The initial implementation for the C driver is here: https://github.com/mongodb/mongo-c-driver/commit/686bff81f565f93db83d99902ce1c3a6f89922c7

      Mock server tests

      Mock server tests specified here:
      https://github.com/mongodb/specifications/commit/e780e91d708fe9c004a0b0023387baa850282881

      The mock server is available here: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/csfle/fake_azure.py

      Please see https://github.com/mongodb/mongo-c-driver/commit/671a15154f0dd0e4af3c8df2ac08dfe4acf01795#diff-d353a218f6d4ac77dfb35cc757a96af121a9ce1d3cf7b01535fa23e6d0c58016R98 for a reference implementation of the mock server tests in C.

      Integration tests

      Integration tests are specified here:
      https://github.com/mongodb/specifications/commit/cf778cb8add04c0c6d8f366e6352f3d0ac9c1694

      Scripts in the drivers-evergreen-tools .evergreen/csfle/azurekms directory may be used to create the temporary Azure Virtual Machine. Get credentials from DRIVERS-2411 Test Credentials.

      To test, add an Evergreen task group to do the following:

      • Create an Azure VM instance in a setup_group.
      • Destroy the Azure VM instance in a teardown_group. Using a teardown_group will destroy the instance if the task fails.

      Add a task in the task group to do the following:

      • Build and copy files to the remote Azure VM.
      • Install necessary dependencies on the remote Azure VM instance.
      • Run the test remotely.

      Please see https://github.com/mongodb/mongo-c-driver/pull/1124 and https://github.com/mongodb/mongo-c-driver/pull/1234/ for a reference implementation of the integration tests in C.

      It may be helpful to refer to driver tests for MONGODB-AWS ECS. The ECS tests perform a similar flow (copying and running a test on a remote ECS instance).

      Show
      Implementation libmongocrypt 1.6.0 or higher is required. Binaries for 1.6.0 are available on the upload-all task . The spec changes introduce another method of obtaining KMS credentials automatically, much like with GCP and AWS: When kmsProviders contains an empty azure property, it indicates a request for automatic Azure credentials. To obtain credentials, issue an HTTP request to the Azure Instance Metadata Service (IMDS). IMDS will issue an accessToken that can be used to query the Azure Key Vault (if the instance has sufficient permissions). Additionally, this version of auto-KMS credentials institutes a token caching requirement. The associated spec changes are specified here: https://github.com/mongodb/specifications/commit/d6b8cce6abb3b8e1a0b8f1dc7ee737e18322cfce The initial implementation for the C driver is here: https://github.com/mongodb/mongo-c-driver/commit/686bff81f565f93db83d99902ce1c3a6f89922c7 Mock server tests Mock server tests specified here: https://github.com/mongodb/specifications/commit/e780e91d708fe9c004a0b0023387baa850282881 The mock server is available here: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/csfle/fake_azure.py Please see https://github.com/mongodb/mongo-c-driver/commit/671a15154f0dd0e4af3c8df2ac08dfe4acf01795#diff-d353a218f6d4ac77dfb35cc757a96af121a9ce1d3cf7b01535fa23e6d0c58016R98 for a reference implementation of the mock server tests in C. Integration tests Integration tests are specified here: https://github.com/mongodb/specifications/commit/cf778cb8add04c0c6d8f366e6352f3d0ac9c1694 Scripts in the drivers-evergreen-tools .evergreen/csfle/azurekms directory may be used to create the temporary Azure Virtual Machine. Get credentials from DRIVERS-2411 Test Credentials . To test, add an Evergreen task group to do the following: Create an Azure VM instance in a setup_group . Destroy the Azure VM instance in a teardown_group . Using a teardown_group will destroy the instance if the task fails. Add a task in the task group to do the following: Build and copy files to the remote Azure VM. Install necessary dependencies on the remote Azure VM instance. Run the test remotely. Please see https://github.com/mongodb/mongo-c-driver/pull/1124 and https://github.com/mongodb/mongo-c-driver/pull/1234/  for a reference implementation of the integration tests in C. It may be helpful to refer to driver tests for MONGODB-AWS ECS . The ECS tests perform a similar flow (copying and running a test on a remote ECS instance).
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4454 Fixed 1.24.0
      CXX-2565 Works as Designed 3.8.0
      CSHARP-4294 Fixed 2.19.0
      GODRIVER-2521 Fixed 1.12.0
      JAVA-4706 Fixed 4.8.0
      NODE-4537 Fixed 5.2.0, mongodb-client-encryption-2.7.1
      MOTOR-1017 Duplicate
      PYTHON-3396 Fixed pymongocrypt-1.4, 4.3.3
      PHPLIB-938 Done
      RUBY-3088 Fixed 2.19.0
      RUST-1442 Fixed 2.6.0
      SWIFT-1625 Won't Do
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4454 Fixed 1.24.0 CXX-2565 Works as Designed 3.8.0 CSHARP-4294 Fixed 2.19.0 GODRIVER-2521 Fixed 1.12.0 JAVA-4706 Fixed 4.8.0 NODE-4537 Fixed 5.2.0, mongodb-client-encryption-2.7.1 MOTOR-1017 Duplicate PYTHON-3396 Fixed pymongocrypt-1.4, 4.3.3 PHPLIB-938 Done RUBY-3088 Fixed 2.19.0 RUST-1442 Fixed 2.6.0 SWIFT-1625 Won't Do

      Summary

      At present, using Azure Key Vault for KMS requires a tenant ID, client ID, and client secret (password) in order to obtain an OAuth2 token to subsequently query the Azure Key Vault for key management.

      Azure VMs are automatically assigned a Managed Identity which allows the VM to obtain an OAuth2 token by querying a private metadata HTTP endpoint without any special credentials. The token obtained can then be used to request access to Azure resources, including the Key Vault, if applicable.

      We have added support for automatic KMS credentials for AWS (DRIVERS-2280) and GCP (DRIVERS-2377). The Azure VM Managed Identity can be used for the same purpose, removing the need for the client to provide credentials if they are running within an Azure VM that has the appropriate access to the Key Vault:

      1. Allow the user to omit Azure credentials as part of kmsProviders, and request that credentials be obtained automatically from the environment.
      2. Attempt to load OAuth2 token from the Azure metadata endpoint. If we are running within an Azure VM, this token corresponds to the managed identity of that VM.
      3. Use the token from (2) as the KMS credential for client encryption.

      Motivation

      Who is the affected end user?

      All end users of client-side encryption that wish to use Azure Key Vault as their KMS provider, and are running their client within an Azure VM.

      How does this affect the end user?

      Supporting the VM's managed identity alleviates the need to manage a separate set of credentials for their client application.

      How likely is it that this problem or use case will occur?

      As Azure KMS users are likely running within an Azure VM, most Azure users will likely find benefit in delegating the credential management to the Azure platform.

      If the problem does occur, what are the consequences and how severe are they?

      Managing and securing additional credentials creates an additional security concern and barrier to adoption for client-side encryption, whereas requesting the client driver to automatically work with the Azure host is simpler, more secure, and less error-prone.

      Is this issue urgent?

      The first mention of using Azure Managed Identities appears in December of 2020. The recent addition of automatic credentials for AWS and GCP makes this change prudent to match platform support.

      Is this ticket required by a downstream team?

      No

      Is this ticket only for tests?

      No

            Assignee:
            colby.pike@mongodb.com Colby Pike
            Reporter:
            colby.pike@mongodb.com Colby Pike
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: