Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2377

Add support for GCP attached service accounts when using GCP KMS

XMLWordPrintableJSON

    • Needed
    • Hide

      Summary of required changes

      • Upgrade dependency on libmongocrypt to 1.6.0 or higher. Binaries for 1.6.0 are available on the upload-all task.
      • Call mongocrypt_setopt_use_need_kms_credentials_state to opt in to handling the new MONGOCRYPT_CTX_NEED_KMS_CREDENTIALS state.
      • Handle the new MONGOCRYPT_CTX_NEED_KMS_CREDENTIALS state. If the originally configured KMS providers have an empty gcp: {}, attempt to obtain GCP credentials by sending an HTTP request described in the specification. Pass the new credentials back with mongocrypt_ctx_provide_kms_providers.
      • Add an integration test with a Google Compute Engine (GCE) instance. Get credentials from DRIVERS-2377 test credentials.

      Additional background

      Please see https://github.com/mongodb/specifications/commit/847d9ba741201f9c9d1305831a9c60e8ab2a1544 for the specification change.

      Please see https://github.com/mongodb/mongo-go-driver/commit/91b240c6aab86680ed5e78746a5a5edcd408c237 for a reference implementation in Go.

      Consider using the mock server for local development to test the HTTP request to the Metadata Server.

      GCP access token is not cached. See the scope for rationale.

      Integration test

      Drivers are expected to run an integration test with a temporary Google Compute Engine instance. Scripts in the drivers-evergreen-tools .evergreen/csfle/gcpkms directory may be used.

      To test, add an Evergreen task group to do the following:

      • Create a GCE instance in a setup_group.
      • Destroy the GCE instance in a teardown_group. Using a teardown_group will destroy the instance if the task fails.

      Add a task in the task group to do the following:

      • Build and copy files to the remote GCE instance.
      • Install necessary dependencies on the remote GCE instance.
      • Run the test remotely.

      Please see https://github.com/mongodb/mongo-go-driver/commit/91b240c6aab86680ed5e78746a5a5edcd408c237#diff-2bc841e86ce96b7b422ae203fd8315d0b2a461956cecbe0e096420656fc3fb12R2248 for a reference implementation of the integration test in Go.

      It may be helpful to refer to driver tests for MONGODB-AWS ECS. The ECS tests perform a similar flow (copying and running a test on a remote ECS instance).

      Show
      Summary of required changes Upgrade dependency on libmongocrypt to 1.6.0 or higher. Binaries for 1.6.0 are available on the upload-all task . Call mongocrypt_setopt_use_need_kms_credentials_state to opt in to handling the new MONGOCRYPT_CTX_NEED_KMS_CREDENTIALS state. Handle the new MONGOCRYPT_CTX_NEED_KMS_CREDENTIALS state. If the originally configured KMS providers have an empty gcp: { }, attempt to obtain GCP credentials by sending an HTTP request described in the specification. Pass the new credentials back with mongocrypt_ctx_provide_kms_providers . Add an integration test with a Google Compute Engine (GCE) instance. Get credentials from DRIVERS-2377 test credentials . Additional background Please see https://github.com/mongodb/specifications/commit/847d9ba741201f9c9d1305831a9c60e8ab2a1544 for the specification change. Please see https://github.com/mongodb/mongo-go-driver/commit/91b240c6aab86680ed5e78746a5a5edcd408c237 for a reference implementation in Go. Consider using the mock server for local development to test the HTTP request to the Metadata Server. GCP access token is not cached. See the scope for rationale. Integration test Drivers are expected to run an integration test with a temporary Google Compute Engine instance. Scripts in the drivers-evergreen-tools .evergreen/csfle/gcpkms directory may be used. To test, add an Evergreen task group to do the following: Create a GCE instance in a setup_group . Destroy the GCE instance in a teardown_group . Using a teardown_group will destroy the instance if the task fails. Add a task in the task group to do the following: Build and copy files to the remote GCE instance. Install necessary dependencies on the remote GCE instance. Run the test remotely. Please see https://github.com/mongodb/mongo-go-driver/commit/91b240c6aab86680ed5e78746a5a5edcd408c237#diff-2bc841e86ce96b7b422ae203fd8315d0b2a461956cecbe0e096420656fc3fb12R2248 for a reference implementation of the integration test in Go. It may be helpful to refer to driver tests for MONGODB-AWS ECS . The ECS tests perform a similar flow (copying and running a test on a remote ECS instance).
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4435 Fixed 1.24.0
      CXX-2551 Works as Designed 3.8.0
      CSHARP-4266 Fixed 2.18.0
      GODRIVER-2501 Fixed 1.11.0
      JAVA-4685 Fixed 4.8.0
      NODE-4462 Fixed 5.1.0
      MOTOR-999 Won't Do
      PYTHON-3367 Fixed pymongocrypt-1.4, 4.3.3
      PHPLIB-917 Done
      RUBY-3062 Fixed 2.19.0
      RUST-1417 Done 2.6.0
      SWIFT-1608 Won't Do
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4435 Fixed 1.24.0 CXX-2551 Works as Designed 3.8.0 CSHARP-4266 Fixed 2.18.0 GODRIVER-2501 Fixed 1.11.0 JAVA-4685 Fixed 4.8.0 NODE-4462 Fixed 5.1.0 MOTOR-999 Won't Do PYTHON-3367 Fixed pymongocrypt-1.4, 4.3.3 PHPLIB-917 Done RUBY-3062 Fixed 2.19.0 RUST-1417 Done 2.6.0 SWIFT-1608 Won't Do

      We have a customer on GCP, who is trying to use GCP KMS for the CMK and we require a service account key, where an email and privateKey is provided for the service account, in order to use the GCP KMS API. They are using an attached service account according to GCP best practices which says "Use attached service accounts when possible. For applications deployed on Google Cloud that need to use a service account, attach the service account to the underlying compute resource. By attaching a service account, you enable the application to obtain tokens for the service account and to use these tokens to access Google Cloud APIs and resources." and it also says "Use service account keys only if there is no viable alternative". Google even displays the warning "Service account keys could pose a security risk if compromised" when creating a key for a service account. 

      The customer is requesting that we follow GCP best practices and when using an "attached" service account, that we skip/bypass the email and privateKey that is used for authentication and just access the GCP KMS API directly.

      For reference https://cloud.google.com/iam/docs/best-practices-for-using-and-managing-service-accounts#use-attached-service-accounts

      The customer is running on CloudRun and sometimes Compute Engine on GCP so use attached service accounts. Their service accounts do not have user accessible keys on them.

      The customer is using the MongoDB C#/.NET driver on Linux.

       During local development they use an environment variable GOOGLE_APPLICATION_CREDENTIALS that if you point it to a local service account file (purely for development purposes) that means the Google libraries behave exactly the same as when deployed in a attached service account environment.

            Assignee:
            kevin.albertson@mongodb.com Kevin Albertson
            Reporter:
            william.chow@mongodb.com William Chow (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: