[DRIVERS-2011] On-demand callback for AWS credentials Created: 19/Dec/21  Updated: 19/Dec/22  Resolved: 27/Sep/22

Status: Closed
Project: Drivers
Component/s: Authentication
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Kevin Albertson Assignee: Unassigned
Resolution: Won't Do Votes: 6
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by GODRIVER-2241 AWS credential refreshing Closed
Duplicate
is duplicated by DRIVERS-1941 Add MONGODB-AWS Support for EKS Servi... Closed
Issue split
split to CDRIVER-4467 On-demand callback for AWS credentials Closed
split to CSHARP-4033 On-demand callback for AWS credentials Closed
split to CXX-2437 On-demand callback for AWS credentials Closed
split to GODRIVER-2293 On-demand callback for AWS credentials Closed
split to JAVA-4464 On-demand callback for AWS credentials Closed
split to MOTOR-877 On-demand callback for AWS credentials Closed
split to NODE-3934 On-demand callback for AWS credentials Closed
split to PHPC-2048 On-demand callback for AWS credentials Closed
split to PYTHON-3091 On-demand callback for AWS credentials Closed
split to RUBY-2890 On-demand callback for AWS credentials Closed
split to RUST-1164 On-demand callback for AWS credentials Closed
Related
related to JAVA-4310 AWS credential refreshing Closed
related to DRIVERS-2333 Cache AWS Credentials Where Possible Closed
is related to DRIVERS-2179 Add support for updating expired AWS ... Closed
is related to DRIVERS-1746 Add native support for AWS IAM Roles ... Closed
is related to DRIVERS-2280 Obtain AWS credentials for CSFLE in t... Closed
is related to RUBY-2512 Support AWS authentication with tempo... Closed
is related to DRIVERS-1941 Add MONGODB-AWS Support for EKS Servi... Closed
Driver Changes: Not Needed
Case:
Engineering Lead: Kevin Albertson Kevin Albertson
Product Manager: Rachelle Palmer Rachelle Palmer
Program Manager: Esha Bhargava Esha Bhargava
Driver Compliance:
Key Status/Resolution FixVersion
CXX-2437 Won't Do
CSHARP-4033 Won't Do
GODRIVER-2293 Won't Do
JAVA-4464 Won't Do
NODE-3934 Won't Do
MOTOR-877 Duplicate
PYTHON-3091 Won't Do
PHPC-2048 Won't Do
RUBY-2890 Won't Do
RUST-1164 Won't Do
SWIFT-1471 Won't Do
CDRIVER-4467 Won't Do

 Description   

Summary

Add a callback to supply AWS credentials on-demand.

This is a request to specify an API equivalent to JAVA-4310.

Motivation

The Driver Authentication specification describes four ways of obtaining credentials for the MONGODB-AWS authentication mechanism.

1. From the URI username, password, and options.
2. From environment variables.
3. From querying an endpoint for credentials in ECS.
4. From querying an endpoint for credentials in EC2.

A callback helps with these use cases:
1. Caching credentials. In (3) and (4) the endpoint is queried each time a connection handshake results in authentication. This may result in hitting rate limits.
2. Avoid session token expiration. The AWS session token set in (1) or (2) may be temporary and can expire. A callback enables passing and refreshing credentials in environments like EKS with assigned IAM roles.
3. Obtain credentials in EKS environments.

Who is the affected end user?

Users authenticating with MONGODB-AWS.

How does this affect the end user?

I do not know if there is a workaround for hitting rate limits in ECS. Users may be blocked.

If AWS credentials are passed through URI options, credentials may expire and result in failed authentication attempts. The workaround is requires recreating a MongoClient. The workaround is undesirable.

Authenticating with AWS in EKS requires an undesirable workaround by passing URI options.

How likely is it that this problem or use case will occur?

Likely for EKS users.

If the problem does occur, what are the consequences and how severe are they?

Hitting rate limits may result in temporary unavailability.

Handling credential expiration is less severe. The workaround is to recreate the MongoClient. It is an undesirable workaround.

Authenticating with AWS in EKS is less severe. The workaround is to recreate the MongoClient. It is an undesirable workaround.

Is this issue urgent?

No.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

No.



 Comments   
Comment by Martin Bajana [ 27/Sep/22 ]

We prioritized DRIVERS-2333 to handle the rate-limiting case and DRIVERS-1746 to handle IAM roles for service accounts.

Comment by Steve Silvester [ 11/Jul/22 ]

Yes, I agree

Comment by Steve Silvester [ 15/Jun/22 ]

Hi qingyang.hu@mongodb.com, the concerns you raise are part of the reasons we have put this ticket on the backlog, and are instead pursuing other options for improving our support of AWS.

Comment by Qingyang Hu [ 14/Jun/22 ]

The GO driver currently takes static strings for AWS token, access key, and key ID. To support callback without breaking changes, we need to introduce additional methods aside from the existing setting method. Therefore, we look for a priority among different setting methods in the specification. i.e., if static values and callbacks are given simultaneously, which one will be applied?

Comment by Steve Silvester [ 25/May/22 ]

It is public now, thanks for the heads up rodolfo.matos@forto.com!

Comment by Rodolfo Matos [ 24/May/22 ]

Hey Steve, the https://jira.mongodb.org/browse/DRIVERS-1746 ticket is not publicly visible. It is possible to make it publicly available?

Comment by Steve Silvester [ 24/May/22 ]

After further discussion, we are not going to implement this feature at this time. 

Instead, we will prioritize DRIVERS-2333(https://jira.mongodb.org/browse/DRIVERS-2333] to handle the rate-limiting case and DRIVERS-1746 to handle IAM roles for service accounts.

The motivating factors are the fact that it would be confusing to have authorization refresh only available for one authentication type, and it would be currently impossible for us to provide warnings or errors if the server pool ended up in a state of mixed privileges. The changes in DRIVERS-2333 and DRIVERS-1746 are small in scope and transparent to the user.

Comment by Steve Silvester [ 17/May/22 ]

Good catch daniel.lamando@forto.com, thank you!  After doing some more digging today, here are my current thoughts:

In general it seems like not providing username and password seems like the the best approach in environments where we can get that auth for the user.  This includes EKS (which uses EC2 or Fargate nodes), EC2, ECS, and Lambda.  In all cases we can get the credentials using link-local URI endpoints or from environment variables.  These scenarios are already covered in the drivers specification

For cases where this approach does not work, we have the problem that the user could inadvertently change to a new role with differing privileges.    As a result, they may have access to some databases and not others, depending on which socket connection they are using for a given operation.  Ideally we could verify that the ARN role does not change, but the credentials given do not allow us to inspect the ARN.  We cannot guarantee that we can use the equivalent of aws sts get-caller-identity from the client.  However, the server will get the ARN back from the request it makes during authentication.

This means that clients should require an AWS_ROLE_ARN mechanism property that may be enforced by the server (at some future date), when using a dynamic callback.   

The new proposed API is a function that accepts a MongoCredential and returns a MongoCredential, which is only supported for AWS auth. The returned MongoCredential must have the required AWS_ROLE_ARN mechanism property set.

Comment by Daniel Lamando [ 16/May/22 ]

Thanks for the public update, Steve!

From my reading of this, the callback would return only a string for an updated AWS_SESSION_TOKEN. I'd like to point out that every call to the AssumeRole or AssumeRoleWithWebIdentity APIs returns a brand new set of credentials, where all three fields are different every time, not just AWS_SESSION_TOKEN.

I understand the desire to keep this callback feature limited in scope. However it seems that accepting a fresh MongoCredential object (or similar) is unavoidable in the context of temporary AWS credentials.

Also,

eventually OIDC when we support it

Yes please :^)

Comment by Steve Silvester [ 16/May/22 ]

Current thoughts on this as I prepare to write the spec:

We should only use this mechanism for token refresh, for AWS and eventually OIDC when we support it. It would be too easy to misuse the feature otherwise. It should be given as a parameter the MongoClient constructor called dynamic_credential_callback that accepts the current MongoCredential object and returns a string representing the new token.

If the callback is given, we will use it to fetch the current token, ignoring any static value given for AWS_SESSION_TOKEN.

If the callback is provided and the auth mechanism is not MONGODB-AWS, the driver should raise an exception.

Comment by Bernie Hackett [ 27/Apr/22 ]

kevincent@tradestation.com, our "quarters" follow MongoDB Inc. fiscal calendar. FY23Q2 starts May 2022.

Comment by Kekoa Vincent [ 27/Apr/22 ]

I understand the desire to improve the design/architecture of the driver, but EKS users are inadequately served compared to the lambda or ECS use cases. Do we have to wait a year to get the same level of integration support? As currently tagged it looks like FY23 is when we can expect it.

I would kindly ask for a reconsideration of priority or perhaps allow for the less elegant solution so we can use an increasingly popular EKS runtime environment without resorting to our own driver hacks to leverage passwordless authentication. Please see my original post at https://jira.mongodb.org/browse/DRIVERS-1941 which was closed in favor of this.

If the callback implementation is going to take another year, how long would it take to get something that we can use in official drivers?

Comment by Srikar Yekollu [ 15/Apr/22 ]

@Rodolfo, we would be interested in this. We already did something like this for python. Looking for a similar solution with the node driver.

Comment by Rodolfo Matos [ 07/Apr/22 ]

Hello everyone, currently we are monkeypatching the MongoDBAWS class to get around this limitation. Would you be interested in a pull request upstream that implements the requirements defined in this ticket?

Generated at Thu Feb 08 08:24:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.