[DRIVERS-1746] Add native support for AWS IAM Roles for service accounts, EKS in particular Created: 11/May/21  Updated: 29/Jan/24  Resolved: 29/Jan/24

Status: Closed
Project: Drivers
Component/s: Authentication
Fix Version/s: None

Type: New Feature Priority: Unknown
Reporter: Ross Lawley Assignee: Steve Silvester
Resolution: Done Votes: 9
Labels: Kubernetes, MDBW23, leads-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by JAVA-4118 Add support for EKS when using AWS Ia... Closed
Duplicate
Issue split
split to CDRIVER-4081 Add native support for AWS IAM Roles ... Closed
split to CSHARP-3740 Add native support for AWS IAM Roles ... Closed
split to CXX-2307 Add native support for AWS IAM Roles ... Closed
split to GODRIVER-2081 Add native support for AWS IAM Roles ... Closed
split to JAVA-4234 Add native support for AWS IAM Roles ... Closed
split to MOTOR-773 Add native support for AWS IAM Roles ... Closed
split to NODE-3445 Add native support for AWS IAM Roles ... Closed
split to NODE-5135 [Node] Add native support for AWS IAM... Closed
split to PHPC-1895 Add native support for AWS IAM Roles ... Closed
split to PYTHON-2818 Add native support for AWS IAM Roles ... Closed
split to RUBY-2732 Add native support for AWS IAM Roles ... Closed
split to RUST-906 Add native support for AWS IAM Roles ... Closed
Problem/Incident
Related
related to DRIVERS-2011 On-demand callback for AWS credentials Closed
related to DRIVERS-1941 Add MONGODB-AWS Support for EKS Servi... Closed
is related to JAVA-4292 AWS credential refreshing Closed
Driver Changes: Needed
Quarter: FY23Q3, FY24Q1
Downstream Changes Summary:

Summary of required changes

  • Add support for AssumeRoleWithWebIdentity in AWS Auth
  • Add integration tests to verify usage
  • Use the credentials found in this document

Additional background

Please see https://github.com/mongodb/specifications/commit/bc4257fed21186ba9b53e2c0b7e92482da196882 for the specification change and https://github.com/mongodb/specifications/commit/cdd93a4c7639014c8837d34a3e26e408d7b14d5b for a clarification.

Please see https://github.com/mongodb/mongo-csharp-driver/commit/daa88998837aace9296b7c1f599c901f3cdac86f for a reference implementation in C#.

Integration test

Drivers are expected to add an integration test as described in the specification change

Case:
Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4081 Fixed 1.24.0
CXX-2307 Works as Designed 3.8.0
CSHARP-3740 Fixed 2.19.0
GODRIVER-2081 Fixed 1.12.0
JAVA-4234 Fixed 4.8.0
NODE-3445 Fixed 5.1.0
MOTOR-773 Duplicate
PYTHON-2818 Fixed 4.4
PHPC-1895 Won't Do
RUBY-2732 Fixed 2.19.0
RUST-906 Fixed 2.6.0
SWIFT-1258 Won't Do
NODE-5135 Duplicate

 Description   

Currently drivers support authenticating against a database using AWS IAM roles for applications running on EC2 or ECS tasks.

In scope of DRIVERS-2011 (and already implemented in Java driver in scope of JAVA-4118), applications will be able to supply a callback which will allow it to supply credentials for any type of service, including Elastic Kubernetes Service (EKS).

This ticket would add support for EKS natively to our drivers, without requiring use of an application-supplied callback.

This improvement would be useful for those who are running on Kubernetes in AWS and reduce friction by supporting EKS natively in our drivers

Note

EKS IAM Roles for service accounts give pods an IAM role, this is exposed as an environment variable with an open id connect token that can get handed to AWS sts for a set of temporary creds (much like ECS/EC2).



 Comments   
Comment by Githook User [ 02/Mar/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-1746 replace typo of `AWS_ROLE_SESSION_NAME` to `AWS_ROLE_ARN` (#1384)

  • replace typo of AWS_ROLE_SESSION_NAME to AWS_ROLE_ARN
Comment by Githook User [ 14/Nov/22 ]

Author:

{'name': 'Steven Silvester', 'email': 'steven.silvester@ieee.org', 'username': 'blink1073'}

Message: DRIVERS-1746 Add native support for AWS IAM Roles for service accounts, EKS in particular (#223)
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/86e713089877338ce5387a01ddea0a8f3d95ed72

Comment by Githook User [ 02/Nov/22 ]

Author:

{'name': 'Steven Silvester', 'email': 'steven.silvester@ieee.org', 'username': 'blink1073'}

Message: DRIVERS-1746 Add note about AssumeRole distinction (#1334)
Branch: master
https://github.com/mongodb/specifications/commit/cdd93a4c7639014c8837d34a3e26e408d7b14d5b

Comment by Githook User [ 06/Sep/22 ]

Author:

{'name': 'Steven Silvester', 'email': 'steven.silvester@ieee.org', 'username': 'blink1073'}

Message: DRIVERS-1746 Add native support for AWS IAM Roles for service accounts, EKS in particular (#1299)
Branch: master
https://github.com/mongodb/specifications/commit/bc4257fed21186ba9b53e2c0b7e92482da196882

Comment by Lance Nehring [ 17/Aug/22 ]

This is high priority for our team too.  I know there are pull requests with a fix to include IAM ROLE support.   It didn't appear to be a significant change to the code base - but perhaps I'm wrong.   I'm wondering why it's still blocked.   I'm sorely tempted to use a driver such as: https://github.com/HoutanSadafi/mongo-java-driver/tree/add-support-for-eks-with-aws-iam-role-authentication    but I'm concerned about support.

 

Ah, I now see JAVA-4292,  I'm going to investigate using that.

Comment by Jason Penniman [ 19/Jul/22 ]

This is very critical for us as well.

Comment by Valer Orlovsky [ 15/Jul/22 ]

Hi, we run EKS and I've just stumbled upon this issue. Just wondering, wouldn't it be better to just use the relevant AWS SDKs in the drivers? That would provide a proper, consistent experience as the SDKs know where to look for credentials in the entire chain, including the EKS IRSA model. Additionally it would also provide a better UX for connecting locally via roles, using the AWS_PROFILE env var and ~/.aws/config. 
Has there been a decision to deliberately not depend on the AWS SDKs ?

Comment by Steve Silvester [ 16/Jun/22 ]

Update: we have merged (but not yet released) support for the Python driver (https://github.com/mongodb/pymongo-auth-aws/pull/9).

Comment by Kekoa Vincent [ 15/Jun/22 ]

Great to hear, @Steve Silvester I will see what I can do.

Comment by Steve Silvester [ 15/Jun/22 ]

Hi kevincent@tradestation.com, yes, this is the favored ticket. We are planning to include this feature in the upcoming driver releases. If you'd like to resume https://github.com/mongodb/specifications/pull/1075 by addressing the feedback on the PR that would be a great help.

Indeed, the level of effort is small. We have a WIP PR for the Python driver in https://github.com/mongodb/pymongo-auth-aws/pull/9.

Comment by Kekoa Vincent [ 15/Jun/22 ]

Is this issue now the favored one for the implementation of now closed DRIVERS-1941 issue? This issue appears to be the same, if so, I had some details in the other ticket that would be useful to the implementation. We were able to implement this in a fork of the Go driver for our own use, and put some notes together to help with the spec changes.

There are some issues that need to be worked out, but a lot of EKS users now would like to avoid password-based auth, and provide an equally secure alternative to AWS data store solutions.

I've been following this group of issues for a while, and I am thinking that the level of effort involved in this change might be misunderstood or overestimated. This is very doable as we have implemented this ourselves in our own fork of the driver and have been using it for a while successfully. This would greatly improve EKS user experience and ability to match the same security as AWS provides for its own datastores. Callback or no-callback, AWS ECS and Lambda runtimes are fully supported but EKS support is languishing which doesn't make logical sense given the rise in popularity of EKS.

Comment by Steve Silvester [ 23/May/22 ]

One additional note if we decide to implement this feature, we should be handling the AWS_STS_REGIONAL_ENDPOINTS variable if set (on Kubernetes 1.22+).

If the value is set to "regional", we use the regional endpoint: https://sts.<AWS_REGION>.amazonaws.com, where AWS_REGION is also an environment variable. If AWS_STS_REGIONAL_ENDPOINTS is not set or is equal to "Global", then we should use the global https://sts.amazonaws.com endpoint.

https://docs.aws.amazon.com/eks/latest/userguide/configure-sts-endpoint.html

Comment by Jeffrey Yemin [ 08/Sep/21 ]

We are considering JAVA-4292 as an alternative means of satisfying this use case.

Comment by Ross Lawley [ 11/May/21 ]

Example PR adding support for the Java driver: https://github.com/mongodb/mongo-java-driver/pull/700

Generated at Thu Feb 08 08:23:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.