[JAVA-4718] Depend on AWS SDK for fetching credentials Created: 30/Aug/22  Updated: 28/Oct/23  Resolved: 19/Oct/22

Status: Closed
Project: Java Driver
Component/s: Authentication
Affects Version/s: None
Fix Version/s: 4.8.0

Type: New Feature Priority: Major - P3
Reporter: Jeffrey Yemin Assignee: Jeffrey Yemin
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
Problem/Incident
causes JAVA-4836 AWS dependency should be set as optio... Closed
Related
is related to JAVA-4292 AWS credential refreshing Closed
is related to JAVA-4690 Cache AWS Credentials Where Possible Closed
Backwards Compatibility: Fully Compatible
Documentation Changes: Needed

 Description   

Proposal

Add an optional dependency on both v1 and v2 of the AWS SDK. If v2 is present on the runtime classpath, the driver will use its software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider to fetch credentials from the environment. Otherwise, if v1 is present, the driver will use com.amazonaws.auth.DefaultAWSCredentialsProviderChain to fetch the credentials. If neither are present, the driver will use its existing code to fetch the credentials.

Motivation

By relying on the AWS SDK when it's available, users of AWS authentication can take advantage of all the features of the default credentials provider in the SDK, which is a superset of what's in the driver authentication spec. The extra functionality includes:

  • support for system properties in additional to environment variables
  • support for additional environment variables
  • support for additional providers, e.g. file-based configuration profiles


 Comments   
Comment by Githook User [ 19/Oct/22 ]

Author:

{'name': 'Jeff Yemin', 'email': 'jeff.yemin@mongodb.com', 'username': 'jyemin'}

Message: Use AWS SDK, if available, for AWS credential fetching (#1017)

  • Add optional dependencies for both AWS SDK v2 and v1
  • For credential fetching, use AWS SDK v2 if available. Otherwise use AWS SDK v1. Otherwise, use the existing driver implementations

JAVA-4718
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/7b1904d3038e042ece2486b214971661da656334

Comment by Jeffrey Yemin [ 19/Oct/22 ]

We're going to add v1 support to the current PR. Thanks for the feedback.

Regards,
Jeff

Comment by Rohit Pant [ 18/Oct/22 ]

Hi Ross,

While I can't find any concrete documentation for the jars provided by the Glue runtime, a very basic Glue job which does nothing but - 

Class.forName("software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider")

fails with -

ERROR [main] glue.ProcessLauncher (Logging.scala:logError(73)): Exception in User Class: java.lang.ClassNotFoundException : software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider

which would suggest that v2 SDK is missing by default. Meanwhile, I have been able to successfully fetch AWS credentials using the v1 AWS SDK. In fact, we have actually created a custom MongoClientFactory implementation which fetches credentials using the v1 SDK as the DefaultMongoClientFactory implementation failed to fetch them. While this works fine as a workaround, we would rather prefer the default implementation to work. We have an open support ticket for this and this ticket was pointed out to us as the potential resolution. So, it would definitely help us out if v1 support was added as well.

Mongo Spark docs for reference on the mongoClientFactory bit - https://www.mongodb.com/docs/spark-connector/current/configuration/write/ 

Regards,

Rohit Pant

Comment by Ross Lawley [ 18/Oct/22 ]

Hi rohit.pant@gs.com,

I believe the Glue runtime only provides the v1 SDK by default.

Do you have any information or links regarding that? I understand that the namespaces are different for the different versions com.amazonaws.auth and software.amazon.awssdk.auth so it would be possible. Its just v2 is stable and preferred by aws - it came out of beta in 2018. So just wanted to clarify if new users would be wanting to support the older version.

Many thanks,

Ross

Comment by Rohit Pant [ 18/Oct/22 ]

Hi Jeffrey,

Thanks for adding this feature to the Mongo Java Driver! This might help us resolve MONGODB-AWS authn issues that we have been facing with the Mongo Spark Connector in AWS Glue jobs. I was going through the MR that you have raised. Can you also consider adding support for the v1 of the AWS SDK as you have mentioned in the proposal? I believe the Glue runtime only provides the v1 SDK by default. While we could always package the additional jar on our own, this would definitely make it simpler.

Regards,

Rohit Pant

Comment by Jeffrey Yemin [ 17/Oct/22 ]

Documentation

Document that applications should add a dependency to software.amazon.awssdk:auth to get access to all the different way to get credentials that are supported by software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider but not supported natively by the Java driver. See https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html for details.

Generated at Thu Feb 08 09:02:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.