'Unable to load AWS credentials from any provider in the chain Spring Cloud Stream Kinesis binder

I am not able to connect to AWS kinesis in Spring Cloud Stream Kinesis binder (1.2.0.RELEASE) without using the default configuration at the system level. Only if the system has already been configured to use the default profile and the access key id and secret access key are set with the [default] profile the application works. Otherwise, it is not able to connect to AWS resources by throwing this exception:

Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3b2c8bda: Unable to load credentials from service endpoint, com.amazonaws.auth.profile.ProfileCredentialsProvider@688d619c: No AWS profile named 'default']
    at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1225)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:801)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:751)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:3768)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3737)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeDescribeTable(AmazonDynamoDBClient.java:1836)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:1804)
    at com.amazonaws.services.dynamodbv2.document.Table.describe(Table.java:137)
    at org.springframework.integration.aws.metadata.DynamoDbMetadataStore.afterPropertiesSet(DynamoDbMetadataStore.java:145)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774)

I have tried the following options and none of them worked for me:

  • Set environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • Set Java system properties aws.accessKeyId and aws.secretKey
  • Set cloud.aws.credentials.accessKey and cloud.aws.credentials.secretKey in the application.yml file.
spring-cloud-streamaws-java-sdkspring-integration-aws


Solution 1:[1]

It appears that for some reason the AWSCredentialProvider bean has been loaded incorrectly, so I was able to address this issue temporarily by setting the following bean. It's not a right fix, but it did unblock my work:

@Configuration
public class AWSCredentialProvider {
  @Value("${aws.access-key}")
  protected String accessKey;

  @Value("${aws.secret-key}")
  protected String secretKey;

  @Bean
  @Primary
  public AWSCredentialsProvider buildAWSCredentialsProvider() {
    AWSCredentials awsCredentials = new BasicAWSCredentials(accessKey, secretKey);
    return new AWSStaticCredentialsProvider(awsCredentials);
  }
}

Solution 2:[2]

This just about drove me nuts. After careful tracing through the AWS code, I found that if you set the system property

cloud.aws.credentials.use-default-aws-credentials-chain: true

it switches in the DefaultAWSCredentialsProviderChain. Otherwise, it uses a chain of two providers - EC2ContainerCredentialsProviderWrapper and ProfileCredentialsProvider.

This is for Spring Boot 2.3.4. For version 2.2.5, the system property is

cloud.aws.credentials.useDefaultAwsCredentialsChain: true

I haven't researched what version it changed to kebab case. Without this, it will ignore credentials passed in either environment variables or system properties. This code is in ContextCredentialsAutoConfiguration.registerBeanDefinitions().

Solution 3:[3]

I solved this problem.

I think spring-cloud-starter-aws not working to auto configuration. So I set the access key and secret key programmatically like this:

@Configuration
class AWSS3Configuration {

  @Value("\${cloud.aws.credentials.access-key}")
  val accessKey: String = ""

  @Value("\${cloud.aws.credentials.secret-key}")
  val secretKey: String = ""

  @Bean
  fun amazonS3(): AmazonS3 =
    AmazonS3ClientBuilder.standard()
      .withCredentials(AWSStaticCredentialsProvider(BasicAWSCredentials(accessKey, secretKey)))
      .build()

}

Solution 4:[4]

Setting instance-profile: true worked for me in an ECS environment.

cloud:
  aws:
    credentials:
      instance-profile: true

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 du-it
Solution 2 kenny_k
Solution 3 Sunhyoup Lee
Solution 4 charlb

Related Questions