9

I have a small Spring Boot app, using Spring Cloud AWS (1.0.0.RELEASE) to access SQS queue. It is beeing deployed on an EC2 instance with Instance Profile set. It appears that AWS side of things is working, as I can access both relevant metadata links: iam/info and iam/security-credentials/role-name, and they do contain correct information. Just to be sure, I've used aws cmdline utility (aws sqs list-queues) and it does work, so I guess setup is ok. However, when the app starts, it reads application.properties (which contains line cloud.aws.credentials.instanceProfile=true) then drops following warning: com.amazonaws.util.EC2MetadataUtils: Unable to retrieve the requested metadata and finally throws following exception:

Caused by: com.amazonaws.AmazonServiceException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
        at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1071)
        at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:719)
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:454)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:294)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2291)
        at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:516)
        at com.amazonaws.services.sqs.buffered.AmazonSQSBufferedAsyncClient.getQueueUrl(AmazonSQSBufferedAsyncClient.java:278)
        at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:78)
        at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:37)
        at org.springframework.messaging.core.CachingDestinationResolverProxy.resolveDestination(CachingDestinationResolverProxy.java:88)
        at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:295)
        at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.start(SimpleMessageListenerContainer.java:38)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
        ... 17 common frames omitted

...which means that for some reason Spring Cloud AWS is not picking up on Instance Profile credentials. I've enabled debug log level on com.amazonaws.request and it appears that request is sent without access key and secret key.

DEBUG --- com.amazonaws.request                    : Sending Request: POST https://sqs.eu-west-1.amazonaws.com / Parameters: (Action: GetQueueUrl, Version: 2012-11-05, QueueName: xxxxxxxxxxxxx, ) Headers: (User-Agent: aws-sdk-java/1.9.3 Linux/3.14.35-28.38.amzn1.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/25.45-b02/1.8.0_45 AmazonSQSBufferedAsyncClient/1.9.3, )

Anybody has any idea what am I missing or at least any hints how to further debug this?

EDIT: After going through spring-cloud-aws code a bit, I've kinda moved forward. Configuration file application.properties bundled with jar had some text value for accessKey and secretKey. My customized application.properties haven't got those properties and that probably caused spring to use values in bundled file as defaults. I've included them with empty values, which changed the exception to com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain. It appears that AWS SDK is configured with DefaultProviderChain, yet it still fails to pick up instance profile credentials.

2 Answers 2

10

The solution to this problem comes from two distinct facts.

  1. Instance profile credentials are going to be used only and only if application.properties has instanceProfile property set to true and accessKey set to null (ContextCredentialsAutoConfiguration).

  2. Even if you will provide your custom application.properties file, Spring is going to read application.properties file bundled with app jar (if it does exist). If that's the case, properties from both files will sum up to create an execution enviroment. I suspect that bundled file is parsed first, then custom second, overriding any property present in bundled file.

In my case, bundled application.properties had accessKey and secretKey placeholders (with phony values) which were filled out by developer whenever he wanted some testing outside of EC2 enviroment. That made accessKey not null and therefore, excluded instance profile path. I just removed the application.properties file from jar and that solved the problem.

4
cloud:
  aws:
    credentials:
      accessKey:
      secretKey:
      instanceProfile: true
      useDefaultAwsCredentialsChain: true

This would do the trick, if you were using the latest (2.X.X) Spring AWS Cloud.

1
  • so… if your .aws/credentials profile is not named default how do you specify it? NVM my IDE recommended cloud.aws.credentials.profile-name=prod
    – dlamblin
    Feb 25, 2021 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.