21

Running the following docker command on mac works and on linux, running ubuntu cannot find the aws cli credentials. It returns the following message: Unable to locate credentials Completed 1 part(s) with ... file(s) remaining

The command which runs an image and mounts a data volume and then copies a file from and s3 bucket, and starts the bash shell in the docker container.

sudo docker run -it --rm -v ~/.aws:/root/.aws username/docker-image sh -c 'aws s3 cp s3://bucketname/filename.tar.gz /home/emailer && cd /home/emailer && tar zxvf filename.tar.gz && /bin/bash'

What am I missing here?

This is my Dockerfile:

FROM ubuntu:latest

#install node and npm
RUN apt-get update && \
    apt-get -y install curl && \
    curl -sL https://deb.nodesource.com/setup | sudo bash - && \
    apt-get -y install python build-essential nodejs

#install and set-up aws-cli
RUN sudo apt-get -y install \
    git \
    nano \
    unzip && \
    curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && \
    unzip awscli-bundle.zip

RUN sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

# Provides cached layer for node_modules
ADD package.json /tmp/package.json
RUN cd /tmp && npm install
RUN mkdir -p /home/emailer && cp -a /tmp/node_modules /home/emailer/

8 Answers 8

24

Mounting $HOME/.aws/ into the container should work. Make sure to mount it as read-only.

It is also worth mentioning, if you have several profiles in your ~/.aws/config -- you must also provide the AWS_PROFILE=somethingsomething environment variable. E.g. via docker run -e AWS_PROFILE=xxx ... otherwise you'll get the same error message (unable to locate credentials).

Update: Added example of the mount command

docker run -v ~/.aws:/root/.aws …
3
  • Could you provide a an example of the mount command we should use with docker run? Commented Dec 22, 2021 at 15:29
  • @Cleanshooter sure, here you go Commented Dec 23, 2021 at 9:28
  • For Mac users this might not be enough since that folder is not accessible by default by Docker. Newer Docker versions have a simple workaround via the UI though, as shown here.
    – Jakob
    Commented Feb 24, 2023 at 16:44
8

You can use environment variable instead of copying ~/.aws/credentials and config file into container for aws-cli

docker run \ -e AWS_ACCESS_KEY_ID=AXXXXXXXXXXXXE \ -e AWS_SECRET_ACCESS_KEY=wXXXXXXXXXXXXY \ -e AWS_DEFAULT_REGION=us-west-2 \ <img>

Ref: AWS CLI Doc

2
  • How would you do this via an env-file (without putting the credentials in the env file - but rather just the names of the ENV variables so that they are correctly detected)?
    – jtlz2
    Commented Jun 22, 2022 at 9:42
  • can i do it with docker build? it's saying "unknown shorthand flag: 'e' in -e"
    – Raksha
    Commented Aug 7, 2023 at 16:05
8

the only solution that worked for me in this case is:

volumes:
  - ${USERPROFILE}/.aws:/root/.aws:ro
3
  • 1
    This worked for me (though I used ~/.aws instead of the $USERPROFILE var).
    – tptcat
    Commented Feb 13, 2022 at 3:22
  • How do you do this using a dockerfile instead of docker compose?
    – Ash
    Commented Dec 5, 2022 at 8:44
  • 1
    @ash you can't use it in a dockerfile, cause a dockerfile doesn't have volumes. You can run a docker file using docker run and specify the volumes as commands though. Or you can use a COPY command in the dockerfile to bake your aws credentials file into it, but I wouldn't recommend doing that.
    – Luke
    Commented Jun 29, 2023 at 14:50
7

what do you see if you run

ls -l ~/.aws/config

within your docker instance?

5
  • -rw------- 1 1000 staff 29 May 25 14:04 /root/.aws/config
    – hyprstack
    Commented Jun 26, 2015 at 13:16
  • if do cat ~/.aws/config does it show you your access id and secret?
    – devfubar
    Commented Jun 26, 2015 at 13:30
  • to be honest I wasn't expecting that. That error says the problem, It cant find the config file which is usually located at ~/.aws/config. I suspected the problem was that you were assuming the home directory was /root/ which might not be the case in every instance
    – devfubar
    Commented Jun 26, 2015 at 13:37
  • 2
    additionally if this is your issue perhaps you should set the environment variable AWS_CONFIG_FILE=/root/.aws/config just to double make sure your docker knows exactly where to look
    – devfubar
    Commented Jun 26, 2015 at 13:43
  • Ok, one important thing is missing here is the env variable: AWS_SDK_LOAD_CONFIG=1
    – Matteo
    Commented Apr 28, 2020 at 1:46
2

There are a few things that could be wrong. One, as mentioned previously you should check if your ~/.aws/config file is set accordingly. If not, you can follow this link to set it up. Once you have done that you can map the ~/.aws folder using the -v flag on docker run.

If your ~/.aws folder is mapped correctly, make sure to check the permissions on the files under ~/.aws so that they are able to be accessed safely by whatever process is trying to access them. If you are running as the user process, simply running chmod 444 ~/.aws/* should do the trick. This will give full read permissions to the file. Of course, if you want write permissions you can add whatever other modifiers you need. Just make sure the read octal is flipped for your corresponding user and/or group.

2
  • yes, for me it was the permission as @juicedatom mentionned chmod 444 ~/.aws/* is the trick
    – Remus 007
    Commented Mar 7, 2022 at 13:41
  • 1
    Doesn't this make the credentials available to all users on the system? Sounds dangerous...
    – jtlz2
    Commented Jun 22, 2022 at 9:49
1

The issue I had was that I was running Docker as root. When running as root it was unable to locate my credentials at ~/.aws/credentials, even though they were valid.

Directions for running Docker without root on Ubuntu are here: https://askubuntu.com/a/477554/85384

2
  • 1
    I had the same issue ~/.aws/credentials was not loading. I had to use /root/.aws/credentials. thank you.
    – Imran
    Commented Feb 22, 2021 at 16:52
  • Should above be mentioned in the docker-compose at Volume right? I tried but still not able to connect to aws
    – Karthik
    Commented Sep 17, 2021 at 3:58
1

Another case of Unable to locate credentials inside docker running on an ec2 with the right IAM profile was due to using metadata HTTP tokens required.

When checking iam security-credentials metadata endpoint it would return 401:

curl -I http://169.254.169.254/latest/meta-data/iam/security-credentials/
python
import requests
requests.get('http://169.254.169.254/latest/meta-data/iam/security-credentials/')

You're meant to pass a token, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-v2-how-it-works.html

Otherwise set the http token as optional in the instance metadata options config

0

You just have to pass the credential in order to be the AWS_PROFILE, if you do not pass anything it will use the default, but if you want you can copy the default and add your desired credentials.

In Your credentials

[profile_dev]
aws_access_key_id = xxxxxxxxxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
output = json
region = eu-west-1

In Your docker-compose

version: "3.8"
services:
  cenas:
    container_name: cenas_app
    build: .
    ports:
      - "8080:8080"
    environment:
      - AWS_PROFILE=profile_dev
    volumes:
      - ~/.aws:/app/home/.aws:ro

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.