15

During GitlabCi I got: "fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied"

My bucket policy :

{
"Version": "2008-10-17",
"Statement": [
    {
        "Sid": "AllowPublicRead",
        "Effect": "Allow",
        "Principal": {
            "AWS": "*"
        },
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::BUCKET-NAME/*"
    }
]

}

In gitlabCI settings set:

  • AWS_ACCESS_KEY_ID: YOUR-AWS-ACCESS-KEY-ID
  • AWS_SECRET_ACCESS_KEY: YOUR-AWS-SECRET-ACCESS-KEY
  • S3_BUCKET_NAME: YOUR-S3-BUCKET-NAME
  • DISTRIBUTION_ID: CLOUDFRONT-DISTRIBUTION-ID

My .gitlab-ci.yml

image: docker:latest

stages:
  - build
  - deploy

build:
  stage: build
  image: node:8.11.3
script:
  - export API_URL="d144iew37xsh40.cloudfront.net"
  - npm install
  - npm run build
  - echo "BUILD SUCCESSFULLY"
artifacts:
   paths:
    - public/
expire_in: 20 mins
environment:
name: production
only:
   - master
deploy:
  stage: deploy
  image: python:3.5
dependencies:
   - build
script:
  - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
  - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
  - export S3_BUCKET_NAME=$S3_BUCKET_NAME
  - export DISTRIBUTION_ID=$DISTRIBUTION_ID
  - pip install awscli --upgrade --user
  - export PATH=~/.local/bin:$PATH
  - aws s3 sync --acl public-read --delete public $S3_BUCKET_NAME
  - aws cloudfront create-invalidation --distribution-id 
$DISTRIBUTION_ID --paths '/*'
  - echo "DEPLOYED SUCCESSFULLY"

environment:
   name: production
only:
  - master

10 Answers 10

26

I'm not sure the accepted answer is actually acceptable, as it simply allows all operations on the bucket. Also the Sid is misleading... ;-)

This AWS article mentions the required permissions for aws s3 sync.

This is how a corresponding policy looks like:

{
"Version": "version_id",
"Statement": [
    {
        "Sid": "AllowBucketSync",
        "Effect": "Allow",
        "Action": [
            "s3:GetObject", 
            "s3:PutObject", 
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::BUCKET-NAME",
            "arn:aws:s3:::BUCKET-NAME/*"
        ]
    }
] }
2
  • 1
    It looks like s3:ListBucket is depreciated and one should use s3:ListObjectsV2 ? ListBucket within the actions table on Actions defined by Amazon S3 now links to the ListObjects page, and that page now encourages the use of ListObjectsV2.
    – daevski
    Sep 1, 2021 at 15:10
  • 2
    That is not correct. s3:ListBucket is the name of the permission that allows a user to list the objects in a bucket. ListObjectsV2 is the name of the API call that lists the objects in a bucket. Dec 3, 2022 at 22:12
12

I had this problem recently. No matter what I did, no matter what permissions I provided, I kept getting "An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied" when running aws s3 ls <bucket>

I had forgotten that I have multiple aws profiles configured in my environment. The aws command was using the default profile, which has a different set of access keys. I had to specify the --profile flag to the command:

aws s3 ls <bucket> --profile <correct profile>

That worked. It's a niche situation, but maybe it'll help someone out.

4

I got "AccessDenied" errors, too, even though the policy was correct. I gave mrbranden's solution a try though I only have one (the default) credentials configured. And lo and behold,

$ aws s3 ls <bucket> --profile=default

made it work!

My aws --version is aws-cli/1.18.69 Python/3.8.5 Linux/5.4.0-1035-aws botocore/1.16.19

3

For Amazon users who have enabled MFA, please use this: aws s3 ls s3://bucket-name --profile mfa.

And prepare the profile mfa first by running aws sts get-session-token --serial-number arn:aws:iam::123456789012:mfa/user-name --token-code 797395 --duration 129600. (replace 123456789012, user-name and 797395). enter image description here

3

I hit this as a misleading error (similar to Soren Bjornstad's comment). I had excluded "arn:aws:s3:::BUCKET-NAME" from the "Resource" list.

So instead of

"Resource": [
  "arn:aws:s3:::BUCKET-NAME/directory/*"
]

it must be

"Resource": [
  "arn:aws:s3:::BUCKET-NAME",
  "arn:aws:s3:::BUCKET-NAME/directory/*"
]
1
  • I needed to make this change as well. Thanks for sharing May 7 at 3:18
1

I just ran into this issue and the resolution was to not protect that access key and secret variables. Silly issue, and easy to overlook since I was working on the dev branch.

enter image description here

0

I got this as a rather misleading error message when I mistakenly used the full domain name with the s3:// prefix to select the bucket to operate on, like s3://s3.amazonaws.com/bucket_name. Switching to s3://bucket_name fixed the problem.

0

In my case, I was trying to download a file from an EC2 instance. Somehow it was not working. I was able to make the bucket and file public and then download it to my localhost, but this still didn't work from the EC2 instance.

This reminded me that we have a tighter setup around our VPC and network policies, which let me to check the NAT GWs and S3 VPC endpoint policies. That fixed the problem.

For reference this article has a detailed list of things to check: https://repost.aws/knowledge-center/ec2-instance-access-s3-bucket

-1

Try to update your bucket policy to:

{
"Version": "version_id",
"Statement": [
    {
        "Sid": "AllowPublicRead",
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::BUCKET-NAME",
            "arn:aws:s3:::BUCKET-NAME/*"
        ]
    }
] }

I hope you understand this is very insecure.

3
  • 7
    Despite the comment - I don't think s3:* is acceptable for anything but debugging purposes.
    – Jan Groth
    Apr 2, 2020 at 22:33
  • 2
    That's the reason of the comment. I don't think it deserves a down vote since the OP is using it. I'm helping him and warning him. I don't know if he needs to read/write/change the object its metadata, ... .
    – lvthillo
    Apr 3, 2020 at 16:46
  • 3
    Appreciate your comment. From my experience these are the 'dangerous' solutions on SOF that people simply c&p before moving on with their lifes. While disabling all checks silents the error it also creates a gaping security hole. It's a bit like 'fixing' a broken door lock by removing the door (and commenting that not having doors is potentially insecure). Honestly, service:* should never be the answer to an IAM question. If you agree with my stance, but SOF reputation is your concern - why don't you copy the suggested solution into yours and I remove my answer and give you an upvote?
    – Jan Groth
    Apr 3, 2020 at 20:54
-1

Principle is required now and it should look like this:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "AllowBucketSync",
        "Effect": "Allow",
        "Principal": "*",
        "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::YOURBUCKET",
            "arn:aws:s3:::YOURBUCKET/*"
        ]
    }
]

}

1
  • 1
    This policy contains the following error: Has prohibited field Principal For more information about the IAM policy grammar, see AWS IAM Policies
    – Teebu
    Nov 24, 2022 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.