37

I was thinking about deleting and then re-creating bucket (bad option which I realised later).

Then how can delete all objects from the bucket?

I tried this : http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Bucket.delete_objects

But it deletes multiple objects not all.

can you suggest what is the best way to empty bucket ?

1
  • This is a limitation of the S3 API - delete multiple object. You will have to implement pagination and delete 1000 at a time.
    – AChampion
    Apr 10, 2017 at 17:51

5 Answers 5

77

Just use aws cli.

aws s3 rm s3://mybucket --recursive

Well, for longer answer if you insists to use boto3. This will send a delete marker to s3. No folder handling required. bucket.Object.all will create a iterator that not limit to 1K .

import boto3    
s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
# suggested by Jordon Philips 
bucket.objects.all().delete()
14
  • 12
    You can also just do bucket.objects.all().delete(), though it gets a bit more complicated if your bucket is versioned Apr 11, 2017 at 21:15
  • @JordonPhillips Thank for your answer. This worked for me as I'm not using versioning for my bucket. But will this find all the objects inside the bucket and then delete them one by one? what happens when there are huge amounts of data in bucket ? Will it be slow? Apr 12, 2017 at 5:32
  • 2
    @TusharNiras, Under the hood boto3 will use delete_objects when you use that snippet, so you will make one request per page of objects (up to 1000 at a time). This is as fast as you can make it without threading. Apr 12, 2017 at 15:27
  • 2
    @Davos After second though, I have remove bucket.delete() , just in case people testing the script get their bucket deleted.
    – mootmoot
    May 15, 2019 at 9:56
  • 9
    Looks like you can now skip the .all() and just do bucket.objects.delete(). boto3.amazonaws.com/v1/documentation/api/latest/reference/….
    – Ryan
    Feb 17, 2020 at 22:31
26

If versioning is enabled, there's a similar call to the other answer to delete all object versions:

import boto3
s3 = boto3.resource('s3')
bucket = s3.Bucket('bucket-name')
bucket.object_versions.delete()
1
  • 1
    i found this to be incredibly slow. It seems to collect and return details about each object when complete, maybe this is why? any hints there? (I know the question asked specifically for boto3, but the console "empty" option is much faster) Jan 21, 2022 at 2:28
17

Based on previous responses, and adding check of versioning enabled, you can empty a bucket, with versions enabled or not doing:

s3 = boto3.resource('s3')
s3_bucket = s3.Bucket(bucket_name)
bucket_versioning = s3.BucketVersioning(bucket_name)
if bucket_versioning.status == 'Enabled':
    s3_bucket.object_versions.delete()
else:
    s3_bucket.objects.all().delete()
0

Or if you need to use low-level operations on boto3 and yoy need to use boto3.client instead of boto3.resource here's a code that can help. In which I called all the objects with s3_client.list_objects_v2 and extracted the key on them in order to delete them with s3_client.delete_objects

import boto3

s3_client = boto3.client("s3")
objects = s3_client.list_objects_v2(Bucket = "bucket-name")["Contents"]
objects = list(map(lambda x: {"Key":x["Key"]},objects))
s3_client.delete_objects(Bucket = "bucket-name", Delete = {"Objects":objects})
1
  • be aware this piece of code does not have pagination
    – halil
    Jun 25 at 9:15
0

This is what I'm using

def filter_list_of_dicts_key_startswith(list_of_dicts: list, key: str, value: str) -> list:
    return [item for item in list_of_dicts if item[key].startswith(value)]

def delete_s3_bucket(session: any, region: str, bucket_name: str) -> None:
    client = session.client("s3", region_name=region)
    client.delete_bucket(Bucket=bucket_name)
    print(f"       -> 🗑️  S3 Bucket Deleted: {bucket_name}")


def delete_s3_bucket_objects(session: any, region: str, bucket_name: str) -> None:
    client = session.client("s3", region_name=region)
    objects = client.get_paginator("list_objects_v2")

    objects_iterator = objects.paginate(Bucket=bucket_name)
    for page in objects_iterator:
        if "Contents" in page:
            objects = [{"Key": obj["Key"]} for obj in page["Contents"]]
            client.delete_objects(Bucket=bucket_name, Delete={"Objects": objects})
            print(f"       -> 🗑️  S3 Bucket Objects Deleted: {len(objects)}")



...

session = boto3.Session(profile_name="my-profile-account")
client = session.client("s3", region_name="us-east-1")
list_buckets = client.list_buckets()
buckets = list_buckets["Buckets"]

print(f"    📚 S3 Available Buckets: {len(buckets)}")
for bucket in buckets:
    print(f"      -> {bucket['Name']}, Creation Date = {bucket['CreationDate']}")

buckets_filtered = filter_list_of_dicts_key_startswith(buckets, "Name", "this-is-a-test-bucket-prefix-")

print(f"    ⛔ S3 Buckets to Delete: {len(buckets_filtered)}")
for bucket in buckets_filtered:
    print(f"      -> {bucket['Name']}, Creation Date = {bucket['CreationDate']}")

    delete_s3_bucket_objects(session, "us-east-1", bucket["Name"])
    delete_s3_bucket(session, "us-east-1", bucket["Name"])
...

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.