I need to write code in python that will delete the required file from an Amazon s3 bucket. I am able to connect to the Amazon s3 bucket, and also to save files, but how can I delete a file?
Using boto3
(currently version 1.4.4) use S3.Object.delete()
.
import boto3
s3 = boto3.resource('s3')
s3.Object('your-bucket', 'your-key').delete()
-
1
-
3@AkashTantri I haven't personally tried, but the doc says it removes the null version (if there is one) [...] If there isn't a null version, Amazon S3 does not remove any objects. So my guess is that it won't throw an error. If you happen to try it (just do something like
s3.Object('existing-bucket', 'bogus-key').delete()
and see what happens. Also trys3.Object('bogus-bucket', 'bogus-key').delete()
. – Kohányi Róbert Feb 8 '19 at 17:28 -
1
-
-
Does the
your-key
here means the actual file name inyour-bucket
on S3? – Underoos Oct 10 '19 at 10:06
found one more way to do it using the boto:
from boto.s3.connection import S3Connection, Bucket, Key
conn = S3Connection(AWS_ACCESS_KEY, AWS_SECERET_KEY)
b = Bucket(conn, S3_BUCKET_NAME)
k = Key(b)
k.key = 'images/my-images/'+filename
b.delete_key(k)
-
10If you wanted to delete EVERYTHING in a bucket, you could do:
for x in b.list(): b.delete_key(x.key)
– jontsai Jul 31 '12 at 17:02 -
19
-
For this code snippet to work as presented, you'll need to import
Bucket
andKey
, too. As in:from boto.s3.connection import S3Connection, Bucket, Key
– Nick Chammas Apr 26 '14 at 1:46 -
I get
>>> from boto.s3.connection import S3Connection, Bucket, Key Traceback (most recent call last): File "<console>", line 1, in <module> ImportError: No module named boto.s3.connection
please update the answer to boto3 – Harry Moreno Apr 24 '17 at 20:50 -
1figured it out and wrote up a solution harrymoreno.com/2017/04/24/… – Harry Moreno Apr 25 '17 at 21:16
Using the Python boto3 SDK (and assuming credentials are setup for AWS), the following will delete a specified object in a bucket:
import boto3
client = boto3.client('s3')
client.delete_object(Bucket='mybucketname', Key='myfile.whatever')
-
6@Rob The boto3 documentation is misleading. it will create a delete marker if the object is versioned. It will delete the object otherwise. – jarmod Jun 6 '18 at 14:43
-
1Clean and simple. Could be the accepted answer, and should definitely be merged with @Kohányi Róbert s answer as both are best approaches for the task. – PaulB Aug 12 '19 at 17:24
Welcome to 2020 here is the answer in Python/Django:
from django.conf import settings
import boto3
s3 = boto3.client('s3')
s3.delete_object(Bucket=settings.AWS_STORAGE_BUCKET_NAME, Key=f"media/{item.file.name}")
Took me far too long to find the answer and it was as simple as this.
I'm surprised there isn't this easy way : key.delete()
:
from boto.s3.connection import S3Connection, Bucket, Key
conn = S3Connection(AWS_ACCESS_KEY, AWS_SECERET_KEY)
bucket = Bucket(conn, S3_BUCKET_NAME)
k = Key(bucket = bucket, name=path_to_file)
k.delete()
please try this code
import boto3
s3 = boto3.client('s3')
s3.delete_object(Bucket="s3bucketname", Key="s3filepath")
Try to look for an updated method, since Boto3 might change from time to time. I used my_bucket.delete_objects():
import boto3
from boto3.session import Session
session = Session(aws_access_key_id='your_key_id',
aws_secret_access_key='your_secret_key')
# s3_client = session.client('s3')
s3_resource = session.resource('s3')
my_bucket = s3_resource.Bucket("your_bucket_name")
response = my_bucket.delete_objects(
Delete={
'Objects': [
{
'Key': "your_file_name_key" # the_name of_your_file
}
]
}
)
Via which interface? Using the REST interface, you just send a delete:
DELETE /ObjectName HTTP/1.1
Host: BucketName.s3.amazonaws.com
Date: date
Content-Length: length
Authorization: signatureValue
Via the SOAP interface:
<DeleteObject xmlns="http://doc.s3.amazonaws.com/2006-03-01">
<Bucket>quotes</Bucket>
<Key>Nelson</Key>
<AWSAccessKeyId> 1D9FVRAYCP1VJEXAMPLE=</AWSAccessKeyId>
<Timestamp>2006-03-01T12:00:00.183Z</Timestamp>
<Signature>Iuyz3d3P0aTou39dzbqaEXAMPLE=</Signature>
</DeleteObject>
If you're using a Python library like boto, it should expose a "delete" feature, like delete_key()
.
-
yes, i am using that python library, but will that delete, the file ? should i do it this way: k.key = 'images/anon-images/small/'+filename k.delete_key() is this correct ? please let me know. – Suhail Jun 29 '10 at 13:23
-
@Suhail: I haven't used that library, but from the source I linked, what it's actually doing is a
DELETE
call via the REST interface. So yes, despite the name "delete_key" (which I agree is unclear), it's really deleting the object referenced by the key. – T.J. Crowder Jun 29 '10 at 13:40 -
1What about removing lot of files with a common prefix in name? Does S3 allow some bulk delete for such case, or deleting them one by one (which is slow) is the must? – Illarion Kovalchuk Jul 5 '10 at 10:11
-
@Shaman: I'm not an S3 expert, but as far as I know, you can only delete a specific file. But you probably want to actually ask that as a question so it gets attention from S3 experts. – T.J. Crowder Jul 5 '10 at 12:17
-
Right after commenting here I've added such a question. It has 2 views yet :) – Illarion Kovalchuk Jul 6 '10 at 9:38
Simplest way to do this is:
import boto3
s3 = boto3.resource("s3")
bucket_source = {
'Bucket': "my-bcuket",
'Key': "file_path_in_bucket"
}
s3.meta.client.delete(bucket_source)
For now I have resolved the issue by using the Linux utility s3cmd. I used it like this in Python:
delFile = 's3cmd -c /home/project/.s3cfg del s3://images/anon-images/small/' + filename
os.system(delFile)
-
1It's not exactly pythonic to invoke a subshell to communicate with S3 (a library or a direct HTTP transaction would be more elegant), but it still works. I don't think it deserves a downvote. +1. – Randall Cook Apr 18 '13 at 18:34
It's worked for me try it.
import boto
import sys
from boto.s3.key import Key
import boto.s3.connection
AWS_ACCESS_KEY_ID = '<access_key>'
AWS_SECRET_ACCESS_KEY = '<secret_access_key>'
Bucketname = 'bucket_name'
conn = boto.s3.connect_to_region('us-east-2',
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
is_secure=True,
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)
bucket = conn.get_bucket(Bucketname)
k = Key(bucket)
k.key = 'filename to delete'
bucket.delete_key(k)
you can do it using aws cli : https://aws.amazon.com/cli/ and some unix command.
this aws cli commands should work:
aws s3 rm s3://<your_bucket_name> --exclude "*" --include "<your_regex>"
if you want to include sub-folders you should add the flag --recursive
or with unix commands:
aws s3 ls s3://<your_bucket_name>/ | awk '{print $4}' | xargs -I% <your_os_shell> -c 'aws s3 rm s3:// <your_bucket_name> /% $1'
explanation:
- list all files on the bucket --pipe-->
- get the 4th parameter(its the file name) --pipe--> // you can replace it with linux command to match your pattern
- run delete script with aws cli
if you are trying to delete file using your own local host console then you can try running this python script assuming that you have have already assigned your access id and secret key in the system
import boto3
#my custom sesssion
aws_m=boto3.session.Session(profile_name="your-profile-name-on-local-host")
client=aws_m.client('s3')
#list bucket objects before deleting
response = client.list_objects(
Bucket='your-bucket-name'
)
for x in response.get("Contents", None):
print(x.get("Key",None));
#delete bucket objects
response = client.delete_object(
Bucket='your-bucket-name',
Key='mydocs.txt'
)
#list bucket objects after deleting
response = client.list_objects(
Bucket='your-bucket-name'
)
for x in response.get("Contents", None):
print(x.get("Key",None));
The following worked for me (based on example for a Django model, but you can pretty much use the code of the delete
method on its own).
import boto3
from boto3.session import Session
from django.conf import settings
class Video(models.Model):
title=models.CharField(max_length=500)
description=models.TextField(default="")
creation_date=models.DateTimeField(default=timezone.now)
videofile=models.FileField(upload_to='videos/', null=True, verbose_name="")
tags = TaggableManager()
actions = ['delete']
def __str__(self):
return self.title + ": " + str(self.videofile)
def delete(self, *args, **kwargs):
session = Session (settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
s3_resource = session.resource('s3')
s3_bucket = s3_resource.Bucket(settings.AWS_STORAGE_BUCKET_NAME)
file_path = "media/" + str(self.videofile)
response = s3_bucket.delete_objects(
Delete={
'Objects': [
{
'Key': file_path
}
]
})
super(Video, self).delete(*args, **kwargs)
Below is code snippet you can use to delete the bucket,
import boto3, botocore
from botocore.exceptions import ClientError
s3 = boto3.resource("s3",aws_access_key_id='Your-Access-Key',aws_secret_access_key='Your-Secret-Key')
s3.Object('Bucket-Name', 'file-name as key').delete()
Use the S3FileSystem.rm
function in s3fs
.
You can delete a single file or several at once:
import s3fs
file_system = s3fs.S3FileSystem()
file_system.rm('s3://my-bucket/foo.txt') # single file
files = ['s3://my-bucket/bar.txt', 's3://my-bucket/baz.txt']
file_system.rm(files) # several files