16

Boto's S3 Key object contains last_modified date (available via parse_ts) but the base_field "date" (i.e., ctime) doesn't seem to be accessible, even though it's listed in key.base_fields.

Based on the table at http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html, it does seem that it is always automatically created (and I can't imagine a reason why it wouldn't be). It's probably just a simple matter of finding it somewhere in the object attributes, but I haven't been able to find it so far, although I did find the base_fields attribute which contains 'date'. (They're just a set and don't seem to have an available methods and I haven't been able to find documentation regarding ways to inspect them.)

For example, Amazon S3 maintains object creation date and size metadata and uses this information as part of object management.

Interestingly, create_time (system metadata field "Date" in link above) does not show up in the AWS S3 console, either, although last_modified is visible.

TL;DR: Because overwriting an S3 object is essentially creating a new one, the "last modified" and "creation" timestamp will always be the same.

4 Answers 4

28

Answering the old question, just in case others run into the same issue.

Amazon S3 maintains only the last modified date for each object. For example, the Amazon S3 console shows the Last Modified date in the object Properties pane. When you initially create a new object, this date reflects the date the object is created. If you replace the object, the date changes accordingly. So when we use the term creation date, it is synonymous with the term last modified date.

Reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html

2
  • 5
    This is a great link, Sameer, thank you! AFAIK this documentation didn't even exist back then, but it's definitely the correct answer now; I'm choosing your answer as the correct current answer. Thanks! Commented Sep 24, 2018 at 15:09
  • (See also my old comment below for some possibly useful examples.) Commented Jan 7, 2023 at 17:46
5

i suggest use key.last_modified since key.date seems to return the last time you viewed the file
so something like this :

key = bucket.get_key(key.name)
print(key.last_modified)
3

After additional research, it appears that S3 key objects returned from a list() may not include this metadata field!

The Key objects returned by the iterator are obtained by parsing the results of a GET on the bucket, also known as the List Objects request. The XML returned by this request contains only a subset of the information about each key. Certain metadata fields such as Content-Type and user metadata are not available in the XML. Therefore, if you want these additional metadata fields you will have to do a HEAD request on the Key in the bucket. (docs)

In other words, looping through keys:

for key in conn.get_bucket(bucket_name).list():
     print (key.date)

... does not return the complete key with creation date and some other system metadata. (For example, it's also missing ACL data).

Instead, to retrieve the complete key metadata, use this method:

key = bucket.get_key(key.name)
print (key.date)

This necessitates an additional HTTP request as the docs clearly state above. (See also my original issue report.)

Additional code details:

import boto

# get connection
conn = boto.connect_s3()

# get first bucket
bucket = conn.get_all_buckets()[0]

# get first key in first bucket
key = list(bucket.list())[0]

# get create date if available
print (getattr(key, "date", False))
# (False)

# access key via bucket.get_key instead:
k = bucket.get_key(key.name)

# check again for create_date
getattr(k, "date", False)
# 'Sat, 03 Jan 2015 22:08:13 GMT'
# Wait, that's the current UTC time..?

# Also print last_modified...
print (k.last_modified)
# 'Fri, 26 Apr 2013 02:41:30 GMT'
6
  • Do you mean print key.date will export the create date? I did test, and seem it is a date after last_modify time. And can you give the full code on how you print key.date?
    – BMW
    Commented Jan 3, 2015 at 3:45
  • Yes, same as I got. So that's not created time.
    – BMW
    Commented Jan 4, 2015 at 3:45
  • Yes. It's supposed to be.. but it's not. Looks like a bug. Commented Jan 4, 2015 at 3:47
  • if you run print key.__dict__, you will get all attributes, but there is no ctime. Seems you are asking an enhancement, which is not exist currently.
    – BMW
    Commented Jan 4, 2015 at 3:49
  • Let us continue this discussion in chat. Commented Jan 4, 2015 at 6:34
1

If you have versioning enabled for your S3 bucket, you can use list_object_versions and find the smallest date for the object you're looking for which should be the date it was created

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.