2

I am trying to get the commonPrefixes of a bucket from amazon s3. I am using following code to get the list of all objects:

ObjectListing listing = s3Client.listObjects(new ListObjectsRequest().withBucketName(bucket).withPrefix("used/").withDelimiter("/"));
for (String name : listing.getCommonPrefixes()) 
{
    System.out.println(name);
    objectNames.add(name);
}
System.out.println("\n\n\nSize: " + objectNames.size());

But it just prints some of the commonPrefixes. There are more than 2000 prefixes, but it just prints 950. How can I get all the prefixes?

2
  • Do you mean objectNames.size() > 2000 but only 950 lines get printed? Sep 7, 2017 at 12:20
  • No, there are more than 2000 keys in that bucket, but getCommonPrefixes() gives just 950 keys.
    – backslashN
    Sep 7, 2017 at 12:52

1 Answer 1

6

Your ObjectListing is only going to contain at most 1000 objects at a time. When you call getCommonPrefixes it is only returning the common prefixes for those 1000 objects. You need to check the ObjectListing's isTruncated() method to determine if there are more records, and then use the getNextMarker() method along with subsequent listObjects calls to get the remaining object records, and the common prefixes for those objects.

1
  • 1
    wrapping the code in while (listing.isTruncated()) did the task. And listing = s3Client.listNextBatchOfObjects(listing) brought in the next 1000 prefixes.
    – backslashN
    Sep 8, 2017 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.