Always go with CloudFront (CF), especially if you operate in North America or/and Europe.
I've personally never had an edge case, only one I can think of right now is that if you're very price sensitive & you operate in Asia or/and South America which have higher CF data transfer prices.
Why should you prefer CF?
- Data transfer from S3 to CloudFront is free, I think this used to be a big cost
- S3 is in a single geographical region, CF is global, closer to user, faster
- It's possible to "overwhelm" S3 bucket, CF is practically infinite scale
- CF has built-in features to prevent/protect them & you from many types of attacks
- CF is optimized for low latency and high speed with real-world latency measurements
Where do you operate?
North America or/and Europe:
Data transfer is cheaper than S3 which basically cancels out all extra costs from CF.
South America or/and Asia:
It's more expensive. CF data transfer prices are roughly 2x more expensive on average.
Any kind of combination or all:
Create an example scienario, take a look at the prices & do the math. It's probably going to be more expensive but like I listed above: CloudFront has many very important advantages.