0

I am uploading images to a public directory and I would like to prevent users from downloading the whole lot using wget. Is there a way to do this?

As far as I can see, there must be. I have found a number of sites where, as a public browser, I can download a single image, but as soon as I run wget against them I get a 403 (Forbidden). I have tried using the no-robot argument, but I'm still not able to download them. (I won't name the sites here, for security reasons).

1
  • If you are on nginx, you can rate limit access to folders if i am not mistaken. On apache, this might help: httpd.apache.org/docs/trunk/mod/mod_ratelimit.html -- another option is to write rewrite rules and block direct access and control access with php sessions and ip adres, but this could be a long shot.
    – Nitin
    Jul 16, 2016 at 8:31

1 Answer 1

2

You can restrict access using user-agent string, see apache 2.4 mod_authz_core for example. Wget also respects robots.txt directives by default. This should repent any casual user.

However, a careful look into wget manual will let to bypass these restrictions. Wget also lets to add random delays between requests, so even advanced techniques based on access pattern analysis may be bypassed.

So the proper way is to mess with wget link/reference recognition engine. Namely, the content you want to keep unmirrored should be loaded dynamically using javascript and the urls must be encoded in a way that would require js code to decode. This would protect your content, but would require to manually provide unobfuscated version for web bots you want to index your site, such as google bot (and no, it is not the only one one should care about). Also, some people do not run js scripts by default (esoteric browsers, low-end machines, mobile devices may demand such policy).

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.