25

I'm trying to cache static content which are basically inside the paths below in virtual server configuration. For some reason files are not being cached. I see several folders and files inside the cache dir but its always something like 20mb no higher no lower. If it were caching images for example would take at least 500mb of space.

Here is the nginx.conf cache part:

** nginx.conf **
proxy_cache_path /usr/share/nginx/www/cache levels=1:2 keys_zone=static$
proxy_temp_path /usr/share/nginx/www/tmp;
proxy_read_timeout 300s;

Heres the default virtual server.

**sites-available/default**
server {
    listen   80; 

    root /usr/share/nginx/www;
    server_name myserver;
    access_log /var/log/nginx/myserver.log main;
    error_log /var/log/nginx/error.log;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    location ~* ^/(thumbs|images|css|js|pubimg)/(.*)$ {
            proxy_pass http://backend;
            proxy_cache static;
            proxy_cache_min_uses 1;
            proxy_cache_valid 200 301 302 120m;
            proxy_cache_valid 404 1m;
            expires max;
    }

    location / {
            proxy_pass http://backend;
    }
}
2
  • Have tou tryed to turn on error logging? Feb 10, 2012 at 17:47
  • 1
    yes. nothing conclusive. Feb 10, 2012 at 18:07

5 Answers 5

54

Make sure your backend does not return Set-Cookie header. If Nginx sees it, it disables caching.

If this is your case, the best option is to fix your backend. When fixing the backend is not an option, it's possible to instruct Nginx to ignore Set-Cookie header

proxy_ignore_headers "Set-Cookie";
proxy_hide_header "Set-Cookie";

See the documentation

proxy_ignore_header will ensure that the caching takes place. proxy_hide_header will ensure the Cookie payload is not included in the cached payload. This is important to avoid leaking cookies via the NGINX cache.

8
  • 9
    actually it was not set-cookie but cache-control. I found out that IIS was actually caching static files and the response to nginx was to not cache those files prolly because it was already caching it. Once i removed those files from iis cache it worked. :) Feb 10, 2012 at 19:14
  • This worked for me on nginx in front of a django backend. Otherwise it wasn't caching at all
    – dlrust
    Feb 26, 2014 at 18:49
  • 8
    If I understand correctly then you almost always want to also set proxy_hide_header "Set-Cookie" or you will be serving all your visitors the cookies that were generated for the first user who requested the resource. Mar 12, 2014 at 15:38
  • 5
    @alexander I updated your answer, proxy_ignore_headers without proxy_hide_header is a huge security risk. May 22, 2015 at 6:01
  • 8
    in case someone else has the same problem i had: proxy_buffering off; will prevent caching
    – Kaworu
    Oct 1, 2015 at 14:51
21

I would like to add that multiple configuration options and combinations can disable proxy caching in Nginx. Unfortunately this is poorly documented.

In my configuration I set proxy_buffering on and it enabled caching as expected.

6
  • 2
    So, how is proxy_buffering on related, may you explain please?
    – Meglio
    Mar 18, 2016 at 8:01
  • 3
    I was looking for some official documentation on the relation between proxy buffering and proxy caching, but I could not find any good resources to cite here. I added this answer, because switching proxy_buffering off simply disables any configured caching behaviour, without a warning or anything else. My guess is that without buffering the nginx has no way to store and thus cache a response from the server it is proxying to. This answer hints some things, maybe even ask Tero Kilkanen who gave the answer: serverfault.com/a/692585/144118
    – Overbryd
    Mar 18, 2016 at 10:57
  • 1
    Made my day. I've spent 3 days! To figure out what the **** wrong with our config. Big, great thank you @Overbryd , from bottom of my heart.
    – dr.dimitru
    Apr 3, 2016 at 13:37
  • @Overbryd gj! was fighting nginx for 2 days, and your answers is freaking awesome! Dec 7, 2016 at 10:47
  • 2
    Opened ticket to document this crap: trac.nginx.org/nginx/ticket/2048
    – pva
    Sep 22, 2020 at 12:17
15

after going through multiple answers and comments, i found this configuration finally works:

10m = 10mb key cache, max_size to 2GB, inactive=120m (refresh from source after 120minutes of inactive), use_temp_path=off (to reduce io)

proxy_cache_valid - cache status of 200 and 302 for 60 minutes

proxy_cache_path /tmp/cache levels=1:2 keys_zone=default_cache:10m max_size=2g
                 inactive=120m use_temp_path=off;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_valid 200 302 60m;

server {
    listen       80;
    server_name  example.com;

    # https://www.nginx.com/blog/nginx-caching-guide
    location / {
        proxy_cache default_cache;
        proxy_buffering on;
        proxy_ignore_headers Expires;
        proxy_ignore_headers X-Accel-Expires;
        proxy_ignore_headers Cache-Control;
        proxy_ignore_headers Set-Cookie;

        proxy_hide_header X-Accel-Expires;
        proxy_hide_header Expires;
        proxy_hide_header Cache-Control;
        proxy_hide_header Pragma;

        add_header X-Proxy-Cache $upstream_cache_status;
        proxy_pass http://ip-of-host:80;

        #set            $memcached_key "$uri?$args";
        #memcached_pass 127.0.0.1:11211;
        # error_page     404 502 504 = @fallback;
    }
}
2
  • 2
    Good and clear answer. I followed and fixed my issues.
    – Hoang
    Mar 26, 2021 at 10:32
  • In my case my upstream server was setting Cache-Control private. "By default, NGINX respects other directives in the Cache-Control header: it does not cache responses when the header includes the Private, No-Cache, or No-Store directive". I was able to remove "private" and did not need to use proxy_hide_header but this answer led me to the right place Mar 13 at 18:23
1

Another cause which i just ran into is with the nginx server system clock being in the future (or has too much drift into the future against the clock of the origin server that is defining the Expire headers), it will just consider content not to be cached since it is already expired (according to it's own time reference). The server was not syncing against NTP...

From https://www.nginx.com/blog/nginx-caching-guide/ :

NGINX caches a response only if the origin server includes either the Expires header with a date and time in the future, or the Cache-Control header with the max-age directive set to a non‑zero value.

0

For what it's worth, my experience is that nginx does not always cache things where you tell it to.

For example, on centos7, with the configuration option

proxy_cache_path /tmp/my_nginx_cache levels=1:2 keys_zone=my_zone:10m inactive=24h max_size=1g;

nginx actually caches the files at:

/tmp/systemd-private-phJlfG/tmp/my_nginx_cache
5
  • 2
    That could possibly be systemd, or a patch from RedHat. Try systemctl cat nginx.service and see if there is an option in there to give chroot it or something similar
    – Jason
    Apr 19, 2016 at 12:35
  • 1
    Most probably you have to specify proxy_cache_path /tmp/my_nginx_cache use_temp_path=off ..., otherwise nginx appends as prefix proxy_temp_path, which is set to /tmp/systemd-private-phJlfG/ in your case. May 23, 2017 at 11:36
  • Confirmed - 'PrivateTmp=true' in the nginx.service systemd unit file from RHEL nginx rpm. If you put your cache under /tmp, it won't be persistent in this setup (restart of nginx will result in it getting a fresh new cache dir). Nov 19, 2017 at 21:39
  • There seems to be a known issue with Centos 7 and Nginx caching
    – TrojanName
    Jun 7, 2018 at 13:42
  • @JoshuaMiller so how did you solve this issue? I have the same issue.
    – Khom Nazid
    Oct 27, 2018 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.