1

Why does:

http://example.com/robots.txt

redirect to:

http://www.example.com/mvc/view/robots/live-robots.txt

with these rules:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} ^(www\.)?example\.com
RewriteRule ^robots.txt /mvc/view/robots/live-robots.txt [L]
#.... 20 irrelevant lines for mobile rewrites
# Force the "www."
RewriteCond %{HTTP_HOST} ^example\.com$ [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

loading this however:

http://www.example.com/robots.txt

rewrites the live-robots.txt as expected.

Shouldn't the L flag stop the redirect in both cases and not get to the latter rule?

The L flag can be useful in this context to end the current round of mod_rewrite processing.

https://httpd.apache.org/docs/current/rewrite/flags.html

Current execution paths:

  1. http://example.com/robots.txt
  2. 301 served
  3. http://www.example.com/mvc/view/robots/live-robots.txt

and then

  1. http://www.example.com/robots.txt
  2. 200 (contents of mvc/view/robots/live-robots.txt are served)

I'm pretty sure it isn't a regex issue but here's testing of that too, https://regex101.com/r/eI9aC4/1.

5
  • What do your rewrite logs look like?
    – larsks
    Mar 29, 2016 at 19:13
  • From the access logs or are there re-write specific logs somewhere? Access logs have 80 GET /robots.txt 301 then 80 GET /mvc/view/robots/live-robots.txt 304.
    – chris85
    Mar 29, 2016 at 19:17
  • 1
    Take a look at the documentation (or here for Apache 2.2).
    – larsks
    Mar 29, 2016 at 19:19
  • Try RewriteCond %{HTTP_HOST} ^example\.com [OR] RewriteCond %{HTTP_HOST} ^www\.example\.com (after [OR] of course a linebreak) Mar 29, 2016 at 19:20
  • @larsks thanks for that link. That was useful, had to be in httpd.conf, not .htaccess.
    – chris85
    Mar 29, 2016 at 20:42

1 Answer 1

1

Flag L doesn't stop other rules to execute. It just acts as continue in a while loop and makes mod_rewrite loop to run again.

Your rules need to be reversed in order. In general keep redirect rules before internal rewrite ones:

RewriteEngine On
RewriteBase /

# Force the "www."
RewriteCond %{HTTP_HOST} ^example\.com$ [NC]
RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L,NE]

RewriteCond %{HTTP_HOST} ^(www\.)?example\.com$ [NC]
RewriteRule ^robots\.txt$ mvc/view/robots/live-robots.txt [L,NC]
#.... 20 irrelevant lines for mobile rewrites

Make sure to clear your browser cache for this change.

However if you have Apache 2.4+ then you can use END flag to completely stop all the rules below:

RewriteRule ^robots\.txt$ mvc/view/robots/live-robots.txt [END]
3
  • I'm running 2.2.15 so end is not available to me. There's no way in 2.2 to explicitly have the file stop processing?
    – chris85
    Mar 29, 2016 at 19:37
  • No END was added from Apache 2.4+ onwards. For 2.2 you should keep redirect rules before rewrite rules as I suggested.
    – anubhava
    Mar 29, 2016 at 19:42
  • 1
    Thanks, that answers this question. On to looking into upgrading to 2.4.
    – chris85
    Mar 29, 2016 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.