1

I have a simple php page views counter and was wondering how can I stop spiders and bots as being counted as views specifically google bot?

2
  • Not to discourage you from writing your own script, but Google Analytics is a very powerful tool if you're looking for a counter. If you're looking to create one by yourself, please ignore this.
    – user183037
    Feb 3, 2011 at 6:14
  • possible duplicate of Tell bots apart from human visitors for stats?
    – Quentin
    Feb 3, 2011 at 7:06

2 Answers 2

1

I added this script on my site:

# Spiders list from http://linksku.com
$spiders = array('aspseek','abachobot','accoona','acoirobot','adsbot','alexa','alta vista','altavista','ask jeeves','baidu','crawler','croccrawler','dumbot','estyle','exabot','fast-enterprise','fast-webcrawler','francis','geonabot','gigabot','google','heise','heritrix','ibm','iccrawler','idbot','ichiro','lycos','msn','msrbot','majestic-12','metager','ng-search','nutch','omniexplorer','psbot','rambler','seosearch','scooter','scrubby','seekport','sensis','seoma','snappy','steeler','synoo','telekom','turnitinbot','voyager','wisenut','yacy','yahoo');
foreach($spiders as $spider) if(stripos($_SERVER['HTTP_USER_AGENT'], $spider) !== false) {
    $_SERVER['HTTP_CRAWLER'] = true;
    break;
}
if(!isset($_SERVER['HTTP_CRAWLER'])) $_SERVER['HTTP_CRAWLER'] = false;

Then you can just check the value of $_SERVER['HTTP_CRAWLER'] and block your script from counting the hit.

1
  • 1
    The spiders list is probably better off as an external include, there are at least 52 spiders that I know of. This might be a list you would want to update as and when you recognize new spiders.
    – user183037
    Feb 3, 2011 at 9:08
0

An easy method is to implement the pagecounter as image script:

<img src="counter.php" width="1" height="1" alt="Oh I'm just counting">

And mark that URL as inaccessible to spiders/crawlers via robots.txt, where you can either use * for all, or just Googlebot to exclude it:

User-agent: *
Disallow: /counter.php

The alternative would be to just check for stristr($_SERVER["HTTP_USER_AGENT", "Googlebot") in the script, and simply not up the counter then.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.