0

As a part of the requirement I need to find out whether a domain is parked or not. As there is no efficient way to find out this, I'm going to check the DOM for phrases like "Buy this domain", "may be for sale".. etc.

I found some parked domains which can be accessed through browser, but cannot able to get them using file_get_contents.

Example

$url = 'http://buythisdomain.com/'
$get = file_get_contents($url);

For the above got the following message at output.

Warning: file_get_contents(http://buythisdomain.com/): failed to open stream: HTTP request failed!

But able to access the same URL via browser.I tried fopen method too, but same result. Is there any way to achieve this?

3
  • The site is probably blocking requests that don't contain any browser signature, perhaps because they don't want you accessing them from a script
    – Mark Baker
    Jan 27, 2015 at 11:48
  • @MarkBaker oh okay. Thanks. so is there any way to achieve this? Jan 27, 2015 at 11:55
  • 1
    Yes, create a filestream context as shown in @Brain's answer
    – Mark Baker
    Jan 27, 2015 at 11:58

1 Answer 1

5

Many sites, not only parked domains use some mechanism to block basic requests without valid browser headers.

Try to use stream context that send that required headers like a browser like this

$url = "http://buythisdomain.com/"
$context = stream_context_create(array(
    'http' => array(
        'method' => "GET",
        'header' =>
            "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" .
            "Accept-Language: en-US,en;q=0.8\r\n".
            "Keep-Alive: timeout=3, max=10\r\n",
            "Connection: keep-alive",
        'user_agent' => "User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11",
        "ignore_errors" => true,
        "timeout" => 3
    )
));
file_get_contents($url, false, $context);
5
  • Just need small confirmation, It's not working for some domains ex: http://makeoffer.in' Getting 404 Not Found response at firebug. but it is directly accessable through browser. any idea? Jan 27, 2015 at 12:15
  • @KrishnaMohan I've tested that domain also, works fine. If you get a 404 in the browser than it's not because of the script because your are having that 404 in your browser? I'm a bit confused. Jan 27, 2015 at 12:19
  • yeah!, I think that domain has meta refresh in it's DOM. so when I test my code at browser, getting first response HTML like ` <meta HTTP-EQUIV="REFRESH" content="0; url=/Zm9yY2VTUg">` then immediate response became like http://localhost:81/glp?r=&u=http%3A%2F%2Flocalhost%3A81%2Fparse%2Ftest.php so I think that causes object not found issue. Jan 27, 2015 at 12:31
  • @KrishnaMohan Sure that sounds reasonable, but the script should work as expected. Jan 27, 2015 at 12:39
  • I am still getting "Not Found". The url I am trying to fetch is https://www.auction.com/residential/CA/active_lt/land_pt/resi_sort_v2_st/y_nbs/. What can be going wrong? Please help Mar 29, 2019 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.