1

I am currently building a scraper to scrape certain information from a website.

For example, I would like to get a restaurant name, address, opening hours & telephone number from a website.

By using curl, I managed to get the data from the website:

    $url = "http://localhost/test.html";
    $ch = curl_init(); 
    curl_setopt($ch, CURLOPT_URL, $url); 
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
    $data = curl_exec($ch); 
    curl_close($ch);

However, I need some ideas on how would I be able to pin point my scraper to the exact location to scrape these information out.

I have tried regular expressions, but was unable to get it to work.

4
  • 1
    Remember that such actions might conflict with copyrights.
    – Peon
    Oct 5, 2012 at 12:51
  • yep i know about that we own the copyright for this project i am dealing with
    – ben
    Oct 5, 2012 at 12:55
  • Well, if you own the project, just create an XML response on the project side, that will generate and return the data needed.
    – Peon
    Oct 5, 2012 at 12:58
  • would have done it if i could shall not explain the long winded story and get going on trying out the answers :) thanks!
    – ben
    Oct 5, 2012 at 13:04

2 Answers 2

3

Use SimpleHTMLDom parser for php:
http://simplehtmldom.sourceforge.net/

Download here:
http://sourceforge.net/projects/simplehtmldom/files/

Documentation here:
http://simplehtmldom.sourceforge.net/manual.htm

That is as I have experience with parsing the best tool for parsing HTML with php...

Also you don't need to use curl for getting content if it is not necessary, for simpleHTMLDom parser just use:

$remote_html = file_get_html("http://www.somesite.com/");
3
  • thanks for the quick reply! i shall take a look over the weekend
    – ben
    Oct 5, 2012 at 12:50
  • You will not make mistake with simpleHTMLDom parser it is really good, I am personaly using it for years now. If you need some specific help feel free to ask...
    – Develoger
    Oct 5, 2012 at 12:52
  • just used this.. really very useful
    – ben
    Oct 13, 2012 at 8:43
1

Take a look at XPath querying: http://php.net/manual/en/domxpath.query.php

I use the equivalant method for website scraping in C#. Same standard employed here. Most excellent.

1
  • thanks for the quick reply! i shall take a look over the weekend
    – ben
    Oct 5, 2012 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.