2

I have a specific page that I have scraped some data from, but to gather more information I need to follow another href link to gather more data for that specific item, and I don't know how.

Here is what I have so far:

       router.get("/:id",function(req,res){
       var url = "www.someurl.com"

       request(url, function (error, response, html){
       if (!error && response.statusCode == 200){
        $ = cheerio.load(html);

       $('div.ProductDetails').each(function(){// function for details
          var self = this;
          var a = $(this).children().children();
          details = a.attr('href');

          details = {
          details:details
      };

This gives me a link to the details section with the actual details. I just need to know how to do a scrape inside of a scrape.

1 Answer 1

1

The same way you did the first time, details.details is the url in the <a> tag.

So you need to make a request to that url

request(details.details, function(err, res, html) { /* code */ })

and then use cheerio to extract data.

1
  • im was trying to do it this way by doing another request to details but im getting an error saying details is not defined. i need a way to set details into the router function.
    – Simon Cho
    Mar 1, 2016 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.