0

I am trying to scrape information from this website https://www.gumtree.co.za, however I am not sure how to get the URL of the property.

This is what I have:

class GumtreeSpider(scrapy.Spider):
    name = "gumtree"
    start_urls = ['https://www.gumtree.co.za/s-house-rentals-flat-rentals-offered/cape-town/v1c9071l3100006p1',
                  'https://www.gumtree.co.za/s-houses-flats-for-sale/cape-town/v1c9074l3100006p1']

    def parse(self, response):

        for prop in response.css('div.tileV1'):
            link = 'https://www.gumtree.co.za' + prop.css('div.title  a.tile-title-text::attr(href)').get()

I have tried multiple combinations however I cannot seem to get it right. Any suggestions? Thank you!

1

1 Answer 1

0

What are you trying to scrape exactly in the page? The ad list in the carousel in the top?

If yes, you need to look at the Javascript XHR requests, they are loaded dynamically so you have to inspect the different requests to find what you want.

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.