I have this html:
<div id="content">
<h1>Title 1</h1><br><br>
<h2>Sub-Title 1</h2>
<br><br>
Description 1.<br><br>Description 2.
<br><br>
<h2>Sub-Title 2</h2>
<br><br>
Description 1<br>Description 2<br>
<br><br>
<div class="infobox">
<font style="color:#000000"><b>Information Title</b></font>
<br><br>Long Information Text
</div>
</div>
I want to get all text in <div id="content"> with XPath in Scrapy but excluding <div class="infobox">'s content, so the expected result is like this:
Title 1
Sub-Title 1
Descripton 1.
Descripton 2.
Sub-Title 2
Descripton 1.
Descripton 2.
But I haven't reached the excluding part yet, I'm still struggling to grab the text from the <div id="content">.
I have tried this:
response.xpath('//*[@id="content"]/text()').extract()
But it only returns Description 1. and Description 2. from both Sub-Title.
Then I tried:
response.xpath('//*[@id="content"]//*/text()').extract()
It only returns Title 1, Sub-Title 1, Sub-Title 2, Information Title, and Long Information Text.
So there are two questions here:
- How could I get all of children text from
contentdiv? - How to exclude the
infoboxdiv from the selection?