61

I'm trying to get the content "My home address" using the following but got the AttributeError:

address = soup.find(text="Address:")
print address.nextSibling

This is my HTML:

<td><b>Address:</b></td>
<td>My home address</td>

What is a good way to navigate down td tag and pull the content?

4 Answers 4

111

The problem is that you have found a NavigableString, not the <td>. Also nextSibling will find the next NavigableString or Tag so even if you had the <td> it wouldn't work the way you expect.

This is what you want:

address = soup.find(text="Address:")
b_tag = address.parent
td_tag = b_tag.parent
next_td_tag = td_tag.findNext('td')
print next_td_tag.contents[0]

Or more concise:

print soup.find(text="Address:").parent.parent.findNext('td').contents[0]

Actually you could just do

print soup.find(text="Address:").findNext('td').contents[0]

Since findNext just calls next over and over again, and next finds the next element as parsed repeatedly until it matches.

0
21

Try this if you use bs4:

print soup.find(string="Address:").find_next('td').contents[0]
16

I don't know if this was possible in 2011 but in 2021 I'd recommend you to do it using find_next_sibling() like this:

address = soup.find(text="Address:")
b = address.parent
address_td = b.parent
target_td = address_td.find_next_sibling('td')

The accepted answer works in your case but it would not work if you had something like:

<div>
  <div><b>Address:</b><div>THE PROBLEM</div></div>
  <div>target</div>
</div>

You'd end up with <div>THE PROBLEM</div> instead of <div>target</div>.

1
3

You can use findChildren pretty easily to iterate over the td's in a table, assuming it's in a table. You could find the table first, ideally:

table = soup.find('table')
>>> for td in table.findChildren('td'):
...:     print td.text
...:     
...:     
Address:
My home address

Or you could search for address, and get the table container:

>>> import re
>>> search = re.compile('Address')
>>> table = soup.find(text=search).parent.parent.parent
1
  • There're a lot of tables so finding tables then td don't look like a good choice.
    – ready
    May 14, 2011 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.