48

I'd like to extract the content Hello world. Please note that there are multiples <table> and similar <td colspan="2"> on the page as well:

<table border="0" cellspacing="2" width="800">
  <tr>
    <td colspan="2"><b>Name: </b>Hello world</td>
  </tr>
  <tr>
...

I tried the following:

hello = soup.find(text='Name: ')
hello.findPreviousSiblings

But it returned nothing.

In addition, I'm also having problem with the following extracting the My home address:

<td><b>Address:</b></td>

<td>My home address</td>

I'm also using the same method to search for the text="Address: " but how do I navigate down to the next line and extract the content of <td>?

4 Answers 4

50

The contents operator works well for extracting text from <tag>text</tag> .


<td>My home address</td> example:

s = '<td>My home address</td>'
soup =  BeautifulSoup(s)
td = soup.find('td') #<td>My home address</td>
td.contents #My home address

<td><b>Address:</b></td> example:

s = '<td><b>Address:</b></td>'
soup =  BeautifulSoup(s)
td = soup.find('td').find('b') #<b>Address:</b>
td.contents #Address:
21

use next instead

>>> s = '<table border="0" cellspacing="2" width="800"><tr><td colspan="2"><b>Name: </b>Hello world</td></tr><tr>'
>>> soup = BeautifulSoup(s)
>>> hello = soup.find(text='Name: ')
>>> hello.next
u'Hello world'

next and previous let you move through the document elements in the order they were processed by the parser while sibling methods work with the parse tree

6
  • It returns nothing. hello = soup.find(text='Name: ') hello.next
    – ready
    May 14, 2011 at 2:35
  • 2
    does 'Name:' appear anywhere else in the document? May 14, 2011 at 2:45
  • Sorry for the multiple comments as i didn't know the return key actually posted the comment. I was thinking if there's a better method to do this just in case if there's a similar text which is "Name: ".
    – ready
    May 14, 2011 at 3:05
  • you can check for hello.parent.parent.name or hello.parent.parent.attrs or anything else you can latch onto May 14, 2011 at 3:20
  • Do you mind giving a short example to illustrate parent.parent.name?
    – ready
    May 14, 2011 at 3:22
5

Use the below code to get extract text and content from html tags with python beautifulSoup

s = '<td>Example information</td>' # your raw html
soup =  BeautifulSoup(s) #parse html with BeautifulSoup
td = soup.find('td') #tag of interest <td>Example information</td>
td.text #Example information # clean text from html
2
  • Thank you for this code snippet, which might provide some limited, immediate help. A proper explanation would greatly improve its long-term value by showing why this is a good solution to the problem and would make it more useful to future readers with other, similar questions. Please edit your answer to add some explanation, including the assumptions you've made.
    – wscourge
    Nov 27, 2020 at 7:35
  • I decided to use .text since the user wanted to extract plain text from the html. After the user parses the the html with the Beautiful soup python library, he can use 'id', "class" or any other identifier to find the tag or html element of interest and after doing this, if he wants plain text within any of the selected tag, he can use .text on the tag as I decribed above Sep 9, 2021 at 18:49
2
from bs4 import BeautifulSoup, Tag

def get_tag_html(tag: Tag):
    return ''.join([i.decode() if type(i) is Tag else i for i in tag.contents])
1
  • 2
    Hello and welcome to SO! While this code may answer the question, providing additional context regarding how and/or why it solves the problem would improve the answer's long-term value. Please read the tour, and How do I write a good answer? Dec 23, 2020 at 10:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.