5

I crawl a table from a web link and would like to rebuild a table by removing all script tags. Here are the source codes.

response = requests.get(url)
soup = BeautifulSoup(response.text)
table = soup.find('table')

for row in table.find_all('tr') :                                                                                                                                                                                                                                                                                                                                                                                                     
    for col in row.find_all('td'):
        #remove all different script tags
        #col.replace_with('') 
        #col.decompose()  
        #col.extract()
        col = col.contents

How can I remove all different script tags? Take the follow cell as an exampple, which includes the tag a, br and td.

<td><a href="http://www.irit.fr/SC">Signal et Communication</a>
<br/><a href="http://www.irit.fr/IRT">Ingénierie Réseaux et Télécommunications</a>
</td>

My expected result is:

Signal et Communication
Ingénierie Réseaux et Télécommunications
1

2 Answers 2

5

You are asking about get_text():

If you only want the text part of a document or tag, you can use the get_text() method. It returns all the text in a document or beneath a tag, as a single Unicode string

td = soup.find("td")
td.get_text()

Note that .string would return you None in this case since td has multiple children:

If a tag contains more than one thing, then it’s not clear what .string should refer to, so .string is defined to be None

Demo:

>>> from bs4 import BeautifulSoup
>>> 
>>> soup = BeautifulSoup(u"""
... <td><a href="http://www.irit.fr/SC">Signal et Communication</a>
... <br/><a href="http://www.irit.fr/IRT">Ingénierie Réseaux et Télécommunications</a>
... </td>
... """)
>>> 
>>> td = soup.td
>>> print td.string
None
>>> print td.get_text()
Signal et Communication
Ingénierie Réseaux et Télécommunications
1
  • 1
    @SparkandShine it's basically the same (text = property(get_text)). You should though use get_text() method.
    – alecxe
    Jul 18, 2015 at 19:35
1

Try calling col.string. That will give you the text only.

3
  • What is the difference between col.text and col.string? Jul 18, 2015 at 17:51
  • Here is the definition for text from the help documents for text: text | Get all child strings, concatenated using the given separator. And for string If this tag has a single string child, return value is that string. If this tag has no children, or more than one child, return value is None. If this tag has one child tag, return value is the 'string' attribute of the child tag, recursively.
    – blasko
    Jul 18, 2015 at 17:57
  • It doesn't work and return None because of multiple children as @alecxe said. Jul 18, 2015 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.