1

I am dealing with dirty text data (and not with valid html). I am doing natural language processing and short code snippets shouldn't be removed because they can contain valuable information while long code snippets don't.

Thats why I would like to remove text between code tags only if the content that will be removed has character length > n.

Let's say the number of allowed characters between two code tags is n <= 5. Then everything between those tags that is longer than 5 characters will be removed.

My approach so far deletes all of the code characters:

text = "This is a string <code>1234</code> another string <code>123</code> another string <code>123456789</code> another string."
text = re.sub("<code>.*?</code>", '', text)
print(text)

Output: This is a string  another string  another string  another string.

The desired output:

"This is a string <code>1234</code> another string <code>123</code> another string another string."

Is there a way to count the text length for all of the appearing <code ... </code> tags before it will actually be removed?

4
  • 1
    Try just "<code>[^>]{5,}</code>" Feb 11, 2022 at 14:29
  • The result is also 'This is a string another string another string another string.'
    – Daniel
    Feb 11, 2022 at 14:31
  • 1
    See ideone.com/6BnNGn. You must have a bug somewhere in your code. Feb 11, 2022 at 14:33
  • It works. You are right. If you want you can provide an answer and I can accept it. I don't now of this is a good practice. I am pretty new to Stackoverflow.
    – Daniel
    Feb 11, 2022 at 14:36

1 Answer 1

2

In Python, BeautifulSoup is often used to manipulate HTML/XML contents. If you use this library, you can use something like

from bs4 import BeautifulSoup
soup = BeautifulSoup(content,"html.parser")
text = "This is a string <code>1234</code> another string <code>123</code> another string <code>123456789</code> another string."
soup = BeautifulSoup(text,"html.parser")
for code in soup.find_all("code"):
    if len(code.encode_contents()) > 5: # Check the inner HTML length
        code.extract()                  # Remove the node found

print(str(soup))
# => This is a string <code>1234</code> another string <code>123</code> another string  another string.

Note that here, the length of the inner HTML part is taken into account, not the inner text.

With regex, you can use a negated character class pattern, [^<], to match any char other than <, and apply a limiting quantifier to it. If all longer than 5 chars should be removed, use {6,} quantifier:

import re
text = "This is a string <code>1234</code> another string <code>123</code> another string <code>123456789</code> another string."
text = re.sub(r'<code>[^>]{6,}</code>', '', text)
print(text)
# => This is a string <code>1234</code> another string <code>123</code> another string  another string.

See this Python demo.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.