21

I have the script below, which modifies href attributes in an HTML file (in the future, it will be a list of HTML files in a directory). Using BeautifulSoup I managed to access the tag values and modify them like I want, but I don't know how to save back the changes made to the file.

import os
import re
from bs4 import BeautifulSoup


htmlDoc = open('adding_computer_c.html',"r+")
soup = BeautifulSoup(htmlDoc)

replacements= [ ('_', '-'), ('../tasks/', prefixUrl), ('../concepts/', prefixUrl) ]

for link in soup.findAll('a', attrs={'href': re.compile("../")}):


    newlink=str(link)

    for k, v in replacements:

        newlink = newlink.replace(k, v)

    extrachars=newlink[newlink.find("."):newlink.find(">")]
    newlink=newlink.replace(extrachars,'')


    link=newlink
    print(link)
    ##How do I save the link I have modified back to the HTML file?

print(soup)##prints the original html tree

htmlDoc.close()
0

1 Answer 1

46
newlink = link['href']
# .. make replacements
link['href'] = newlink # store it back

Now print(soup.prettify()) will show changed links. To save the changes to a file:

htmlDoc.close()

html = soup.prettify("utf-8")
with open("output.html", "wb") as file:
    file.write(html)

To preserve original character encoding of the document, you could use soup.original_encoding instead of "utf-8". See Encodings.

3
  • 1
    I did this but but I hit maximum recursion depth. Any idea why? Commented Apr 8, 2015 at 2:25
  • 2
    For Python 3, you must use: open("output.html", "wt") Commented Jul 1, 2017 at 10:22
  • 2
    @BrunoGabuzomeu wrong. soup.prettify('utf-8') returns bytes and therefore 'wb' must be used as shown in the answer.
    – jfs
    Commented Jul 1, 2017 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.