0

I'm new to python and web scraping. I've run into a problem and I can't seem to figure out how to fix it.

I am trying to copy the text from a website and when I print everything to the terminal, it pops out okay.

The issue is when I try to save it to a text file. I get this error:

     Traceback (most recent call last):
     File "t3.py", line 43, in <module>
     Case_info.write(item.text+"\n")
     UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-17: ordinal not in range(128)

I'm not sure how to fix it. My code for this particular section is below:

import mechanize
import requests
from bs4 import BeautifulSoup
import re
.
.
.

def main():
   b=mechanize.Browser()
   return b.open(mainlink+case)

html_text=main().read()

soup = BeautifulSoup(html_text)

g_data=soup.findAll("div",{"class":"Print"})

Case_info=open("case_info.txt", "w+")

for item in g_data:
  print item.text
  Case_info.write(item.text+"\n")

Case_info.close()

Also, What caused some of my confusion is this portion of code that did almost the exact same thing perfectly.

.
.
links = soup.findAll('a',href=True)
.
.
for link in links:
  if re.findall('keyword',link['href']):
   #print link.text, link['href'] 
   files.write(link['href']+"\n")
   names.write(link.text+"\n")
.
.

Hopefully someone can help clear things up.

1 Answer 1

0

using the unidecode package, you can eliminate the error.

from unidecode import unidecode
for item in g_data:
  print item.text
  Case_info.write(unidecode (item.text+"\n"))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.