0

I am trying to learn text processing. And using nltk. Trying to follow the NLTK book. When I try to read a text, it is reading it a little different.

import requests
url = "http://www.gutenberg.org/files/2554/2554-0.txt"
response = requests.get(url)
response.text[:25]

enter image description here

How can I read the text without the highlighted part in the image uploaded.

2
  • Try slicing differently? [1:25]? Jun 9, 2018 at 15:09
  • 1
    Your name.... lmao.. Jun 9, 2018 at 15:10

2 Answers 2

0

This is a unicode format that you're seeing here.

What you should do is, convert the unicode string to ascii with ignore if not ascii.

Example:

a=u'\uffefHello World'
print(a.encode('ascii', 'ignore'))
"Hello World"
2
0

The simple answer is to print it and not put it just in the shell:

print(response.text[:25])

Should print:

The Project Gutenberg E8

The shell does repr on the value to find out what it should print

print(repr(response.text[25]))

will again print:

'\ufeffThe Project Gutenberg E8'

Not the answer you're looking for? Browse other questions tagged or ask your own question.