14

I want to read a csv file and process some columns but I keep getting issues. Stuck with the following error:

Traceback (most recent call last):
  File "C:\Users\Sven\Desktop\Python\read csv.py", line 5, in <module>
    for row in reader:
  File "C:\Python34\lib\codecs.py", line 313, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 446: invalid start byte
>>> 

My Code

import csv
with open("c:\\Users\\Sven\\Desktop\\relaties 24112014.csv",newline='', encoding="utf8") as f:
    reader = csv.reader(f,delimiter=';',quotechar='|')
    #print(sum(1 for row in reader))
    for row in reader:
        print(row)
        if row:
            value = row[6]
            value = value.replace('(', '')
            value = value.replace(')', '')
            value = value.replace(' ', '')
            value = value.replace('.', '')
            value = value.replace('0032', '0')
            if len(value) > 0:
                print(value + ' Length: ' + str(len(value)))

I'm a beginner with Python, tried googling, but hard to find the right solution.

Can anyone help me out?

6
  • 1
    can you share your file csv file or parts of it ? Commented Nov 30, 2014 at 23:06
  • 6
    Are you sure that your file is really encoded with UTF-8? Try 'cp1252' if the file comes from Excel in a western country. All supported encodings: docs.python.org/2/library/codecs.html#standard-encodings Commented Nov 30, 2014 at 23:06
  • 1
    Why are you opening the file as UTF-8? Are you certain that that's the correct codec for that file? Commented Nov 30, 2014 at 23:08
  • 2
    89 is not a valid UTF8 byte, so the "error" is correct and your file is not UTF8-encoded. Case closed as far as I am concerned.
    – Jongware
    Commented Nov 30, 2014 at 23:13
  • 2
    Try a `"latin1" encoding were all byte values will be legal.
    – martineau
    Commented Nov 30, 2014 at 23:19

3 Answers 3

8

The first byte of a .PNG file is 0x89. Not saying that is your problem, but the .PNG header is specifically designed so that it is NOT accidentally interpreted as text.

Why you would have a .csv file that is actually a .png I don't know. But it definitely could happen if someone accidentally renamed the file. On windows 10 every once and a while I accidentally mass-rename files by accident because of their stupid checkbox feature. Why Microsoft decided desktop machines having identical UI controls to tablets was I good idea... I don't know.

1
  • I am working with a file which is png, i.e. I know that it's png and it's supposed to be that, but I still get this error. Commented Nov 8, 2021 at 17:36
7

This is the most important clue:

invalid start byte

\x89 is not, as suggested in the comments, an invalid UTF-8 byte. It is a completely valid continuation byte. Meaning if it follows the correct byte value, it codes UTF-8 correctly:

http://hexutf8.com/?q=0xc90x89

So either you (1) do not have UTF-8 data as you expect, or (2) you have some malformed UTF-8 data. The Python codec is simply letting you know that it encountered \x89 in the wrong order in the sequence.

(More on continuation bytes here: http://en.wikipedia.org/wiki/UTF-8#Codepage_layout)

0
4

I was also getting the similar error when trying to read or upload the following kinds of files:

  1. CSV File
  2. JPEG File
  3. PNG File
  4. Zip File

The best way to avoid error like:

  1. 'utf-8' codec can't decode byte 0x89
  2. 'utf-8' codec can't decode byte 0xff

is to read these files as Bytes. When you treat them as byte then you need not provide any encoding value here. So when you open them you should specify:

with open(file_path, 'rb') as file:

Or in your case, the code should be something like:

import csv

with open("c:\\Users\\Sven\\Desktop\\relaties 24112014.csv", newline='', 'rb') as f:

reader = csv.reader(f,delimiter=';',quotechar='|')
1
  • this fixed me upload zip and jpeg files Commented Dec 14, 2021 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.