5

I have a problem with the csv reader and writer in python. Whenever I try to take one CSV file and par down the number of columns from roughly 37 to 6, this is the kind of output I am getting. Example of one row:

0,"JOHNSON, JOHN J.",JOHN J. JOHNSON,TECH879,INSPECTION TECHNICIAN,MOTOR VEHICLE INSPECTION UNIT

Notice the second field is quoted. This is what I see when I view it in a text editor. I want all of the fields to be quoted. There is a problem. First, my code, one example of it:

import os
import csv
alpha = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z']
with open(os.getcwd()+'/dl1/filepfr062220155.csv','w') as fp:
identity = 0
for i in alpha:
    csvholder = open(os.getcwd()+'/dl1/pfr/'+i+'.csv','rb')
    spamreader = csv.reader(csvholder, delimiter=',')
    for row in spamreader:
        if 'Sort Name' in row[0]:
                pass
        else:
                ###SEE HERE###
                data = [[str(identity),row[0],row[1],row[2],row[3],row[37]]]
                identity+=1
                a = csv.writer(fp,delimiter=',',quotechar='"')
                a.writerows(data)

The problem is, about where I have marked "###SEE HERE###, I've tried wrapping the unquoted fields in quotes like so:

data = [['"'+str(identity)+'"',row[0],'"'+row[1]+'"','"'+row[2]+'"','"'+row[3]+'"','"'+row[37]+'"']]

But that gives me output that looks like this, with triple quotes, and that is not acceptable...like so:

"""4""","JOHNSON, JOHN W.","""JOHN W. JOHNSON""","""TEC5681""","""INSPECTION TECHNICIAN""","""MOTOR VEHICLE INSPECTION UNIT"""

The original form of these rows was likes this:

"NAME","NAME","NAME","NAME" and so on...

So I don't know why only the one field was quoted in the final output. I either need all of the fields quoted or none of them quoted.

I have already tried going with and without "quotechar" on both the reader and the writer, all such combinations.

So if you could either tell me how to retain the original quoting from the original file or how to solve this triple quoting problem so I can reformat manually, that would be great.

1 Answer 1

9

quotechar only indicates what character the writer should use for quoting. It's quote=csv.QUOTE_ALL that you need. Create your writer like this:

a = csv.writer(fp, quoting=csv.QUOTE_ALL)

quoting defaults to csv.QUOTE_MINIMAL, meaning that it will only quote fields if they contain the delimiter, which is why it's only quoting "JOHNSON, JOHN J.".

(Note that I left out delimiter since it already defaults to ',' but of course it won't hurt you to define it explicitly if you like.)

quoting and other dialect parameters are keyword arguments to csv.writer, and are described in the Python documentation for the csv module, under section 13.1.2.

2
  • Thanks. There were other answers with this basic solution, but I guess they didnt word it the way I needed because it didnt work for me then. Putting it in the context of my program helped...thanks again.
    – Mr. A
    Commented Jun 23, 2015 at 0:18
  • still works great!
    – maertsoi
    Commented Oct 7, 2022 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.