I have a problem with the csv reader and writer in python. Whenever I try to take one CSV file and par down the number of columns from roughly 37 to 6, this is the kind of output I am getting. Example of one row:
0,"JOHNSON, JOHN J.",JOHN J. JOHNSON,TECH879,INSPECTION TECHNICIAN,MOTOR VEHICLE INSPECTION UNIT
Notice the second field is quoted. This is what I see when I view it in a text editor. I want all of the fields to be quoted. There is a problem. First, my code, one example of it:
import os
import csv
alpha = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z']
with open(os.getcwd()+'/dl1/filepfr062220155.csv','w') as fp:
identity = 0
for i in alpha:
csvholder = open(os.getcwd()+'/dl1/pfr/'+i+'.csv','rb')
spamreader = csv.reader(csvholder, delimiter=',')
for row in spamreader:
if 'Sort Name' in row[0]:
pass
else:
###SEE HERE###
data = [[str(identity),row[0],row[1],row[2],row[3],row[37]]]
identity+=1
a = csv.writer(fp,delimiter=',',quotechar='"')
a.writerows(data)
The problem is, about where I have marked "###SEE HERE###, I've tried wrapping the unquoted fields in quotes like so:
data = [['"'+str(identity)+'"',row[0],'"'+row[1]+'"','"'+row[2]+'"','"'+row[3]+'"','"'+row[37]+'"']]
But that gives me output that looks like this, with triple quotes, and that is not acceptable...like so:
"""4""","JOHNSON, JOHN W.","""JOHN W. JOHNSON""","""TEC5681""","""INSPECTION TECHNICIAN""","""MOTOR VEHICLE INSPECTION UNIT"""
The original form of these rows was likes this:
"NAME","NAME","NAME","NAME" and so on...
So I don't know why only the one field was quoted in the final output. I either need all of the fields quoted or none of them quoted.
I have already tried going with and without "quotechar" on both the reader and the writer, all such combinations.
So if you could either tell me how to retain the original quoting from the original file or how to solve this triple quoting problem so I can reformat manually, that would be great.