1

I have a folder filled with csv files and each file looks something like this:

TPN,201203,by the congress,3,0.000001559523908541200542298447130
TPN,201312,by the congress,2,0.000001975728179317089554819047995
TPN,201308,by the congress,2,0.000002130556224313481520620588417
CR,200910,by the congress,10,0.000001254229103759238181242376639
CR,200911,by the congress,5,6.974221464170843876612631794E-7
MED,200507,by the congress,2,0.000004113271264069958517659301854

I want a script that goes through each file and finds the minimum date value in that file and then prints every line in that file that includes that date value into a new file (so if two rows have the same date value it should print both). I have this:

import csv
import os
import codecs 
import unicodecsv

folder = '/Users/xyz/Desktop/TextAnalysis/PointsOfOrigin/trigramsdated/'

c = csv.writer(open("trigrampointsoforigin.csv", "a"))

for file in os.listdir (folder):
    with open(os.path.join(folder, file), mode='rU') as f:
        m=min(int(line[1]) for line in unicodecsv.reader(f, encoding='utf-8', errors='replace'))
        f.seek(0)
        for line in unicodecsv.reader(f):
            if int(line[1])==m:
                print line
                c.writerow(line)

print "All done."

But for some weird reason, it just consistently prints the last line in each csv to the "trigramspointsoforign.csv" file.

Any help is much, much appreciated.

3
  • "and each file looks something like this:"?? Feb 16, 2015 at 0:30
  • Oops! Thanks @AshwiniChaudhary , I've added the sample file. Feb 16, 2015 at 0:32
  • If it is only writing the last line then perhaps c.writerow(line) is not indented properly. Feb 16, 2015 at 0:35

1 Answer 1

0

Does line[1] actually print out the date value? In any case, you can avoid one of your inner loops:

for file in os.listdir (folder):
    with open(os.path.join(folder, file), mode='rU') as f:
        minline = [-1,-1] # Some min value
        for linenum, line in enumerate(unicodecsv.reader(f)):
            if int(line[1]) < int(minline[1]):
                # Replace with new minline
                minline = copy.copy(line) 
        print minline
        c.writerow(minline)

You will need to import copy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.