22

I am very new to Python so please forgive the following basic code and problem, but I have been trying to figure out what is causing the error I am getting (I have even looked at similar threads on S.O.) but can't get past my issue.

Here is what I am trying to do:

  • loop through a folder of CSV files
  • search for a 'keyword' and delete all lines containing the 'keyword'
  • save output to a separate folder

Here is my code:

import os, fnmatch
import shutil

src_dir = "C:/temp/CSV"
target_dir = "C:/temp/output2"
keyword = "KEYWORD"

for f in os.listdir(src_dir):
    os.path.join(src_dir, f)
    with open(f):
        for line in f:
            if keyword not in line:
                write(line)
                shutil.copy2(os.path.join(src_dir, f), target_dir)

Here is the error I am getting:

IOError: [Errno 2] No such file or directory: 'POS_03217_20120309_153244.csv'

I have confirmed that the folder and file do exist. What is causing the IOError and how to I resolve it? Also, is there anything else wrong with my code that would prevent me from performing the entire task?

1
  • Fundamentally, the problem here was a typo: doing os.path.join(src_dir, f) does not change the value of f, so open(f) does not see the src_dir path. Commented Sep 4, 2022 at 2:03

5 Answers 5

20

Hmm, there are a few things going wrong here.

for f in os.listdir(src_dir):
    os.path.join(src_dir, f)

You're not storing the result of join. This should be something like:

for f in os.listdir(src_dir):
    f = os.path.join(src_dir, f)

This open call is is the cause of your IOError. (Because without storing the result of the join above, f was still just 'file.csv', not 'src_dir/file.csv'.)

Also, the syntax:

with open(f): 

is close, but the syntax isn't quite right. It should be with open(file_name) as file_object:. Then, you use to the file_object to perform read or write operations.

And finally:

write(line)

You told python what you wanted to write, but not where to write it. Write is a method on the file object. Try file_object.write(line).

Edit: You're also clobbering your input file. You probably want to open the output file and write lines to it as you're reading them in from the input file.

See: input / output in python.

1
  • 1
    Thanks all for the great feedback! I'll look over the suggestions and see if I can get my code to work.
    – Keith
    Commented Mar 19, 2012 at 6:08
11

Even though @Ignacio gave you a straightforward solution, I thought I might add an answer that gives you some more details about the issues with your code...

# You are not saving this result into a variable to reuse
os.path.join(src_dir, f)
# Should be
src_path = os.path.join(src_dir, f)

# you open the file but you dont again use a variable to reference
with open(f)
# should be
with open(src_path) as fh

# this is actually just looping over each character 
# in each result of your os.listdir
for line in f
# you should loop over lines in the open file handle
for line in fh

# write? Is this a method you wrote because its not a python builtin function
write(line)
# write to the file
fh.write(line)
0
5

Um...

with open(os.path.join(src_dir, f)) as fin:
    for line in fin:

Also, you never output to a new file.

4

I solved the problem like so:

src_dir = "C:\\temp\\CSV\\"
target_dir = "C:\\temp\\output2\\"
keyword = "KEYWORD"

for f in os.listdir(src_dir):
    file_name = os.path.join(src_dir, f)
    out_file = os.path.join(target_dir, f)
    with open(file_name, "r+") as fi, open(out_file, "w") as fo:
        for line in fi:
            if keyword not in line:
                fo.write(line)
1
  • 1
    You do not need the + in the line open(file_name, "r+"). According to the docs: "'r+' opens the file for both reading and writing." In this case, you only want to read from file_name. Additionally, I suggest (although this is mostly preference) you combine your last three lines to fo.write('\n'.join(line for line in fi if keyword not in line)). Doing this reduces the number of calls to f.write().
    – pzp
    Commented Nov 8, 2015 at 3:28
0

I got this error and fixed by appending the directory path in the loop. script not in the same directory as the files. dr1 ="~/test" directory variable

 fileop=open(dr1+"/"+fil,"r")

Not the answer you're looking for? Browse other questions tagged or ask your own question.