88

Hi: I am trying to use the Pandas DataFrame.to_csv method to save a dataframe to a csv file:

filename = './dir/name.csv'

df.to_csv(filename)

However I am getting the error:

IOError: [Errno 2] No such file or directory: './dir/name.csv'

Shouldn't the to_csv method be able to create the file if it doesn't exist? This is what I am intending for it to do.

6 Answers 6

141

to_csv does create the file if it doesn't exist as you said, but it does not create directories that don't exist. Ensure that the subdirectory you are trying to save your file within has been created first.

I often do something like this in my work:

import os

outname = 'name.csv'

outdir = './dir'
if not os.path.exists(outdir):
    os.mkdir(outdir)

fullname = os.path.join(outdir, outname)    

df.to_csv(fullname)

This can easily be wrapped up in a function if you need to do this frequently.

2
  • 18
    os.mkdir creates only one immediate directory. If there are several levels of directories in the path that don't exist, use os.makedirs. However note that it throws an OSError if the leaf exists
    – mz8i
    Commented Aug 24, 2018 at 9:13
  • 9
    @mz8i If an existing leaf directory is acceptable, os.makedirs can be called with exist_ok=True to avoid the error.
    – stiaan
    Commented Mar 30, 2020 at 15:05
71

Here is an alternative way to do this using the excellent standard library pathlib module, which generally makes things neater.

As explained elsewhere, to_csv will create the file if it doesn't exist, but won't create any non-existent directories in the path to the file, so you need to first ensure that these exist.

from pathlib import Path

output_file = 'my_file.csv'
output_dir = Path('long_path/to/my_dir')

output_dir.mkdir(parents=True, exist_ok=True)

df.to_csv(output_dir / output_file)  # can join path elements with / operator

Setting parents=True will also create any necessary parent directories, and exist_ok=True means it won't raise an error if the directory already exists, so you don't have to explicitly check that separately.

5
  • 1
    This was what is wanted, I also thought of creating directories recursively using the above implementation. But this is better and much simpler.
    – Janith
    Commented Feb 15, 2019 at 5:56
  • 5
    pathlib is such a dream! :D
    – Tim
    Commented Feb 15, 2019 at 21:35
  • 7
    @JKK You can do the exact same thing with os.makedirs('long_path/to/my_dir', exist_ok=True) - has the exact same behavior. However, nothing wrong with the object oriented pathlib approach if that's your cup of tea!
    – qRTPCR
    Commented May 29, 2019 at 20:52
  • 1
    If I'm completely honest, this was partly just an opportunity to introduce people to pathlib 😜 ...and yes, speaking as a Brit, it is very much my cup of tea! 😂👌☕
    – Tim
    Commented Jun 1, 2019 at 16:12
  • 1
    This should be the accepted answer. Much cleaner and safer solution. Commented Nov 8, 2021 at 15:23
3

I had this error when I accidentally added file:// at the begging of the save path. Since search brought me here, might be also helpful to someone.

1
  • 2
    Silly! pd.read_csv has no problems with the file:// prefix but for some reason to_csv does.
    – kmiller96
    Commented Sep 20, 2021 at 5:55
0

Adding to the answer of Tim if you have the whole file path in one string you can use this modified version:

from pathlib import Path

output_file = '/tmp/long_path/to/my_dir/my_file.csv'
output_file_path = Path(output_file)

output_file_path.parent.mkdir(parents=True, exist_ok=True)

df.to_csv(output_file)

By using the full path of the file in Path() it will point to that file, therefore we need to call .parent before .mkdir to not create a directory with the name of our file.

0

In my case it was some permission issue. My os is Windows. There is a antivirus installed and I made that directory protected earlier. I totally forgot that. After disabling it, it worked fine. So, when you face any issue regarding this, do check if the directory is protected or not.

0

I've the same problem but only for one filename. I save a lot files with diferent names in the same folder. And if I change this filename save the file.

I've a question, OS library have some chars reserved?

When I save "PRN.csv" in any folder take this error, if I change filename to "PRN5.csv" I can save it without errors.

2
  • wow! When I create this file manually in windows take a system error. So it is a OS error with some names.
    – Optimvs
    Commented Sep 20, 2023 at 17:56
  • This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review
    – OCa
    Commented Sep 25, 2023 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.