Is there a way to get functionality similar to mkdir -p
on the shell from within Python. I am looking for a solution other than a system call. I am sure the code is less than 20 lines, and I am wondering if someone has already written it?
12 Answers
For Python ≥ 3.5, use pathlib.Path.mkdir
:
import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)
The exist_ok
parameter was added in Python 3.5.
For Python ≥ 3.2, os.makedirs
has an optional third argument exist_ok
that, when True
, enables the mkdir -p
functionality—unless mode
is provided and the existing directory has different permissions than the intended ones; in that case, OSError
is raised as previously:
import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)
For even older versions of Python, you can use os.makedirs
and ignore the error:
import errno
import os
def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise
-
20In the spirit of micro-improving something lots of people will copy+paste: how about replacing
==
with!=
and removing thepass
/else
:-) Aug 4, 2011 at 15:51 -
7This appears to fail if the last portion of path is a file, as exc.errno equals errno.EEXIST and so everything seems ok, but actually using the directory later will obviously fail.– elhefeNov 7, 2012 at 1:53
-
26What about distutils.dir_util.mkpath? It's pretty simple as
mkpath('./foo/bar')
– aurahamJan 15, 2013 at 21:41 -
4Optimize exception handling? except OSError as exc: if exc.errno != errno.EEXIST or not os.path.isdir(path): raise– JulianFeb 4, 2014 at 21:23
-
11@auraham, mkpath has some unexpected behavior due to undocumented caching that may cause problems if you try to use it exactly like mkdir -p: bugs.python.org/issue10948.– romanowsFeb 18, 2015 at 3:00
-
This worked for me. To understand why, see documentation here (search for "makedirs"): docs.python.org/3/library/os.html Jul 15, 2022 at 13:53
This is easier than trapping the exception:
import os
if not os.path.exists(...):
os.makedirs(...)
Disclaimer This approach requires two system calls which is more susceptible to race conditions under certain environments/conditions. If you're writing something more sophisticated than a simple throwaway script running in a controlled environment, you're better off going with the accepted answer that requires only one system call.
UPDATE 2012-07-27
I'm tempted to delete this answer, but I think there's value in the comment thread below. As such, I'm converting it to a wiki.
-
28This way, you make it less probable but not impossible that makedirs will fail, in all multitasking operating systems. It's like saying "256 chars should be enough for any path created".– tzotMar 2, 2009 at 23:42
-
4@Asa Of course. And mkdir -p would complain about that too. Did I miss your point? Mar 3, 2009 at 19:15
-
4@jholloway7: based on the requirements ("mkdir -p"-like functionality) Asa's comment is unnecessary. However, I would like to know whether you do acknowledge that it's possible that the directory can be non-existent when .exists is called, and existent when .makedirs is called.– tzotMar 3, 2009 at 22:38
-
6@TZ Yes, I certainly acknowledge that. Again, without complete specifications from the original poster, my assumption was that he/she wanted a solution that could be used to create a directory tree if not already existing in a simple script, not a HA enterprisey production solution with SLAs. Mar 4, 2009 at 15:06
-
6@Asa That's what exceptions are for, something unexpected went wrong. If you don't have permissions the exception bubbles all the way up and you notice to fix the permissions. As it should be.– jpsimonsJul 11, 2010 at 23:04
Recently, I found this distutils.dir_util.mkpath:
In [17]: from distutils.dir_util import mkpath
In [18]: mkpath('./foo/bar')
Out[18]: ['foo', 'foo/bar']
-
13Beware, mkpath() caches the directory so that you can't re-mkpath() a directory that has been removed with a different method: bugs.python.org/issue10948.– romanowsFeb 18, 2015 at 2:58
-
8@romanows Moreover the method is intended to be private, in case anyone else is tempted to read the bug report to see if it's been 'fixed' (it's not a bug). Jul 9, 2015 at 18:21
-
@MauroBaraldi the point is that if you create a directory with this method, it gets deleted and you try to create it again using this method from the same program, it won't work. Don't use this.– user3064538Dec 28, 2019 at 8:28
With Pathlib from python3 standard library:
Path(mypath).mkdir(parents=True, exist_ok=True)
If parents is true, any missing parents of this path are created as needed; they are created with the default permissions without taking mode into account (mimicking the POSIX mkdir -p command). If exist_ok is false (the default), an FileExistsError is raised if the target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file.
Changed in version 3.5: The exist_ok parameter was added.
-
3For
python
< 3.5 you can usepathlib2
.pip install pathlib2
;from pathlib2 import Path
Feb 3, 2017 at 15:56 -
I don't like this method - prefer the os.mkdir option. SIlent continue if you delete and recreate folders in conjunction with shutil.rmtree - leaving a lock. os version gives access denied -and abends early rather than later. Setting up a folder of results from a huge conversion which would not be able to output results– JGFMKMay 16, 2021 at 23:03
mkdir -p
gives you an error if the file already exists:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory `/tmp/foo': File exists
So a refinement to the previous suggestions would be to re-raise
the exception if os.path.isdir
returns False
(when checking for errno.EEXIST
).
(Update) See also this highly similar question; I agree with the accepted answer (and caveats) except I would recommend os.path.isdir
instead of os.path.exists
.
(Update) Per a suggestion in the comments, the full function would look like:
import os
def mkdirp(directory):
if not os.path.isdir(directory):
os.makedirs(directory)
-
You are absolutely correct about this case; however, the program should catch exceptions later on e.g. when trying to open("/tmp/foo/a_file", "w"), so I don't think an update is necessary. You could update your answer with Python code instead, and watch it being upvoted ;)– tzotMar 3, 2009 at 22:43
-
In a lot of cases that would probably be fine. In general, though, I would prefer the code to fail as early as possible so it's clear what really caused the problem. Mar 3, 2009 at 22:47
-
11If it already exists AS A DIRECTORY, mkdir -p does not error. It does error if you ask it to create a directory, and a FILE by that name already exists. Dec 6, 2012 at 18:49
-
@FrankKlotz that's why I'm calling os.path.isdir not os.path.exists Dec 6, 2012 at 23:45
-
−1 because the only part of this answer that actually answers the question (the last code block) answers it incorrectly and also duplicates other answers.– wcharginFeb 4, 2019 at 21:50
As mentioned in the other solutions, we want to be able to hit the file system once while mimicking the behaviour of mkdir -p
. I don't think that this is possible to do, but we should get as close as possible.
Code first, explanation later:
import os
import errno
def mkdir_p(path):
""" 'mkdir -p' in Python """
try:
os.makedirs(path)
except OSError as exc: # Python >2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
As the comments to @tzot's answer indicate there are problems with checking whether you can create a directory before you actually create it: you can't tell whether someone has changed the file system in the meantime. That also fits in with Python's style of asking for forgiveness, not permission.
So the first thing we should do is try to make the directory, then if it goes wrong, work out why.
As Jacob Gabrielson points out, one of the cases we must look for is the case where a file already exists where we are trying to put the directory.
With mkdir -p
:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory '/tmp/foo': File exists
The analogous behaviour in Python would be to raise an exception.
So we have to work out if this was the case. Unfortunately, we can't. We get the same error message back from makedirs whether a directory exists (good) or a file exists preventing the creation of the directory (bad).
The only way to work out what happened is to inspect the file system again to see if there is a directory there. If there is, then return silently, otherwise raise the exception.
The only problem is that the file system may be in a different state now than when makedirs was called. eg: a file existed causing makedirs to fail, but now a directory is in its place. That doesn't really matter that much, because the the function will only exit silently without raising an exception when at the time of the last file system call the directory existed.
-
2
I think Asa's answer is essentially correct, but you could extend it a little to act more like mkdir -p
, either:
import os
def mkdir_path(path):
if not os.access(path, os.F_OK):
os.mkdirs(path)
or
import os
import errno
def mkdir_path(path):
try:
os.mkdirs(path)
except os.error, e:
if e.errno != errno.EEXIST:
raise
These both handle the case where the path already exists silently but let other errors bubble up.
-
On
Python 2.7.6 ... [GCC 4.8.2] on linux2
, at least, it seems that it should beos.mkdir
, notos.mkdirs
. Jun 6, 2015 at 18:53 -
the first option is susceptible to race conditions (at one instant, the dir is not there, so we proceed to create it but in the middle something else creates it and boom!) second option is the way to go in Python 2 Jul 30, 2021 at 13:10
Function declaration;
import os
def mkdir_p(filename):
try:
folder=os.path.dirname(filename)
if not os.path.exists(folder):
os.makedirs(folder)
return True
except:
return False
usage :
filename = "./download/80c16ee665c8/upload/backup/mysql/2014-12-22/adclient_sql_2014-12-22-13-38.sql.gz"
if (mkdir_p(filename):
print "Created dir :%s" % (os.path.dirname(filename))
import os
import tempfile
path = tempfile.mktemp(dir=path)
os.makedirs(path)
os.rmdir(path)
I've had success with the following personally, but my function should probably be called something like 'ensure this directory exists':
def mkdirRecursive(dirpath):
import os
if os.path.isdir(dirpath): return
h,t = os.path.split(dirpath) # head/tail
if not os.path.isdir(h):
mkdirRecursive(h)
os.mkdir(join(h,t))
# end mkdirRecursive
-
this is a nice answer for 2.7, seems cleaner than trapping an error May 11, 2017 at 15:15
-
fails if part of the tree already exists though, so here's a fix:-
import os; from os.path import join as join_paths def mk_dir_recursive(dir_path): if os.path.isdir(dir_path): return h, t = os.path.split(dir_path) # head/tail if not os.path.isdir(h): mk_dir_recursive(h) new_path = join_paths(h, t) if not os.path.isdir(new_path): os.mkdir(new_path)
May 11, 2017 at 15:29
import os
from os.path import join as join_paths
def mk_dir_recursive(dir_path):
if os.path.isdir(dir_path):
return
h, t = os.path.split(dir_path) # head/tail
if not os.path.isdir(h):
mk_dir_recursive(h)
new_path = join_paths(h, t)
if not os.path.isdir(new_path):
os.mkdir(new_path)
based on @Dave C's answer but with a bug fixed where part of the tree already exists
{Subfolder1,Subfolder2}
syntax is a feature of bash (and some other shells), notmkdir -p
. e.g.echo prefix-{Subfolder1,Subfolder2}
will show "prefix-Subfolder1 prefix-Subfolder2".