66

My python script needs to read files from a directory passed on the command line. I have defined a readable_dir type as below to be used with argparse for validating that the directory passed on the command line is existent and readable. Additionally, a default value (/tmp/non_existent_dir in the example below) has also been specified for the directory argument. The problem here is that argparse invokes readable_dir() on the default value even in a situation where a directory argument is explicitly passed in on the command line. This causes the script to crap out as the default path /tmp/non_existent_dir does not exist in a context where a directory is explicitly passed in on the command line. I could get around this by not specifying a default value and making this argument mandatory, or by deferring the validation until later in the script but is a more elegant solution that anyone is aware of?

#!/usr/bin/python
import argparse
import os

def readable_dir(prospective_dir):
  if not os.path.isdir(prospective_dir):
    raise Exception("readable_dir:{0} is not a valid path".format(prospective_dir))
  if os.access(prospective_dir, os.R_OK):
    return prospective_dir
  else:
    raise Exception("readable_dir:{0} is not a readable dir".format(prospective_dir))

parser = argparse.ArgumentParser(description='test', fromfile_prefix_chars="@")
parser.add_argument('-l', '--launch_directory', type=readable_dir, default='/tmp/non_existent_dir')
args = parser.parse_args()
1
  • 5
    Useful code sample. Raise should raise argparse.ArgumentTypeError, but otherwise, I'm digging the readable_dir type.
    – mlissner
    Jun 30, 2013 at 23:43

3 Answers 3

45

I submitted a patch for "path arguments" to the Python standard library mailing list a few months ago.

With this PathType class, you can simply specify the following argument type to match only an existing directory--anything else will give an error message:

type = PathType(exists=True, type='dir')

Here's the code, which could be easily modified to require specific file/directory permissions as well:

from argparse import ArgumentTypeError as err
import os

class PathType(object):
    def __init__(self, exists=True, type='file', dash_ok=True):
        '''exists:
                True: a path that does exist
                False: a path that does not exist, in a valid parent directory
                None: don't care
           type: file, dir, symlink, None, or a function returning True for valid paths
                None: don't care
           dash_ok: whether to allow "-" as stdin/stdout'''

        assert exists in (True, False, None)
        assert type in ('file','dir','symlink',None) or hasattr(type,'__call__')

        self._exists = exists
        self._type = type
        self._dash_ok = dash_ok

    def __call__(self, string):
        if string=='-':
            # the special argument "-" means sys.std{in,out}
            if self._type == 'dir':
                raise err('standard input/output (-) not allowed as directory path')
            elif self._type == 'symlink':
                raise err('standard input/output (-) not allowed as symlink path')
            elif not self._dash_ok:
                raise err('standard input/output (-) not allowed')
        else:
            e = os.path.exists(string)
            if self._exists==True:
                if not e:
                    raise err("path does not exist: '%s'" % string)

                if self._type is None:
                    pass
                elif self._type=='file':
                    if not os.path.isfile(string):
                        raise err("path is not a file: '%s'" % string)
                elif self._type=='symlink':
                    if not os.path.symlink(string):
                        raise err("path is not a symlink: '%s'" % string)
                elif self._type=='dir':
                    if not os.path.isdir(string):
                        raise err("path is not a directory: '%s'" % string)
                elif not self._type(string):
                    raise err("path not valid: '%s'" % string)
            else:
                if self._exists==False and e:
                    raise err("path exists: '%s'" % string)

                p = os.path.dirname(os.path.normpath(string)) or '.'
                if not os.path.isdir(p):
                    raise err("parent path is not a directory: '%s'" % p)
                elif not os.path.exists(p):
                    raise err("parent directory does not exist: '%s'" % p)

        return string
2
  • 8
    This is very good code. It's a shame your patch wasn't accepted. Apr 27, 2019 at 20:31
  • 1
    Thanks, @BjörnLindqvist. I'm not really sure why. It might benefit from others replying to it and reminding the stdlib maintainers that it's still “out there” and useful…
    – Dan Lenski
    Apr 28, 2019 at 22:35
41

You can create a custom action instead of a type:

import argparse
import os
import tempfile
import shutil
import atexit

class readable_dir(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        prospective_dir=values
        if not os.path.isdir(prospective_dir):
            raise argparse.ArgumentTypeError("readable_dir:{0} is not a valid path".format(prospective_dir))
        if os.access(prospective_dir, os.R_OK):
            setattr(namespace,self.dest,prospective_dir)
        else:
            raise argparse.ArgumentTypeError("readable_dir:{0} is not a readable dir".format(prospective_dir))

ldir = tempfile.mkdtemp()
atexit.register(lambda dir=ldir: shutil.rmtree(ldir))

parser = argparse.ArgumentParser(description='test', fromfile_prefix_chars="@")
parser.add_argument('-l', '--launch_directory', action=readable_dir, default=ldir)
args = parser.parse_args()
print (args)

But this seems a little fishy to me -- if no directory is given, it passes a non-readable directory which seems to defeat the purpose of checking if the directory is accessible in the first place.

Note that as pointed out in the comments, it might be nicer to
raise argparse.ArgumentError(self, ...) rather than argparse.ArgumentTypeError.

EDIT

As far as I'm aware, there is no way to validate the default argument. I suppose the argparse developers just assumed that if you're providing a default, then it should be valid. The quickest and easiest thing to do here is to simply validate the arguments immediately after you parse them. It looks like, you're just trying to get a temporary directory to do some work. If that's the case, you can use the tempfile module to get a new directory to work in. I updated my answer above to reflect this. I create a temporary directory, use that as the default argument (tempfile already guarantees the directory it creates will be writeable) and then I register it to be deleted when your program exits.

5
  • mgilson, i had to change prospective_dir=values[0] to prospective_dir=values. Without this only the first character from the argument was being picked up. Your solution works when an explicit argument is passed in (in that the default value is not validated under these circumstances). However when no argument is passed in, the default value is NOT being validated, which is a problem.
    – iruvar
    Jul 10, 2012 at 15:10
  • @cravoori -- Some reason I thought values would be a list. I suppose that only happens when nargs=... is specified. Anyway, I don't think there's any way to coax argparse to do the validation after the arguments have been parsed (which is what you're really asking for). You'll have to do that yourself. I have updated my code so that there is always a valid directory for you to work in which will get deleted when your program exits. (directories specified on the command line won't get deleted).
    – mgilson
    Jul 10, 2012 at 17:16
  • please note the temp directory was used only an example
    – iruvar
    Jul 13, 2012 at 2:48
  • 1
    Minor suggestion: raising the error using raise argparse.ArgumentTypeError("message") causes a stack trace. If you use raise argparse.ArgumentError(self, "message") it gets formatted nicely without a stack trace. Aug 7, 2017 at 17:49
  • @djova -- Yeah, interesting.
    – mgilson
    Aug 8, 2017 at 11:51
12

If your script can't work without a valid launch_directory then it should be made a mandatory argument:

parser.add_argument('launch_directory', type=readable_dir)

btw, you should use argparse.ArgumentTypeError instead of Exception in readable_dir().

5
  • 2
    argparse.ArgumentError(self, "error string") is best of all, if you want the user to see a nice error message instead of a stack trace. For more info, see: stackoverflow.com/questions/9881933/…
    – Skotch
    Mar 5, 2013 at 22:20
  • 1
    @Skotch: readable_dir defines a type so ArgumentTypeError is appropriate here. I've fixed the typo: action -> type
    – jfs
    Mar 5, 2013 at 22:38
  • J.F. Sebastian: I'm pretty sure it's a custom action we are talking about (see the definition of readable_dir given by mgilson above, it's being derived from argparse.Action). Passing a custom argparse action as a type will not work (at least it didn't when I tried).
    – Apteryx
    Apr 13, 2015 at 18:51
  • 1
    @Apteryx: read the quesiton. readable_dir is a function. My answer has nothing to do with mgilson's one.
    – jfs
    Apr 13, 2015 at 19:33
  • J.F. Sebastian: Sorry, you are right. And now that I understand, it makes more sense to go with a function defined type than to use a custom action. Thanks!
    – Apteryx
    Apr 14, 2015 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.