10

I'm using argparse to parse the inputs to my python3 program. I was recently asked to range check some of the numeric inputs, a seemingly good idea. Argparse has a facility to do just that.

The numeric inputs are port numbers, in the usual range 0-65535, so I altered my parse command line to :

import argparse
cmd_parser = argparse.ArgumentParser()
cmd_parser = add_argument('-p', help='Port number to connect to', dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.parse_args(['-h'])

Now, however, when I request the help, I get flooded with all the possible values from argparse. eg.

optional arguments:
    -h, --help            show this help message and exit
    -p {0,1,2,3,4,5,6,7,8,9,10,11,12,13 ...
    65478,65479,65480,65481,65482,65483,65484,65485,65486,65487,65488,65489,
    65490,65491,65492,65493,65494,65495,65496,65497,65498,65499,65500,65501,
    65502,65503,65504,65505,65506,65507,65508,65509,65510,65511,65512,65513,
    65514,65515,65516,65517,65518,65519,65520,65521,65522,65523,65524,65525,
    65526,65527,65528,65529,65530,65531,65532,65533,65534,65535}
                    Port number to connect to
...

It lists every single port in that range. Is there a way to truncate this or make it realize its a range (0-65535) or for it to use ellipsis or something to make it a bit prettier? Is my only option to explicitly range check my inputs with if statements?

I've been googling this but I'm having trouble finding examples where people used argparse and specified choices. I also checked the documentation on argparse but didn't see anything useful. https://docs.python.org/2/library/argparse.html

5
  • did you try monkey patching parser.print_help() to get the required output ?
    – cmidi
    Commented Jun 7, 2016 at 13:36
  • The print_help() command shows the same exaggerated output (every number between 0 and 65535 is shown).
    – LawfulEvil
    Commented Jun 7, 2016 at 13:38
  • I'd try to open a bug in python's issue tracker. When generating the output it seems like argparse is iterating over all possible choices and displaying all of them. They could add an option to only show up to N choices if they are available (like first N-1 and the last one).
    – Bakuriu
    Commented Jun 7, 2016 at 13:42
  • BTW: pretty closely related question: stackoverflow.com/questions/9366369/… I'm not closing as a duplicate since I don't find those answers satisfying and years have passed.
    – Bakuriu
    Commented Jun 7, 2016 at 13:44
  • I see I mentioned the bug/issue in that SO link as well. Anyways, with your numeric range I think it's easier to test with a custom type rather than patch the choices listings.
    – hpaulj
    Commented Jun 7, 2016 at 16:15

6 Answers 6

6

Use custom action...

import argparse

class PortAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        if not 0 < values < 2**16:
            raise argparse.ArgumentError(self, "port numbers must be between 0 and 2**16")
        setattr(namespace, self.dest, values)

cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        action=PortAction,
                        metavar="{0..65535}")

An invalid port number will display the error message based on the raised ArgumentError. If you enter a value of 65536, the following line will be printed:

error: argument -p: port numbers must be between 0 and 2**16

The usage and help messages will be printed based on the metavar displayed

0
3

Just use int as type in add_argument, and manually verify it's in the permitted range. Or, use a type of your own, which has a constructor that does the checking for you, and a __int__ method for implicit conversion:

class portnumber:
    def __init__(self, string):
        self._val = int(string)
        if (not self._val > 0) or (not self.val < 2**16):
            raise argparse.ArgumentTypeError("port numbers must be integers between 0 and 2**16")
    def __int__(self):
        return self._val

...

parser.add_argument("-p",type=portnumber)
3
  • 1
    Why not if not 0 < self._val < 2**16? Reads better: if the value is not in the range 0-2**16
    – Bakuriu
    Commented Jun 7, 2016 at 13:46
  • Use class portnumber(int) instead.
    – pppery
    Commented Jun 7, 2016 at 15:01
  • So I'm supposed to access this argument with int(args.p)? Because what appears in the Namespace is a portnumber instance.
    – hpaulj
    Commented Jun 7, 2016 at 21:20
2

Supply an explicit metavar argument instead of letting argparse produce one for you.

cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        choices=range(0,65536),
                        metavar="{0..65535}")
3
  • 1
    This doesn't quite do it. For alphanumeric inputs, it is fine. For the help, it makes it look nicer, but if I provide a parameter out of range it still displays the full list of options. eg. -p -5, then it shows all the numbers. I think I'll have to combine this with the port number class to achieve the best results.
    – LawfulEvil
    Commented Jun 7, 2016 at 13:59
  • 1
    Oh, good grief. I'm converted; this should be fixed in argparse itself :)
    – chepner
    Commented Jun 7, 2016 at 14:00
  • @LawfulEvil, see my answer to solve the parameter out of range issue. It is similar to the port number class, but I feel like it is more argparsey...
    – arewm
    Commented Jun 7, 2016 at 15:01
2

There is a Python bug/issue regarding the formatting of large choices lists. Currently choices in the help is formatted with

    def _metavar_formatter: ...
        choice_strs = [str(choice) for choice in action.choices]
        result = '{%s}' % ','.join(choice_strs)

and for errors with:

def _check_value(self, action, value):
    # converted value must be one of the choices (if specified)
    if action.choices is not None and value not in action.choices:
        args = {'value': value,
                'choices': ', '.join(map(repr, action.choices))}
        msg = _('invalid choice: %(value)r (choose from %(choices)s)')
        raise ArgumentError(action, msg % args)

So it expects choices to be an iterable, but does nothing to compress the list or make it pretty. Note that the only test is that value not in action.choices. choices is a very simple feature.

I mentioned this issue in a previous SO question: http://bugs.python.org/issue16468. Proposed patches are involved, so don't expect any fix soon.

I would recommend using your own type testing rather than choices. Or do your own range testing after parsing.

def myrange(astring):
   anint = int(astring)
   if anint in range(0,1000):
        return anint
   else:
        raise ValueError()
        # or for a custom error message
        # raise argparse.ArgumentTypeError('valid range is ...')

parser.add_argument('value',type=myrange,metavar='INT',help='...')

Another old (2012) SO question that addresses the range choices. The answer suggests both a help-formatter fix and the custom type

Python's argparse choices constrained printing

and

Python argparse choices from an infinite set

============================

Out of curiosity I defined a custom range class. It behaves like a regular range (without the step) for the in test, but returns custom values when used as an iterator.

class Range(object):
    def __init__(self, start, stop, n=3):
        self.start = start
        self.stop = stop
        self.n = n
    def __contains__(self, key):
        return self.start<=key<self.stop   
    def __iter__(self):
        if self.stop<(self.start+(self.n*3)):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield i
            yield '...'
            for i in range(self.stop-self.n, self.stop):
                yield i

When used with an argument as

parser.add_argument("-p",type=int, choices=Range(2,10,2))

It produces

1455:~/mypy$ python stack37680645.py -p 3
Namespace(p=3)

1458:~/mypy$ python stack37680645.py -h
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]

optional arguments:
  -h, --help        show this help message and exit
  -p {2,3,...,8,9}

The error message isn't quite what I'd want, but close

1458:~/mypy$ python stack37680645.py -p 30
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]
stack37680645.py: error: argument -p: invalid choice: 30 (choose from 2, 3, '...', 8, 9)

It would be better if the choices formatting allowed the action.choices object to create its own str or repr string.

Actually the iter could be as simple as (just one string):

def __iter__(self):
    yield 'a custom list'

Another option is to use metavar to control the usage/help display, and this __iter__ to control the error display.

One thing to watch out for when using metavar. The usage formatter does not behave when there are special characters like space, '()' and '[]' in the metavar, especially when the usage line extends to 2 or more lines. That's a known bug/issue.

1
  • This is good. I edited it a bit an added my answer below - but the kudos is yours.
    – Konchog
    Commented Aug 17, 2019 at 14:28
0

Monkey patch the print_help to get the desired output

def my_help(): print "0-65535 range"
cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p', help='Port number to connect to',   dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.print_help = my_help
cmd_parser.parse_args()
0

The answer I used is inspired by @hpaulj's answer. However, I agree that the problem really lies at the feet of argparse. I found this SO query when my computer hung trying to allocate gigabytes of space just to output the help text.

My issue with hpaulj's 'Range' class is that when using large upper limits, the message still pushes out very large numbers. The following class uses a infinite upper limit by default.

class ArgRange(object):
    from decimal import Decimal
    huge = Decimal('+infinity')
    huge_str = '{:.4E}'.format(huge)

    def __init__(self, start, stop=huge, n=3):
        self.start = start
        self.stop = stop
        self.n = n

    def __contains__(self, key):
        return self.start <= key < self.stop

    def __iter__(self):
        if self.stop < self.start+(self.n*3):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield I
            if self.stop is self.huge:
                yield '...' + huge_str
            else:
                yield '...'
                for i in range(self.stop - self.n, self.stop):
                    yield i

bounds = ArgRange(2)
balance = ArgRange(0, 1000)
parser = argparse.ArgumentParser(description="Do something fun")
parser.add_argument("width", type=int, choices=bounds,  default=9)
parser.add_argument("height", type=int, choices=balance, default=200)

With an incorrect value, the error is:

argument width: invalid choice: 1 (choose from 2, 3, 4, '...infinity')

or

argument height: invalid choice: 2000 (choose from 0, 1, 2, '...', 997, 998, 999)

And usage looks like:

usage: test.py 
           {2,3,4,...infinity} {0,1,2,...,997,998,999}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.