15

I'm using argparse to take input and pass it to a function that takes as arguments two variables and **kwargs.

Here's my function:

import requests
import sys
import argparse


def location_by_coordinate(LAT, LNG, **kwargs):
    if not kwargs:
        coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
        r = requests.get(coordinate_url).text
    else:
        coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
        for key, value in kwargs.iteritems():
            if 'DISTANCE' in kwargs:
                distance = kwargs.get('DISTANCE')
                if distance > 5000:
                    print distance
                    print "max distance is 5000m, value is reassigned to default of 1000m"
                    distance = 1000
                    coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
                    r = requests.get(coordinate_url).text
                else:
                    pass
                    coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
                    r = requests.get(coordinate_url).text
            if 'FACEBOOK_PLACES_ID' in kwargs:
                fb_places_id = kwargs.get('FACEBOOK_PLACES_ID')
                payload = {'FACEBOOK_PLACES_ID': '%s' % (fb_places_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
            if 'FOURSQUARE_ID' in kwargs:
                foursquare_id = kwargs.get('FOURSQUARE_ID')
                payload = {'FOURSQUARE_ID': '%s' % (foursquare_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
            if 'FOURSQUARE_V2_ID' in kwargs:
                foursquare_v2_id = kwargs.get('FOURSQUARE_V2_ID')
                payload = {'FOURSQUARE_V2_ID': '%s' % (foursquare_v2_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
    #print r
    return r

Given this function and its use of **kwargs, how should I setup the subparsers?

Here's how I've setup the command line parser thus far:

 def main():
        parser = argparse.ArgumentParser(description="API Endpoints tester")
        subparsers = parser.add_subparsers(dest="command", help="Available commands")

        location_by_parser = subparsers.add_parser("location_by_coordinate", help="location function")
        location_by_parser.add_argument("LAT", help="latitude")
        location_by_parser.add_argument("LNG", help="longitude")

        arguments = parser.parse_args(sys.argv[1:])
        arguments = vars(arguments)
        command = arguments.pop("command")
        if command == "location_by_coordinate":
            LAT, LNG = location_by_coordinate(**arguments)
        else:
            print "No command provided..."

    if __name__ == "__main__":
        main()

Obviously, the above main() function works fine with the location_by_coordinate() function when I call it at the command line like this:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148

But with the code the way it is currently, if I try:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 DISTANCE=3000

Obviously, I get:

argstest.py: error: unrecognized arguments: DISTANCE=3000

But I'm not sure how to setup a subparser for **kwargs. If I try to setup a subparser like this:

location_by_parser.add_argument("**kwargs", help="**kwargs")

and then try that command again:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 DISTANCE=3000

That doesn't work because the arguments object (which is a dictionary), becomes this:

{'LAT': '40.5949799', 'LNG': '-73.9495148', 'command': 'location_by_coordinate', '**kwargs': 'DISTANCE=3000'}

And this Traceback is returned:

Traceback (most recent call last):
  File "argstest.py", line 118, in <module>
    main()
  File "argstest.py", line 108, in main
    foo = location_by_coordinate(**arguments)
  File "argstest.py", line 40, in location_by_coordinate
    return r
UnboundLocalError: local variable 'r' referenced before assignment

How can I enable argparse to handle/to parse what is entered at the command line that is intended to be passed to the function via **kwargs?

1
  • You might also look at a package that builds on top of argparse called plac. It tries to populate a parser based on the argument definitions of one or more functions. pypi.python.org/pypi/plac
    – hpaulj
    Commented Nov 18, 2015 at 0:21

2 Answers 2

10

Do you understand what is going on with the

{'LAT': '40.5949799', 'LNG': '-73.9495148', 'command': 'location_by_coordinate', '**kwargs': 'DISTANCE=3000'}

arguments dictionary? You defined a 'positional' argument with the name ('dest') of '**kwargs'. You could just as well named it 'foobar'. The parser assigned the string 'DISTANCE=3000' to that attribute in the args namespace, which turned into a dictionary key:value pair in arguments.

You could, of course, look for arguments['**kwargs'], and parse the value for yourself:

v = arguments['**kwargs']  # or pop if you prefer
if v is not None:
    k, v = v.split('=')
    arguments[k] = int(v)

It could be generalized to handle multiple pairs (defined with `nargs='*').


argparse does not handle arguments the same way as Python functions, so there's nothing exactly analogous the **kwargs.

The normal way to accept something like distance is with 'optionals' or flagged arguments.

parser.add_argument('-d','--distance', type=int, help=...)

which will accept

python argstest.py location_by_coordinate 40.5949799 -73.9495148 --distance=3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148 --distance 3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148 --d3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148

It could also be setup to use --DISTANCE or other names. In the last case args namespace will have a default value for distance. The default default is None.

That's the straight forward way of adding kwarg like arguments to argparse.

Accepting arbitrary dictionary like pairs, distance:3000, distance=3000, has been asked before on SO. The answers have always been some variation of the parsing that I sketched above. It could be done in a custom Action class, or post parsing as I suggest.

oops, this answer is nearly a clone of one I wrote a few days ago: https://stackoverflow.com/a/33639147/901925

A similar 2011 question: Using argparse to parse arguments of form "arg= val"

Python argparse dict arg

=================================

(edit)

Example with a function that takes *args:

In [2]: import argparse
In [3]: def foo(*args, **kwargs):
   ...:     print('args',args)
   ...:     print('kwargs',kwargs)
   ...:     
In [4]: parser=argparse.ArgumentParser()
In [5]: parser.add_argument('arg1')
In [6]: parser.add_argument('arg2',nargs='+')

In [7]: args=parser.parse_args('one two three'.split())
In [8]: args
Out[8]: Namespace(arg1='one', arg2=['two', 'three'])

So I have 2 positional arguments, one with a single string value, the other with a list (due to the + nargs).

Call foo with these args attributes:

In [10]: foo(args.arg1)
args ('one',)
kwargs {}

In [11]: foo(args.arg1, args.arg2)
args ('one', ['two', 'three'])
kwargs {}

In [12]: foo(args.arg1, arg2=args.arg2)
args ('one',)
kwargs {'arg2': ['two', 'three']}

I defined 'positionals', but it would have worked just as well with 'optionals'. The distinction between positionals and optionals disappears in the namespace.

If I convert the namespace to a dictionary, I can pass values to foo in various ways, either through the *args or through **kwargs. It's all in how I call foo, not in how they appear in args or arguments. None of this is unique to argparse.

In [13]: arguments = vars(args)
In [14]: arguments
Out[14]: {'arg2': ['two', 'three'], 'arg1': 'one'}

In [15]: foo(arguments['arg2'], arguments['arg1'])
args (['two', 'three'], 'one')
kwargs {}

In [16]: foo(arguments['arg2'], arguments)
args (['two', 'three'], {'arg2': ['two', 'three'], 'arg1': 'one'})
kwargs {}

In [17]: foo(arguments['arg2'], **arguments)
args (['two', 'three'],)
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}

In [24]: foo(*arguments, **arguments)
args ('arg2', 'arg1')             # *args is the keys of arguments
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}

In [25]: foo(*arguments.values(), **arguments)
args (['two', 'three'], 'one')    # *args is the values of arguments
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}
4
  • Thanks for this answer, very informative. It is possible to create an optional argument with argparse that will work with a function that takes *args? In other words, pass a list with argparse to a function? Commented Nov 17, 2015 at 22:34
  • 1
    I'm not entirely sure what you are asking, but I've added some examples of passing args values to a function that uses *args.
    – hpaulj
    Commented Nov 18, 2015 at 0:20
  • Ok, thank you, that's what I asking. Much appreciated! Commented Nov 18, 2015 at 0:25
  • 1
    Just a hint: you can set parser.add_argument('arg2',nargs='*') which makes optional arguments truly optional
    – Bostone
    Commented Nov 24, 2019 at 2:01
1

How can I enable argparse to handle/to parse what is entered at the command line that is intended to be passed to the function via **kwargs?

This command:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 DISTANCE=3000

does NOT execute the function call:

location_by_coordinate(40.5949799, -73.9495148, DISTANCE=3000)

That is easy to prove:

def location_by_coordinate(x, y, **kwargs):
    print "I was called!"

Go ahead and parse the args, and you'll see that the function isn't called. As a result, all your work setting up a subparser with the name location_by_coordinate was in vain.

The argparse module just examines sys.argv, which is a simple list of strings. Each string is one of the 'words' entered on the command line after the python command.

By default, the argument strings are taken from sys.argv,...
https://docs.python.org/3/library/argparse.html#the-parse-args-method

Yeah, sys.argv is a scary name, but a list of strings is just a list of strings. If you look at the argparse docs, all the examples do this:

parser.parse_args('--foo FOO'.split())

A list of strings you create with split() is no different than some list of strings that sys.argv refers to.

You need to call your location_by_coordinate() function yourself. In order to do that, you need to get the args from the command line, assemble the args that should be kwargs in a dictionary, and call your function like this:

location_by_coordinate(lat, lon, **my_dict)

If you have these values:

lat = 10
lon = 20
my_dict = {'a': 1, 'b': 2}

then the function call above will be equivalent to:

location_by_coordinate(10, 20, a=1, b=2)

Here is an example:

import argparse

def dostuff(x, y, **kwargs):
    print x, y, kwargs

parser = argparse.ArgumentParser()
parser.add_argument("LAT")
parser.add_argument("LON")
parser.add_argument("--distance")
args = parser.parse_args()
my_dict = {}
my_dict["distance"] = args.distance

dostuff(args.LAT, args.LON, **my_dict)

$ python my_prog.py 10 20 --distance 1
10 20 {'distance': '1'}

You can also get a dict from the parser:

...
...
args = parser.parse_args()
args_dict = vars(args)
print args_dict

--output:--
{'LAT': '10', 'distance': '1', 'LON': '20'}

lat = args_dict.pop('LAT')
lon = args_dict.pop('LON')
print args_dict

--output:--
{'distance': '1'}

location_by_coordinates(lat, lon, **args_dict)

If you want to make the user type:

DISTANCE=3000

on the command line, first of all I would not make them type all caps, so lets make the goal:

distance=3000

Add another mandatory argument to the parser:

location_by_parser.add_argument("distance", help="distance")

Then after you parse the following:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 distance=3000

you can do this:

arguments = parser.parse_args()
args_dict = vars(arguments)

The args_dict will contain the key/value pair 'distance': 'distance=3000'. You can change that dict entry to 'distance': '3000' by doing the following:

pieces = args_dict['distance'].split('=')

if len(pieces) == 2 and pieces[0] == 'distance':
    args_dict['distance'] = pieces[1]

Or, you can set things up so that the parser will automatically execute that code by creating a custom action that executes when the distance arg is parsed:

class DistanceAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        #values => The value for the distance command line arg
        pieces = values.split('=')

        if len(pieces) == 2 and pieces[0] in ['distance', 'wave_action']:  #only allow 'distance=' and 'wave_action='
            setattr(namespace, self.dest, pieces[1]) #The dest key specified in the parser gets assigned the value
        else:
            raise argparse.ArgumentTypeError('Usage: distance=3000.  Only distance=, wave_action= allowed.')

You can use the action like this:

location_by_parser.add_argument(
    "distance", 
    help="longitude", 
    action=DistanceAction
)

And if you want to get fancy, you can collect all the name=val args specified on the command line into one dictionary named, say, keyword_args, which will allow you to call your method like this:

args = parser.parse_args()
args_dict = vars(args)
keyword_args = args_dict["keyword_args"]

location_by_coordinates(lat, lon, **keyword_args)

Here's the parser configuration:

location_by_parser.add_argument(
    "keyword_args", 
    help="extra args", 
    nargs='*', 
    action=DistanceAction
)

import argparse
import sys

def location_by_coordinates(x, y, **kwargs):
    print x 
    print y
    print kwargs

class DistanceAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        allowed_keywords = ['distance', 'wave_action']
        keyword_dict = {}

        for arg in values:  #values => The args found for keyword_args
            pieces = arg.split('=')

            if len(pieces) == 2 and pieces[0] in allowed_keywords:
                keyword_dict[pieces[0]] = pieces[1]
            else: #raise an error                                                         
                #Create error message:
                msg_inserts = ['{}='] * len(allowed_keywords)
                msg_template = 'Example usage: distance=3000. Only {} allowed.'.format(', '.join(msg_inserts))
                msg = msg_template.format(*allowed_keywords)

                raise argparse.ArgumentTypeError(msg)

        setattr(namespace, self.dest, keyword_dict) #The dest key specified in the
                                                    #parser gets assigned the keyword_dict--in
                                                    #this case it defaults to 'keyword_args'

parser = argparse.ArgumentParser(description="API Endpoints tester")
subparsers = parser.add_subparsers(dest="command", help="Available commands")

location_by_parser = subparsers.add_parser("location_by_coordinate", help="location function")
location_by_parser.add_argument("LAT", help="latitude")
location_by_parser.add_argument("LNG", help="longitude")
location_by_parser.add_argument("keyword_args", help="extra args", nargs='*', action=DistanceAction)

arguments = parser.parse_args()
args_dict = vars(arguments)

print args_dict

lat = args_dict['LAT']
lon = args_dict['LNG']
keyword_args = args_dict['keyword_args']

location_by_coordinates(lat, lon, **keyword_args)

Example:

$ python prog.py location_by_coordinate 40.5949799 -73.9495148 distance=3000 wave_action=1.4

{'LAT': '40.5949799', 'LNG': '-73.9495148', 'command': 'location_by_coordinate', 'keyword_args': {'distance': '3000', 'wave_action': '1.4'}}

40.5949799
-73.9495148
{'distance': '3000', 'wave_action': '1.4'}

$ python prog.py location_by_coordinate 40.5949799 -73.9495148 x=10
...
...
  File "2.py", line 25, in __call__
    raise argparse.ArgumentTypeError(msg)
argparse.ArgumentTypeError: Example usage: distance=3000. Only distance=, wave_action= allowed.
1
  • vars is what I was looking for. Thanks Commented Aug 8 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.