38

I'm at the point in learning Python where I'm dealing with the Mutable Default Argument problem.

# BAD: if `a_list` is not passed in, the default will wrongly retain its contents between successive function calls
def bad_append(new_item, a_list=[]):
    a_list.append(new_item)
    return a_list

# GOOD: if `a_list` is not passed in, the default will always correctly be []
def good_append(new_item, a_list=None):
    if a_list is None:
        a_list = []
    a_list.append(new_item)
    return a_list

I understand that a_list is initialized only when the def statement is first encountered, and that's why subsequent calls of bad_append use the same list object.

What I don't understand is why good_append works any different. It looks like a_list would still be initialized only once; therefore, the if statement would only be true on the first invocation of the function, meaning a_list would only get reset to [] on the first invocation, meaning it would still accumulate all past new_item values and still be buggy.

Why isn't it? What concept am I missing? How does a_list get wiped clean every time good_append runs?

5 Answers 5

32

It looks like a_list would still be initialized only once

"initialization" is not something that happens to variables in Python, because variables in Python are just names. "initialization" only happens to objects, and it's done via the class' __init__ method.

When you write a = 0, that is an assignment. That is saying "a shall refer to the object that is described by the expression 0". It is not initialization; a can name anything else of any type at any later time, and that happens as a result of assigning something else to a. Assignment is just assignment. The first one is not special.

When you write def good_append(new_item, a_list=None), that is not "initializing" a_list. It is setting up an internal reference to an object, the result of evaluating None, so that when good_append is called without a second parameter, that object is automatically assigned to a_list.

meaning a_list would only get reset to [] on the first invocation

No, a_list gets set to [] any time that a_list is None to begin with. That is, when either None is passed explicitly, or the argument is omitted.

The problem with [] occurs because the expression [] is only evaluated once in this context. When the function is compiled, [] is evaluated, a specific list object is created - that happens to be empty to start - and that object is used as the default.

How does a_list get wiped clean every time good_append runs?

It doesn't. It doesn't need to be.

You know how the problem is described as being with "mutable default arguments"?

None is not mutable.

The problem occurs when you modify the object that the parameter has as a default.

a_list = [] does not modify whatever object a_list previously referred to. It cannot; arbitrary objects cannot magically transform in-place into empty lists. a_list = [] means "a_list shall stop referring to what it previously referred to, and start referring to []". The previously-referred-to object is unchanged.

When the function is compiled, and one of the arguments has a default value, that value - an object - gets baked into the function (which is also, itself, an object!). When you write code that mutates an object, the object mutates. If the object being referred to happens to be the object baked into the function, it still mutates.

But you cannot mutate None. It is immutable.

You can mutate []. It is a list, and lists are mutable. Appending an item to a list mutates the list.

1
  • 1
    Thanks for the great answer. I'm killing myself trying to decide whether to mark this answer or @glglgl's as correct. The other answer contains the single illuminating phrase that made me able to comprehend your answer; your answer as a whole is more thorough and understandable, but somehow didn't make the light click on the same way. If there were a way to give two green checkmarks on a question, yours would absolutely be the other one (and may again become the only one if I keep waffling). May 21, 2012 at 1:34
22

The default value of a_list (or any other default value, for that matter) is stored in the function's interiors once it has been initialized and thus can be modified in any way:

>>> def f(x=[]): return x
...
>>> f.func_defaults
([],)
>>> f.func_defaults[0] is f()
True

resp. for Python 3:

>>> def f(x=[]): return x
...
>>> f.__defaults__
([],)
>>> f.__defaults__[0] is f()
True

So the value in func_defaults is the same which is as well known inside function (and returned in my example in order to access it from outside.

In other words, what happens when calling f() is an implicit x = f.func_defaults[0]. If that object is modified subsequently, you'll keep that modification.

In contrast, an assignment inside the function gets always a new []. Any modification will last until the last reference to that [] has gone; on the next function call, a new [] is created.

In order words again, it is not true that [] gets the same object on every execution, but it is (in the case of default argument) only executed once and then preserved.

4
  • 2
    Thank you very much; the sentence "what happens when calling f() is an implicit x = f.func_defaults[0]" was vital to my understanding. May 20, 2012 at 21:10
  • 1
    …so much so that I'm changing my mind, again, and marking this as the correct answer. May 21, 2012 at 1:35
  • To drive the point home: The assignment x=[] (in the function definition) is executed via proxy, the first part f.__defaults__[0] = [] during definition, the second part x = f.__defaults__[0] during invocation. Apr 9, 2021 at 13:23
  • @user985366 "IOW" is not unusual at all. But better be explicit than implicit, you are right.
    – glglgl
    Dec 20, 2021 at 21:05
17

The problem only exists if the default value is mutable, which None is not. What gets stored along with the function object is the default value. When the function is called, the function's context is initialized with the default value.

a_list = []

just assigns a new object to the name a_list in the context of the current function call. It does not modify None in any way.

2
  • My impression is that the OP's mental model of assignment and scope was erroneous. I rewrote the answer to make that clearer.
    – phihag
    May 20, 2012 at 21:43
  • My mental model of assignment was indeed erroneous; in fact even now that I somewhat better understand the problem, it still may be. What I didn't understand was that when you do the a_list = None in the function definition, the function internally has another name for the same object, and that the parameter's visible name gets reassigned to that object at the beginning of every invocation of the function. May 21, 2012 at 1:24
4

No, in good_append a_list is not initalised only once.

Each time the function is called without specifying the a_list argument, the default is used and a new instance of list is used and returned, the new list does not replace the default value.

0

The python tutorial says that

the default value is evaluated only once.

The evaluated (only once) default value is stored internally (name it x for simplicity).

case []: When you define the function with a_list defaulted to [], if you don't provide a_list, it is assigned the internal variable x when . Therefore, when you append to a_list, you are actually appending to x (because a_list and x refer to the same variable now). When you call the function again without a_list, the updated x is re-assigned to a_list.

case None: The value None is evaluated once and stored in x. If you don't provide, a_list, the variable x is assigned to a_list. But you don't append to x of course. You reassign an empty array to a_list. At this point x and a_list are different variables. The same way when you call the function again without a_list, it first gets the value None from x but then a_list gets assigned to an empty array again.

Note that, for the a_list = [] case, if you provide an explicit value for a_list when you call the function, the new argument does not override x because that's evaluated only once.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.