52

I am trying to iterate the lambda func over a list as in test.py, and I want to get the call result of the lambda, not the function object itself. However, the following output really confused me.

------test.py---------
#!/bin/env python
#coding: utf-8

a = [lambda: i for i in range(5)]
for i in a:
    print i()

--------output---------
<function <lambda> at 0x7f489e542e60>
<function <lambda> at 0x7f489e542ed8>
<function <lambda> at 0x7f489e542f50>
<function <lambda> at 0x7f489e54a050>
<function <lambda> at 0x7f489e54a0c8>

I modified the variable name when print the call result to t as following, and everything goes well. I am wondering what is all about of that. ?

--------test.py(update)--------
a = [lambda: i for i in range(5)]
for t in a:
    print t()

-----------output-------------
4
4
4
4
4

This behaviour is 2.x-specific, and is a special case of the problem described at What do lambda function closures capture?. In 3.x, the list comprehension creates its own scope for the iteration variable, so it isn't modifiable from outside. It is, however, still late binding, so each print will produce 4.

4
  • 1
    Sorry for closing, I misread the question. Commented Jul 14, 2016 at 9:03
  • 1
    Whoops, sorry for flagging. I misread as well! Commented Jul 14, 2016 at 9:28
  • 3
    I changed the title because it was really easy to skim your question and think you had a simple late-binding problem instead of a problem related to reusing the variable name. (This is evidenced in the mistaken closures and answers.) The new title calls attention to the real issue you're concerned about. If you want to reword it from what I've made it, please make sure you preserve that aspect.
    – jpmc26
    Commented Jul 14, 2016 at 16:02
  • 4
    The above demonstration of civility and humility is what I love about the Python community, especially those Pythonistas who frequent SO. Commented Jul 14, 2016 at 17:14

3 Answers 3

47

In Python 2 list comprehension 'leaks' the variables to outer scope:

>>> [i for i in xrange(3)]
[0, 1, 2]
>>> i
2

Note that the behavior is different on Python 3:

>>> [i for i in range(3)]
[0, 1, 2]
>>> i
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined

When you define lambda it's bound to variable i, not its' current value as your second example shows. Now when you assign new value to i the lambda will return whatever is the current value:

>>> a = [lambda: i for i in range(5)]
>>> a[0]()
4
>>> i = 'foobar'
>>> a[0]()
'foobar'

Since the value of i within the loop is the lambda itself you'll get it as a return value:

>>> i = a[0]
>>> i()
<function <lambda> at 0x01D689F0>
>>> i()()()()
<function <lambda> at 0x01D689F0>

UPDATE: Example on Python 2.7:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [lambda: i for i in range(5)]
>>> for i in a:
...     print i()
... 
<function <lambda> at 0x7f1eae7f15f0>
<function <lambda> at 0x7f1eae7f1668>
<function <lambda> at 0x7f1eae7f16e0>
<function <lambda> at 0x7f1eae7f1758>
<function <lambda> at 0x7f1eae7f17d0>

Same on Python 3.4:

Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [lambda: i for i in range(5)]
>>> for i in a:
...     print(i())
... 
4
4
4
4
4

For details about the change regarding the variable scope with list comprehension see Guido's blogpost from 2010.

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension "leaks" the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

However, in Python 3, we decided to fix the "dirty little secret" of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) :-) will print 'before', proving that the 'x' in the list comprehension temporarily shadows but does not override the 'x' in the surrounding scope.

10
  • 2
    The leaky scope of list comprehensions in python 2 doesn't affect the behavior in the OP. Commented Jul 14, 2016 at 9:23
  • 2
    @juanpa.arrivillaga Have you tried running the code in question with both versions? At least I get different results with Python 2.7 & 3.4.
    – niemmi
    Commented Jul 14, 2016 at 9:25
  • Whoops! I totally misread this question! You're totally right! This is surprising behavior to me... Commented Jul 14, 2016 at 9:29
  • 1
    On second thought, this is not surprising behavior at all. Commented Jul 14, 2016 at 9:36
  • Thanks very much for your detailed explanation, and I got it! ;)
    – keyuan7569
    Commented Jul 14, 2016 at 9:43
19

Closures in Python are late-binding, meaning that each lambda function in the list will only evaluate the variable i when invoked, and not when defined. That's why all functions return the same value, i.e. the last value of ì (which is 4).

To avoid this, one technique is to bind the value of i to a local named parameter:

>>> a = [lambda i=i: i for i in range(5)]
>>> for t in a:
...   print t()
... 
0
1
2
3
4

Another option is to create a partial function and bind the current value of i as its parameter:

>>> from functools import partial
>>> a = [partial(lambda x: x, i) for i in range(5)]
>>> for t in a:
...   print t()
... 
0
1
2
3
4

Edit: Sorry, misread the question initially, since these kind of questions are so often about late binding (thanks @soon for the comment).

The second reason for the behavior is list comprehension's variable leaking in Python2 as others have already explained. When using i as the iteration variable in the for loop, each function prints the current value of i (for the reasons stated above), which is simply the function itself. When using a different name (e.g. t), functions print the last value of i as it was in the list comprehension loop, which is 4.

4
  • Well, but why changing the variable name in the for loop changes the output?
    – awesoon
    Commented Jul 14, 2016 at 9:14
  • @soon What variable are you referring to, the t? That does not affect anything, it's just a temporary name we give to each function when iterating over the list. Or did I misunderstand the question?
    – plamut
    Commented Jul 14, 2016 at 9:16
  • 1
    My interpretation of the question was "why for i in a prints lambdas, but for k in a prints numbers?". Of course, this is because of late-binding too, but this may be unclear for the OP. +1 anyway
    – awesoon
    Commented Jul 14, 2016 at 9:26
  • Ah, I see, I may have jumped into the late binding behavior explanation too hastily. :) It's because of the variable leaking in Python 2 as others have already explained.
    – plamut
    Commented Jul 14, 2016 at 9:32
4

lambda: i is an anonymous function with no arguments that returns i. So you are generating a list of anonymous functions, which you can later (in the second example) bind to the name t and invoke with (). Note you can do the same with non-anonymous functions:

>>> def f():
...   return 42
... 
>>> name = f
>>> name
<function f at 0x7fed4d70fb90>
>>> name()
42

@plamut has just answered the implied other part of the question, so I won't.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.