31

EDIT: Looks like this is a very old "bug" or, actually, feature. See, e.g., this mail

I am trying to understand the Python scoping rules. More precisely, I thought that I understand them but then I found this code here:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        print(x)
        print(y)
        y = 1
func()

In Python 3.4 the output is:

xlocal
ytop

If I replace the inner class by a function then it reasonably gives UnboundLocalError. Could you explain me why it behaves this strange way with classes and what is the reason for such choice of scoping rules?

11
  • 1
    It wont give any error unless you call the inner function Mar 29, 2015 at 18:56
  • 3
    Interesting; y should be an unbound exception in the case of y, but it isn't even when you make C a global. Corner cases FTW! Class bodies are special; they do not create a new scope of their own. As such y = 1 would make it a local, but until that point it is a global in class statements only.
    – Martijn Pieters
    Mar 29, 2015 at 18:57
  • 2
    9.2. Python Scopes and Namespaces and 4.1. Naming and binding are a start, you might need to re-read them often till it sinks in.
    – wwii
    Mar 29, 2015 at 18:57
  • 2
    if it's actual python3 code, then it should be print(x) and print(y). print x is python2 code.
    – zmo
    Mar 29, 2015 at 18:59
  • 4
    This question could be helpful stackoverflow.com/questions/4296677/…
    – Mazdak
    Mar 29, 2015 at 19:19

2 Answers 2

13

TL;DR: This behaviour has existed since Python 2.1 PEP 227: Nested Scopes, and was known back then. If a name is assigned to within a class body (like y), then it is assumed to be a local/global variable; if it is not assigned to (x), then it also can potentially point to a closure cell. The lexical variables do not show up as local/global names to the class body.


On Python 3.4, dis.dis(func) shows the following:

>>> dis.dis(func)
  4           0 LOAD_CONST               1 ('xlocal')
              3 STORE_DEREF              0 (x)

  5           6 LOAD_CONST               2 ('ylocal')
              9 STORE_FAST               0 (y)

  6          12 LOAD_BUILD_CLASS
             13 LOAD_CLOSURE             0 (x)
             16 BUILD_TUPLE              1
             19 LOAD_CONST               3 (<code object C at 0x7f083c9bbf60, file "test.py", line 6>)
             22 LOAD_CONST               4 ('C')
             25 MAKE_CLOSURE             0
             28 LOAD_CONST               4 ('C')
             31 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
             34 STORE_FAST               1 (C)
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

The LOAD_BUILD_CLASS loads the builtins.__build_class__ on the stack; this is called with arguments __build_class__(func, name); where func is the class body, and name is 'C'. The class body is the constant #3 for the function func:

>>> dis.dis(func.__code__.co_consts[3])
  6           0 LOAD_NAME                0 (__name__)
              3 STORE_NAME               1 (__module__)
              6 LOAD_CONST               0 ('func.<locals>.C')
              9 STORE_NAME               2 (__qualname__)

  7          12 LOAD_NAME                3 (print)
             15 LOAD_CLASSDEREF          0 (x)
             18 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             21 POP_TOP

  8          22 LOAD_NAME                3 (print)
             25 LOAD_NAME                4 (y)
             28 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             31 POP_TOP

  9          32 LOAD_CONST               1 (1)
             35 STORE_NAME               4 (y)
             38 LOAD_CONST               2 (None)
             41 RETURN_VALUE

Within the class body, x is accessed with LOAD_CLASSDEREF (15) while y is load with LOAD_NAME (25). The LOAD_CLASSDEREF is a Python 3.4+ opcode for loading values from closure cells specifically within class bodies (in previous versions, the generic LOAD_DEREF was used); the LOAD_NAME is for loading values from locals and then globals. However closure cells show up neither as locals nor globals.

Now, because the name y is stored to within the class body (35), it is consistently being used as not a closure cell but a local/global name. The closure cells do not show up as local variables to the class body.

This behaviour has been true ever since implementing PEP 227 - nested scopes. And back then BDFL stated that this should not be fixed - and thus it has been for these 13+ years.


The only change since PEP 227 is the addition of nonlocal in Python 3; if one uses it within the class body, the class body can set the values of the cells within the containing scope:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        nonlocal y  # y here now refers to the outer variable
        print(x)
        print(y)
        y = 1

    print(y)
    print(C.y)

func()

The output now is

xlocal
ylocal
1
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    func()
  File "test.py", line 13, in func
    print(C.y)
AttributeError: type object 'C' has no attribute 'y'

That is, print(y) read the value of the cell y of the containing scope, and y = 1 set the value in that cell; in this case, no attribute was created for the class C.

3
  • It seems that I understood your explanation. Since y is stored it "bypasses" the usual mechanism. But doesn't it look like a bug to you?
    – ivanl
    Mar 29, 2015 at 20:59
  • 5
    If Guido says it is not a bug, then it is a feature. ;) Mar 29, 2015 at 21:25
  • 1
    @ivanl: it is a known 'feature' of the implementation; see the email Antti links to. Class scopes are not like function scopes.
    – Martijn Pieters
    Mar 29, 2015 at 21:25
7

First focus on the case of a closure -- a function within a function:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    def inner():
 #       global y
        print(x)
        print(y)
        y='inner y'
        print(y)
    inner()  

Note the commented out global in inner If you run this, it replicates the UnboundLocalError you got. Why?

Run dis.dis on it:

>>> import dis
>>> dis.dis(func)
  6           0 LOAD_CONST               1 ('xlocal')
              3 STORE_DEREF              0 (x)

  7           6 LOAD_CONST               2 ('ylocal')
              9 STORE_FAST               0 (y)

  8          12 LOAD_CLOSURE             0 (x)
             15 BUILD_TUPLE              1
             18 LOAD_CONST               3 (<code object inner at 0x101500270, file "Untitled 3.py", line 8>)
             21 LOAD_CONST               4 ('func.<locals>.inner')
             24 MAKE_CLOSURE             0
             27 STORE_FAST               1 (inner)

 14          30 LOAD_FAST                1 (inner)
             33 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             36 POP_TOP
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

Note the different access mode of x vs y inside of func. The use of y='inner y' inside of inner has created the UnboundLocalError

Now uncomment global y inside of inner. Now you have unambiguously create y to be the top global version until resigned as y='inner y'

With global uncommented, prints:

xlocal
ytop
inner y

You can get a more sensible result with:

x = "xtop"
y = "ytop"
def func():
    global y, x
    print(x,y)
    x = "xlocal"
    y = "ylocal"
    def inner():
        global y
        print(x,y)
        y = 'inner y'
        print(x,y)
    inner()    

Prints:

xtop ytop
xlocal ylocal
xlocal inner y

The analysis of the closure class is complicated by instance vs class variables and what / when a naked class (with no instance) is being executed.

The bottom line is the same: If you reference a name outside the local namespace and then assign to the same name locally you get a surprising result.

The 'fix' is the same: use the global keyword:

x = "xtop"
y = "ytop"
def func():
    global x, y
    x = "xlocal"
    y = "ylocal"
    class Inner:
        print(x, y)
        y = 'Inner_y'
        print(x, y) 

Prints:

xlocal ylocal
xlocal Inner_y

You can read more about Python 3 scope rules in PEP 3104

6
  • 5
    I understand the behavior for functions you pointed out. But it is still not clear what is the reason for classes to behave differenly.
    – ivanl
    Mar 29, 2015 at 19:41
  • Classes will be similar but you need to further qualify instance vs class variables and nested class and nested instance variables. Bottom line is the same: The presence of the assignment of y=1 make the reference print(y) above the assignment ambiguous inside the closure.
    – dawg
    Mar 29, 2015 at 20:37
  • I would understand if my code would fail with UnboundLocal , but it looks really confusing how y can "bypass" y_local. Not failing with UnboundLocal really looks like a bug
    – ivanl
    Mar 29, 2015 at 20:51
  • 1
    There is a deference in how the interpreter treats each case, true. It is unusual to just have a class definition and execute code by its presence though.
    – dawg
    Mar 29, 2015 at 20:58
  • 4
    PEP 3104 only introduces the nonlocal statement and doesn't actually explain the behaviour. The behaviour is a long-standing PEP 227 side-effect, 3104 never altered that.
    – Martijn Pieters
    Mar 29, 2015 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.