8

I've gotten myself in trouble a few times now with accidentially (unintentionally) referencing global variables in a function or method definition.

My question is: is there any way to disallow python from letting me reference a global variable? Or at least warn me that I am referencing a global variable?

x = 123

def myfunc() :
    print x    # throw a warning or something!!!

Let me add that the typical situation where this arrises for my is using IPython as an interactive shell. I use 'execfile' to execute a script that defines a class. In the interpreter, I access the class variable directly to do something useful, then decide I want to add that as a method in my class. When I was in the interpreter, I was referencing the class variable. However, when it becomes a method, it needs to reference 'self'. Here's an example.

class MyClass :

    a = 1
    b = 2

    def add(self) :
        return a+b


m = MyClass()

Now in my interpreter I run the script 'execfile('script.py')', I'm inspecting my class and type: 'm.a * m.b' and decide, that would be a useful method to have. So I modify my code to be, with the non-intentional copy/paste error:

class MyClass :

    a = 1
    b = 2

    def add(self) :
        return a+b


    def mult(self) :
        return m.a * m.b   # I really meant this to be self.a * self.b

This of course still executes in IPython, but it can really confuse me since it is now referencing the previously defined global variable!

Maybe someone has a suggestion given my typical IPython workflow.

9
  • 10
    Why would you want that? Other functions are usually globals too. So are imports from other modules. Classes are usually globals, etc. You'd get warnings all the time.
    – Martijn Pieters
    May 23, 2013 at 18:41
  • @EpicAdv I assume your problem with global variable starts when you're updating its value somewhere else. May 23, 2013 at 18:44
  • PyLint shows a warning when you use the "global" statement.
    – Markon
    May 23, 2013 at 18:45
  • 4
    @Markon: I don't think that would help him. He's using a global variable accidentally, without using the global statement.
    – abarnert
    May 23, 2013 at 18:47
  • You can't reference a global variable that you don't define in the first place.
    – chepner
    May 23, 2013 at 18:55

2 Answers 2

4

First, you probably don't want to do this. As Martijn Pieters points out, many things, like top-level functions and classes, are globals.

You could filter this for only non-callable globals. Functions, classes, builtin-function-or-methods that you import from a C extension module, etc. are callable. You might also want to filter out modules (anything you import is a global). That still won't catch cases where you, say, assign a function to another name after the def. You could add some kind of whitelisting for that (which would also allow you to create global "constants" that you can use without warnings). Really, anything you come up with will be a very rough guide at best, not something you want to treat as an absolute warning.

Also, no matter how you do it, trying to detect implicit global access, but not explicit access (with a global statement) is going to be very hard, so hopefully that isn't important.


There is no obvious way to detect all implicit uses of global variables at the source level.

However, it's pretty easy to do with reflection from inside the interpreter.

The documentation for the inspect module has a nice chart that shows you the standard members of various types. Note that some of them have different names in Python 2.x and Python 3.x.

This function will get you a list of all the global names accessed by a bound method, unbound method, function, or code object in both versions:

def get_globals(thing):
    thing = getattr(thing, 'im_func', thing)
    thing = getattr(thing, '__func__', thing)
    thing = getattr(thing, 'func_code', thing)
    thing = getattr(thing, '__code__', thing)
    return thing.co_names

If you want to only handle non-callables, you can filter it:

def get_callable_globals(thing):
    thing = getattr(thing, 'im_func', thing)
    func_globals = getattr(thing, 'func_globals', {})
    thing = getattr(thing, 'func_code', thing)
    return [name for name in thing.co_names
            if callable(func_globals.get(name))]

This isn't perfect (e.g., if a function's globals have a custom builtins replacement, we won't look it up properly), but it's probably good enough.


A simple example of using it:

>>> def foo(myparam):
...     myglobal
...     mylocal = 1
>>> print get_globals(foo)
('myglobal',)

And you can pretty easily import a module and recursively walk its callables and call get_globals() on each one, which will work for the major cases (top-level functions, and methods of top-level and nested classes), although it won't work for anything defined dynamically (e.g., functions or classes defined inside functions).


If you only care about CPython, another option is to use the dis module to scan all the bytecode in a module, or .pyc file (or class, or whatever), and log each LOAD_GLOBAL op.

One major advantage of this over the inspect method is that it will find functions that have been compiled, even if they haven't been created yet.

The disadvantage is that there is no way to look up the names (how could there be, if some of them haven't even been created yet?), so you can't easily filter out callables. You can try to do something fancy, like connecting up LOAD_GLOBAL ops to corresponding CALL_FUNCTION (and related) ops, but… that's starting to get pretty complicated.


Finally, if you want to hook things dynamically, you can always replace globals with a wrapper that warns every time you access it. For example:

class GlobalsWrapper(collections.MutableMapping):
    def __init__(self, globaldict):
        self.globaldict = globaldict
    # ... implement at least __setitem__, __delitem__, __iter__, __len__
    # in the obvious way, by delegating to self.globaldict
    def __getitem__(self, key):
        print >>sys.stderr, 'Warning: accessing global "{}"'.format(key)
        return self.globaldict[key]

globals_wrapper = GlobalsWrapper(globals())

Again, you can filter on non-callables pretty easily:

    def __getitem__(self, key):
        value = self.globaldict[key]
        if not callable(value):
            print >>sys.stderr, 'Warning: accessing global "{}"'.format(key)
        return value

Obviously for Python 3 you'd need to change the print statement to a print function call.

You can also raise an exception instead of warning pretty easily. Or you might want to consider using the warnings module.

You can hook this into your code in various different ways. The most obvious one is an import hook that gives each new module a GlobalsWrapper around its normally-built globals. Although I'm not sure how that will interact with C extension modules, but my guess is that it will either work, or be harmlessly ignored, either of which is probably fine. The only problem is that this won't affect your top-level script. If that's important, you can write a wrapper script that execfiles the main script with a GlobalsWrapper, or something like that.

10
  • Thank you for the very thorough response. I'm intrigued by this last idea, but I'm struggling as how to exactly implement it. I get the error: Can't instantiate abstract class GlobalsWrapper with abstract methods delitem, iter, len, setitem
    – EpicAdv
    May 24, 2013 at 23:37
  • @EpicAdv: That's what the # ... is for. You need to implement all of the MutableMapping methods. The documentation explains this. It even has a complete example implementing Set on top of list—not quite the same as Mapping on top of dict, but the chart and text above should make it obvious what to do. (Also, anyone who doesn't know how to implement a collection ABC will probably not be able to implement an import hook.)
    – abarnert
    May 25, 2013 at 0:10
  • @abernert: Thanks for the clarification. I now am creating the GlobalsWrapper class by implementing all of the required methods. My confusion remains in how to best access the globals dict and wrap it. globals is a function the returns the global dict, so we can't instantiate GlobalsWrapper as you show (I think), rather I think it needs to be global_wrapper = GlobalsWrapper(globals()). I also added a __call__ method to the GlobalsWrapper that returns the dict, but that doesn't seem to work (e.g. globals_wrapper()['some_var'] doesn't warn me).
    – EpicAdv
    May 28, 2013 at 15:15
  • @EpicAdv: Yes, sorry, the missing parents in the constructor call were a typo. I'll edit that. But as for the rest… How did you hook this into your code? Just creating this object won't have any effect if you don't actually use it. Also, why did you add a __call__ method?
    – abarnert
    May 28, 2013 at 18:42
  • @EpicAdv: Anyway, when I try this, with a quick hacky test, where I just do globals = lambda: globals_wrapper then print sys and print asdfgh, I get a warnings printed to stderr followed by the module being printed out, then a warning printed to stderr followed by a KeyError traceback. Just as you'd expect.
    – abarnert
    May 28, 2013 at 18:48
2

I've been struggling with a similar challenge (especially in Jupyter notebooks) and created a small package to limit the scope of functions.

>>> from localscope import localscope
>>> a = 'hello world'
>>> @localscope
... def print_a():
...     print(a)
Traceback (most recent call last):
  ...
ValueError: `a` is not a permitted global

The @localscope decorator uses python's disassembler to find all instances of the decorated function using a LOAD_GLOBAL (global variable access) or LOAD_DEREF (closure access) statement. If the variable to be loaded is a builtin function, is explicitly listed as an exception, or satisfies a predicate, the variable is permitted. Otherwise, an exception is raised.

Note that the decorator analyses the code statically. Consequently, it does not have access to the values of variables accessed by closure.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.