47

Suppose I have code that maintains a parent/children structure. In such a structure I get circular references, where a child points to a parent and a parent points to a child. Should I worry about them? I'm using Python 2.5.

I am concerned that they will not be garbage collected and the application will eventually consume all memory.

6 Answers 6

41

"Worry" is misplaced, but if your program turns out to be slow, consume more memory than expected, or have strange inexplicable pauses, the cause is indeed likely to be in those garbage reference loops -- they need to be garbage collected by a different procedure than "normal" (acyclic) reference graphs, and that collection is occasional and may be slow if you have a lot of objects tied up in such loops (the cyclical-garbage collection is also inhibited if an object in the loop has a __del__ special method).

So, reference loops will not affect your program's correctness, but may affect its performance and/or footprint.

If and when you want to remove unwanted loops of references, you can often use the weakref module in Python's standard library.

If and when you want to exert more direct control (or perform debugging, see what exactly is happening) regarding cyclical garbage collection, use the gc module in Python's standard library.

1
  • Plus 1 for the note about __del__. If your object destructors have side effects, then you may want to think about cyclic references (and when things get destroyed) a bit more carefully.
    – speedplane
    Commented Oct 4, 2017 at 0:35
18

Experimentally: you're fine:

import itertools

for i in itertools.count():
    a = {}
    b = {"a":a}
    a["b"] = b

It consistently stays at using 3.6 MB of RAM.

4
  • Which implementation did you use? Commented Jun 12, 2014 at 12:56
  • @SargeBorsch CPython 2.something. I imagine any of the major implementations would behave the same way though.
    – cobbal
    Commented Jun 12, 2014 at 17:02
  • That's short term object life cycle and no more than 2 are used at any time. Can we expect the same in other scenarios?
    – Jimmy T.
    Commented Jul 5, 2021 at 18:01
  • @JimmyT. TL;DR: No, nothing is guaranteed, and you can't count on specific garbage collection strategies of any python implementation. It probably depends on the reference-counted implementation of the python interpreter. So probably you can count on the behavior in any situation where CPython is used, and no closures or other references are grabbing onto the values, but you probably can't count on it where a pure mark-and-sweep or stop-and-copy implementation is used. In short, nothing is guaranteed. It all depends. Many different garbage collectors are valid by the standard.
    – cobbal
    Commented Jul 6, 2021 at 5:59
13

Python will detect the cycle and release the memory when there are no outside references.

1
  • 2
    Assuming of course there are no __del__ methods. Which there normally shouldn't be, but you never know. For a while, even collections.OrderedDict had one for some reason.
    – Antimony
    Commented Oct 1, 2012 at 4:52
7

Circular references are a normal thing to do, so I don't see a reason to be worried about them. Many tree algorithms require that each node have links to its children and its parent. They're also required to implement something like a doubly linked list.

2
  • 1
    Thanks Colin. I didn't know they were "a normal thing." They seemed very special to me. But now I learned otherwise. :)
    – bodacydo
    Commented Mar 11, 2010 at 20:21
  • Also, they are obviously required for graphs.
    – Antimony
    Commented Oct 1, 2012 at 4:41
3

I don't think you should worry. Try the following program and will you see that it won't consume all memory:

while True:
    a=range(100)
    b=range(100)
    a.append(b)
    b.append(a)
    a.append(a)
    b.append(b)
2
  • don't you mean a.extend(b), not append?
    – richizy
    Commented Feb 23, 2014 at 19:47
  • 4
    @richizy I really mean append because I want to save the reference to a and b inside a and b, not the values. In this way the circular reference will happen.
    – douglaz
    Commented Feb 25, 2014 at 2:55
3

There seems to be a issue with references to methods in lists in a variable. Here are two examples. The first one does not call __del__. The second one with weakref is ok for __del__. However, in this later case the problem is that you cannot weakly reference methods: http://docs.python.org/2/library/weakref.html

import sys, weakref

class One():
    def __init__(self):
        self.counters = [ self.count ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(One)
one = One()
sys.getrefcount(One)
del one
sys.getrefcount(One)


class Two():
    def __init__(self):
        self.counters = [ weakref.ref(self.count) ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(Two)
two = Two()
sys.getrefcount(Two)
del two
sys.getrefcount(Two)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.