145

Is there a reference for the memory size of Python data stucture on 32- and 64-bit platforms?

If not, this would be nice to have it on SO. The more exhaustive the better! So how many bytes are used by the following Python structures (depending on the len and the content type when relevant)?

  • int
  • float
  • reference
  • str
  • unicode string
  • tuple
  • list
  • dict
  • set
  • array.array
  • numpy.array
  • deque
  • new-style classes object
  • old-style classes object
  • ... and everything I am forgetting!

(For containers that keep only references to other objects, we obviously do not want to count the size of the item themselves, since it might be shared.)

Furthermore, is there a way to get the memory used by an object at runtime (recursively or not)?

3

8 Answers 8

167

The recommendation from an earlier question on this was to use sys.getsizeof(), quoting:

>>> import sys
>>> x = 2
>>> sys.getsizeof(x)
14
>>> sys.getsizeof(sys.getsizeof)
32
>>> sys.getsizeof('this')
38
>>> sys.getsizeof('this also')
48

You could take this approach:

>>> import sys
>>> import decimal
>>> 
>>> d = {
...     "int": 0,
...     "float": 0.0,
...     "dict": dict(),
...     "set": set(),
...     "tuple": tuple(),
...     "list": list(),
...     "str": "a",
...     "unicode": u"a",
...     "decimal": decimal.Decimal(0),
...     "object": object(),
... }
>>> for k, v in sorted(d.iteritems()):
...     print k, sys.getsizeof(v)
...
decimal 40
dict 140
float 16
int 12
list 36
object 8
set 116
str 25
tuple 28
unicode 28

2012-09-30

python 2.7 (linux, 32-bit):

decimal 36
dict 136
float 16
int 12
list 32
object 8
set 112
str 22
tuple 24
unicode 32

python 3.3 (linux, 32-bit)

decimal 52
dict 144
float 16
int 14
list 32
object 8
set 112
str 26
tuple 24
unicode 26

2016-08-01

OSX, Python 2.7.10 (default, Oct 23 2015, 19:19:21) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin

decimal 80
dict 280
float 24
int 24
list 72
object 16
set 232
str 38
tuple 56
unicode 52
11
  • 1
    Thanks, and sorry for the dupe for the second question... too bad I am using 2.5 and not 2.6...
    – LeMiz
    Commented Aug 25, 2009 at 23:16
  • I forgot I had a virtual box with a recent ubuntu on it! That's strange, sys.getsizeof(dict) is 136 for me (python 2.6 running on a kubuntu vm, hosted by OS X, so I am not sure of anything)
    – LeMiz
    Commented Aug 25, 2009 at 23:39
  • @LeMiz: For me (Python 2.6, Windows XP SP3), sys.getsizeof(dict) -> 436; sys.getsizeof(dict()) -> 140 Commented Aug 26, 2009 at 10:58
  • LeMiz-Kubuntu:python2.6 Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.getsizeof(dict) 436 >>> sys.getsizeof(dict()) 136
    – LeMiz
    Commented Aug 26, 2009 at 15:43
  • 1
    shouldn't the values be 0, 0.0, '' and u'' for consistency? Commented Aug 26, 2009 at 16:38
62

These answers all collect shallow size information. I suspect that visitors to this question will end up here looking to answer the question, "How big is this complex object in memory?"

There's a great answer here: https://goshippo.com/blog/measure-real-size-any-python-object/

The punchline:

import sys

def get_size(obj, seen=None):
    """Recursively finds size of objects"""
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        size += sum([get_size(i, seen) for i in obj])
    return size

Used like so:

In [1]: get_size(1)
Out[1]: 24

In [2]: get_size([1])
Out[2]: 104

In [3]: get_size([[1]])
Out[3]: 184

If you want to know Python's memory model more deeply, there's a great article here that has a similar "total size" snippet of code as part of a longer explanation: https://code.tutsplus.com/tutorials/understand-how-much-memory-your-python-objects-use--cms-25609

4
  • So this should yield the total amount of memory used by, for example, a dict containing multiple array and/or other dicts ? Commented Oct 8, 2019 at 13:11
  • 1
    @CharlyEmpereur-mot yep.
    – Kobold
    Commented Oct 10, 2019 at 7:44
  • 1
    Great answer. It does however not seem to work for compiled cython objects. In my case this method returns 96 for a pointer to an in-memory cython object
    – ferdynator
    Commented Nov 27, 2019 at 12:42
  • moin. Google will bring up your answer depending on how the search is made - so, if you'd mind editing this to make it more correct taking into account __slot__ using objects and a few other edgecases, please check my answer bellow. Best regards!
    – jsbueno
    Commented Sep 5 at 0:44
39

I've been happily using pympler for such tasks. It's compatible with many versions of Python -- the asizeof module in particular goes back to 2.2!

For example, using hughdbrown's example but with from pympler import asizeof at the start and print asizeof.asizeof(v) at the end, I see (system Python 2.5 on MacOSX 10.5):

$ python pymp.py 
set 120
unicode 32
tuple 32
int 16
decimal 152
float 16
list 40
object 0
dict 144
str 32

Clearly there is some approximation here, but I've found it very useful for footprint analysis and tuning.

5
  • 1
    Some curiosities: most of you numbers are 4 higher; object is 0; and decimal is about 4 times larger by your estimate.
    – hughdbrown
    Commented Aug 26, 2009 at 2:26
  • 1
    Yep. The "4 higher" actually mostly look like "rounding up to a multiple of 8" which I believe is correct for the way malloc behaves here. No idea why decimal gets so distorted (with pympler on 2.6, too). Commented Aug 26, 2009 at 2:40
  • 2
    Actually, you should use pympler.asizeof.flatsize() to get a similar functionality to sys.getsizeof(). There is also an align= parameter you can use (which is 8 by default as Alex pointed out).
    – Pankrat
    Commented Sep 11, 2009 at 19:41
  • @AlexMartelli Hi Alex! .. Why minimum size of a char in python is 25 bytes. >>> getsizeof('a') gives 25 and >>> getsizeof('ab') gives 26 ` Commented Jan 17, 2013 at 5:13
  • 1
    I suppose size is in bytes, but why it's not written anywhere, even in pythonhosted.org/Pympler
    – Zhomart
    Commented Jan 26, 2016 at 1:08
11

Try memory profiler. memory profiler

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a
2
  • 1
    Precision seems to be 1/100MB, or 10.24 bytes. This is fine for macro-analysis, but I doubt that such precision would lead to an accurate comparison of the data structures as asked in the question. Commented Nov 30, 2014 at 18:49
  • True. But this is really convenient for class objects of which internal definition is not clear (and not only for basic datatypes, then).
    – Olivier
    Commented Jun 28 at 13:25
7

Also you can use guppy module.

>>> from guppy import hpy; hp=hpy()
>>> hp.heap()
Partition of a set of 25853 objects. Total size = 3320992 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11731  45   929072  28    929072  28 str
     1   5832  23   469760  14   1398832  42 tuple
     2    324   1   277728   8   1676560  50 dict (no owner)
     3     70   0   216976   7   1893536  57 dict of module
     4    199   1   210856   6   2104392  63 dict of type
     5   1627   6   208256   6   2312648  70 types.CodeType
     6   1592   6   191040   6   2503688  75 function
     7    199   1   177008   5   2680696  81 type
     8    124   0   135328   4   2816024  85 dict of class
     9   1045   4    83600   3   2899624  87 __builtin__.wrapper_descriptor
<90 more rows. Type e.g. '_.more' to view.>

And:

>>> hp.iso(1, [1], "1", (1,), {1:1}, None)
Partition of a set of 6 objects. Total size = 560 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  17      280  50       280  50 dict (no owner)
     1      1  17      136  24       416  74 list
     2      1  17       64  11       480  86 tuple
     3      1  17       40   7       520  93 str
     4      1  17       24   4       544  97 int
     5      1  17       16   3       560 100 types.NoneType
1

When you use the dir([object]) built-in function, you can get the __sizeof__ of the built-in function.

>>> a = -1
>>> a.__sizeof__()
24
0

One can also make use of the tracemalloc module from the Python standard library. It seems to work well for objects whose class is implemented in C (unlike Pympler, for instance).

0

more accurate rercursive get_size

take in account open files, __slots__ and memoryview objects

While I myself had used the pattern here a few times, and rewritten it from scratch, I needed it today for objects featuring a few edgecases -

Since google brings one here when lookiing for a Python recursive getsizeof, I think it can help some people if this version taking care of the edgecases can be found here.

Based on the original https://stackoverflow.com/a/40880923/108205 answer by @Kobold

import sys
from collections.abc import Mapping, Sequence
from _io import _IOBase

exclude_sequences = (bytes, str, bytearray, _IOBase, memoryview)

def get_size(obj, seen=None):
    """Recursively finds size of objects"""
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    size = sys.getsizeof(obj)
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif isinstance(obj, memoryview):
        size += get_size(obj.obj, seen)  # this way the memoryview target is taken in account in the "seen" mechanism"
    elif hasattr(obj, '__iter__') and not isinstance(obj, exclude_sequences):
        size += sum([get_size(i, seen) for i in obj])
    if hasattr(obj, "__slots__"):
        for attrname in obj.__slots__:
            if attr:=getattr(obj, attrname, None):
                size += get_size(attr, seen)
    return size

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.