After Mark Shannon's optimisation of Python objects, is a plain object different from an object with slots? I understand that after this optimisation in a normal use case, objects have no dictionary. Have the new Python objects made the use of slots totally unnecessary?
1 Answer
No, __slots__
still produces more compact objects.
With __slots__
for attributes, an object's memory layout only needs one pointer per slot. With the new lazy __dict__
creation, an object needs to store a PyDictValues
object and a pointer to that PyDictValues
object.
The PyDictValues
object contains room for a number of pointers based on the "usable size" of the shared keys object when the PyDictValues
object is created, which is usually more pointers than what you would have gotten with __slots__
. It also holds some extra metadata and padding, stored before those pointers: a "prefix size" representing the size of this metadata, a "used size" representing the number of values stored, and an array of bytes tracking insertion order. The padding is used to ensure this extra metadata doesn't interfere with the alignment of the pointers. (None of this is reflected in the PyDictValues
struct definition, since C struct definitions aren't expressive enough for this - it mostly has to be handled manually.)
So, without __slots__
, you've got an extra PyDictValues *
in the object directly, usually more room allocated for attribute pointers than necessary, and a bunch of extra metadata in the PyDictValues
itself, relative to using __slots__
.
Plus, with __slots__
, if you don't explicitly declare a __weakref__
slot, you don't get one, saving some memory at the cost of not being able to create weak references to your objects. With no __slots__
, Python will automatically add a __weakref__
slot to your objects.