15

With the following code :

import types

class Foo():
    def __getitem__(self, x):
        return x

def new_get(self, x):
    return x + 1

x = Foo()
x.__getitem__ = types.MethodType(new_get, x)

x.__getitem__(42) will return 43, but x[42] will return 42.

Is there a way to override __getitem__ at instance level in Python?

7
  • I'm sure there is
    – WiseDev
    Commented Aug 8, 2019 at 13:16
  • 2
    You wouldn't want to do this anyway. An object with a different method definition isn't, conceptually, part of the class anymore.
    – chepner
    Commented Aug 8, 2019 at 13:18
  • If you want the full story: I want to time some process, so basically, I override methods of the instance I want to time by adding code to time the call to those methods, in particular, I want to do that for call to __getitem__. (The idea is to modify an object, pass it through some existing code and record the time it took at each call of a method). Commented Aug 8, 2019 at 13:23
  • 1
    Related: “implicit uses of special methods always rely on the class-level binding of the special method” (the text is wrong actually; not all special methods rely on the class-level binding, but you can't count on any given one of them not doing so, and it could change from version to version because instance level rebinding is not officially supported for any of them). Commented Aug 8, 2019 at 14:39
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? Commented Mar 16, 2023 at 1:20

4 Answers 4

11

This is unfortunately, and quite surprisingly, not allowed:

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

Source: https://docs.python.org/3/reference/datamodel.html#special-lookup

5
  • 1
    Searched on the docs a little and couldn't find this info. Thanks a lot ! I guess i'll have to find another solution to achieve my desired solution. Commented Aug 8, 2019 at 13:21
  • 1
    An attempt at this: x.__class__.__bases__.__getitem__ = new_get gives AttributeError: 'tuple' object attribute '__getitem__' is read-only, so you're right.
    – WiseDev
    Commented Aug 8, 2019 at 13:22
  • It's not exactly unfortunate; if they allowed it, you'd be able to, for example, subscript the sequence classes themselves, e.g. list[2], and rather than immediately rejecting that as nonsensical, it would have to have a check in list.__getitem__ to make sure it was passed a list instance, not the class itself, every time. Looking it up on the class only also speeds things up in other ways (99.9% of the time it would come from C level slots on the class, but you'd have to check the instance dictionary every time to make sure it wasn't the < 0.1% case where it was overridden). Commented Aug 8, 2019 at 14:37
  • @ShadowRanger Yeah, just meant unfortunately for OP's question.
    – ruohola
    Commented Aug 8, 2019 at 14:53
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? Commented Mar 16, 2023 at 1:20
5

Don't do it...

The item lookup protocol will always recover __getitem__ from the class, it will not even look at instance __dict__. This is actually a good thing in general as doing otherwise would allow instances of the same class to be conceptually different from one another, which goes against the whole idea behind classes.

But...

Nonetheless, there are situation where this could potentially be helpful, by example when monkey-patching for test purpose.

Because the dunder is looked up directly at class level, the item lookup logic must also be updated at the class level.

A solution is thus to update __getitem__ so that it first looks for an instance-level function in the instance __dict__.

Here is an example where we are subclassing dict to allow for instance-level __getitem__.

class Foo(dict):
    def __getitem__(self, item):
        if "instance_getitem" in self.__dict__:
            return self.instance_getitem(self, item)
        else:
            return super().__getitem__(item)

foo = Foo()
foo.instance_getitem = lambda self, item: item + 1
print(foo[1]) # 2
5
  • 1
    This is a very good suggestion, but I don't think it will help me in my case. The idea is that I want to time some calls of __getitem__ inside some code not written by me. So I take an object, modify it's __getitem__ method to record time on each call, and just pass it to the code not written by me. I'll just have to do it on class level instead of instance level. Commented Aug 8, 2019 at 13:26
  • 1
    @StatisticDean The above snippet of code shows you how to do what you want with inheritance Commented Aug 8, 2019 at 14:33
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? Commented Mar 16, 2023 at 1:21
  • @CharlieParker This solution works with Python3 if that is what you are asking. Although, you can also directly use a dict which contain functions which you can call explicitly. Commented Mar 16, 2023 at 1:51
  • I just want d.__getitem__ = new_getitem to work. Your solution requires me to hard code per object e.g. dict the behaviour. I have multiple data set objects that need this as an input to modify the class. Basically, I don't want to write a new class. Commented Mar 16, 2023 at 17:48
2

You can make __getitem__ a descriptor that would first try to return the same-named attribute of a given instance before defaulting to binding the original method to the instance:

class customizable:
    def __init__(self, method):
        self.method = method

    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, obj_type=None):
        if obj:
            try:
                return vars(obj)[self.name]
            except KeyError:
                return self.method.__get__(obj, obj_type)
        return self.method

so that:

import types

class Foo():
    @customizable
    def __getitem__(self, x):
        return x

def new_get(self, x):
    return x + 1

x = Foo()
y = Foo()
x.__getitem__ = types.MethodType(new_get, x)
print(Foo.__getitem__)
print(x[42])
print(y[42])

outputs:

<function Foo.__getitem__ at 0x154f7e55c310>
43
42

Demo: https://ideone.com/fLjlWD

Note that if you don't own the code of Foo you can either patch its __getitem__ method with the descriptor:

Foo.__getitem__ = customizable(Foo.__getitem__) # would not work on built-in types

or override __getitem__ in a subclass of Foo instead:

class Bar(Foo):
    __getitem__ = customizable(Foo.__getitem__)
1

I ended up having to do something stupid like this just making a new object, calls the old __getitem__ and does something different:

class USLDatasetFromL2L(datasets.Dataset):

    def __init__(self, original_l2l_dataset: datasets.Dataset):
        self.original_l2l_dataset = original_l2l_dataset
        self.transform = self.original_l2l_dataset.transform
        self.original_l2l_dataset.target_transform = label_to_long
        self.target_transform = self.original_l2l_dataset.target_transform

    def __getitem__(self, index: int) -> tuple:
        """ overwrite the getitem method for a l2l dataset. """
        # - get the item
        img, label = self.original_l2l_dataset[index]
        # - transform the item only if the transform does exist and its not a tensor already
        # img, label = self.original_l2l_dataset.x, self.original_l2l_dataset.y
        if self.transform and not isinstance(img, Tensor):
            img = self.transform(img)
        if self.target_transform and not isinstance(label, Tensor):
            label = self.target_transform(label)
        return img, label

    def __len__(self) -> int:
        """ Get the length. """
        return len(self.original_l2l_dataset)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.