20

I've found myself in an unusual situation where I need to change the MRO of a class at runtime.

The code:

class A(object):
    def __init__(self):
        print self.__class__
        print "__init__ A"
        self.hello()

    def hello(self):
        print "A hello"

class B(A):
    def __init__(self):
        super(B, self).__init__()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        print "%s hello" % self.msg_str

a = A()
b = B()

As to be expected, this fails as the __init__ method of A (when called from B) calls B's hello which attempts to access an attribute before it exists.

The issue is that I'm constrained in the changes I can make:

  • B must subclass A
  • A cannot be changed
  • Both A and B require a hello method
  • B cannot initialise other attributes before calling the super __init__

I did solve this conceptually, by changing the MRO at runtime. In brief, during B's __init__, but before calling super __init__, the MRO would be changed so that A would be searched for methods first, thereby calling A's hello instead of B's (and therefore failing).

The issue is that MRO is read only (at class runtime).

Is there another way to implement this ? Or possibly a different solution altogether (that still respects the aforementioned constraints) ?

9
  • Must B actually subclass A, or does it just have to appear as a subclass of A?
    – Blender
    Dec 29, 2013 at 7:11
  • @Blender, I'm very sure it must subclass, not just appear to have the same attributes/methods.
    – dilbert
    Dec 29, 2013 at 7:15
  • 1
    Maybe I should word that differently: do you use any property of B being a subclass of A other than issubclass(B, A) returning True?
    – Blender
    Dec 29, 2013 at 7:17
  • 2
    Can you explain the situation that allows you to set B.__mro__ before calling the superclass __init__, but doesn't allow you to just set self.msg_str at that time?
    – BrenBarn
    Dec 29, 2013 at 7:53
  • 1
    Such manipulation of MRO is not thread safe, and can result with unexpected behavior
    – shx2
    Dec 29, 2013 at 11:20

9 Answers 9

24

The other provided answers are advisable if you are not bound by the constraints mentioned in the question. Otherwise, we need to take a journey into mro hacks and metaclass land.

After some reading, I discovered you can change the mro of a class, using a metaclass.

This however, is at class creation time, not at object creation time. Slight modification is necessary.

The metaclass provides the mro method, which we overload, that is called during class creation (the metaclass' __new__ call) to produce the __mro__ attribute.

The __mro__ attribute is not a normal attribute, in that:

  1. It is read only
  2. It is defined BEFORE the metaclass' __new__ call

However, it appears to be recalculated (using the mro method) when a class' base is changed. This forms the basis of the hack.

In brief:

  • The subclass (B) is created using a metaclass (change_mro_meta). This metaclass provides:
    • An overloaded mro method
    • Class methods to change the __mro__ attribute
    • A class attribute (change_mro) to control the mro behaviour

As mentioned, modifying the mro of a class while in its __init__ is not thread safe.

The following may disturb some viewers. Viewer discretion is advised.

The hack:

class change_mro_meta(type):
    def __new__(cls, cls_name, cls_bases, cls_dict):
        out_cls = super(change_mro_meta, cls).__new__(cls, cls_name, cls_bases, cls_dict)
        out_cls.change_mro = False
        out_cls.hack_mro   = classmethod(cls.hack_mro)
        out_cls.fix_mro    = classmethod(cls.fix_mro)
        out_cls.recalc_mro = classmethod(cls.recalc_mro)
        return out_cls

    @staticmethod
    def hack_mro(cls):
        cls.change_mro = True
        cls.recalc_mro()

    @staticmethod
    def fix_mro(cls):
        cls.change_mro = False
        cls.recalc_mro()

    @staticmethod
    def recalc_mro(cls):
        # Changing a class' base causes __mro__ recalculation
        cls.__bases__  = cls.__bases__ + tuple()

    def mro(cls):
        default_mro = super(change_mro_meta, cls).mro()
        if hasattr(cls, "change_mro") and cls.change_mro:
            return default_mro[1:2] + default_mro
        else:
            return default_mro

class A(object):
    def __init__(self):
        print "__init__ A"
        self.hello()

    def hello(self):
        print "A hello"

class B(A):
    __metaclass__ = change_mro_meta
    def __init__(self):
        self.hack_mro()
        super(B, self).__init__()
        self.fix_mro()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        print "%s hello" % self.msg_str

a = A()
b = B()

Some notes:

The hack_mro, fix_mro and recalc_mro methods are staticmethods to the metaclass but classmethods to the class. It did this, instead of multiple inheritance, because I wanted to group the mro code together.

The mro method itself returns the default ordinarily. Under the hack condition, it appends the second element of the default mro (the immediate parent class) to the mro, thereby causing the parent class to see its own methods first before the subclass'.

I'm unsure of the portability of this hack. Its been tested on 64bit CPython 2.7.3 running on Windows 7 64bit.

Don't worry, I'm sure this won't end up in production code somewhere.

0
2

There may be grander solutions but a simple option is to write class B defensively. For example:

class B(A):
    def __init__(self):
        super(B, self).__init__()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        if not hasattr(self, 'msg_str'):
            A.hello(self)
            return
        print "%s hello" % self.msg_str

A good editor with regex capability could auto-insert appropriate if not hasattr(self, 'some_flag'):... lines as the first lines of any methods in B.

3
  • I did consider this, however this would require a large refactoring as (although not shown in the example code) there are quite a number of methods (similar to hello) called in init. I'm still holding out for a MRO based hack/solution.
    – dilbert
    Dec 29, 2013 at 6:59
  • @dilbert, OK. I'll be interested to see what mro hacks others can come up with.
    – John1024
    Dec 29, 2013 at 7:41
  • 3
    A possible extension of this would be to create a decorator called something like needs_attr that is passed the name of an attribute (or a list of attributes) that the method needs. You could then decorate hello with @needs_attr('msg_str') and have it handle the delegation to the superclass when necessary.
    – BrenBarn
    Dec 29, 2013 at 7:50
1

For your particular example, one solution is to give B a class attribute to hold a default message:

class B(A):

    msg_str = "default msg"

    def __init__(self):
        super(B, self).__init__()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        print "%s hello" % self.msg_str

Usually this causes confusion, but in this case it might be helpful. If B.hello is called before the instance's msg_str is set, it will read the class one. Once the instance msg_str is set, it shadows the class one so that future accesses of self.msg_str will see the instance-specific one.

I don't quite understand why you can't set attributes before calling the superclass __init__. Depending on exactly what the underlying situation is, there may be other solutions.

1

My solution would be to ask for forgiveness:

class A(object):
    def __init__(self):
        print self.__class__
        print "__init__ A"
        self.hello()

    def hello(self):
        print "A hello"

class B(A):
    def __init__(self):
        super(B, self).__init__()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        try:
            print "%s hello" % self.msg_str
        except AttributeError:
            pass  # or whatever else you want

a = A()
b = B()

or if you do not want to refactor methods called from init:

class A(object):
    def __init__(self):
        print self.__class__
        print "__init__ A"
        self.hello()

    def hello(self):
        print "A hello"

class B(A):
    def __init__(self):
        try:
            super(B, self).__init__()
        except AttributeError:
            pass  # or whatever else you want
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        print "%s hello" % self.msg_str

a = A()
b = B()
1
  • Thanks for the suggestion. The example code has been greatly simplified, where there are many 'hello' methods, some of which very long. Putting all that code in a try/except block would mask a possibly legitimate exception.
    – dilbert
    Dec 30, 2013 at 2:53
1

I'd like to point out a solution which is very specific to the example you present in your question, and therefor unlikely to help. (But in case it does help at all...)

You can bypass hello's polymorphism by defining it as a class member, instead of a method.

class B(A):
    def __init__(self):
        super(B, self).__init__()
        print "__init__ B"
        self.msg_str = "B"
        self.hello = lambda: print "%s hello" % self.msg_str
        self.hello()

(A remains unchanged).

This solution will break if:

  • you subclass B and need to override hello in the subclass
  • msg_str is modified after __init__ runs
  • probably several other cases...
1
  • I hadn't thought of that though it doesn't quite 'solve' the problem (the example code doesn't exactly line up with my problem, my apologies). Thanks for the info.
    – dilbert
    Dec 30, 2013 at 2:48
1

I've found a way to change object's class or rewrite it's mro.

The easiest way is to build a new class with type function:

def upgrade_class(obj, old_class, new_class):
    if obj.__class__ is old_class:
        obj.__class__ = new_class
    else:
        mro = obj.__class__.mro()

        def replace(cls):
            if cls is old_class:
                return new_class
            else:
                return cls

        bases = tuple(map(replace, mro[1:]))
        old_base_class = obj.__class__
        new_class = type(old_base_class.__name__, bases, dict(old_base_class.__dict__))
        obj.__class__ = new_class
0

I don't know if it's relevant to the specific problem, but it seems to me that changing the MRO on the fly like that could be risky in a concurrent program, and could definitely have issues if any of these objects turn out to be created recursively.

A non-MRO-based solution occurs to me, depending on the nature of the errors this code would have encountered. (Keeping in mind that this is belated. Perhaps somebody else will want a different answer.)

Basically, each hello() method on B would be wrapped in a decorator. Something along the lines of

class deferring(object):

    def __init__(self, function):
        self.function = function

    def __get__(self, instance, owner):
        # Return an unbound method, or whatever, when called from B.
        if instance is None:
            return self.function.__get__(None, owner)
        else:
            # Indicate that an instance is ready via a flag.
            # May need to tweak this based on the exact problem.
            if hasattr(instance, '_set_up'):
                return self.function.__get__(instance, owner)
            else:
                # Walk the mro manually.
                for cls in owner.__mro__:
                    # Crazy inefficient. Possible to mitigate, but risky.
                    for name, attr in vars(cls).items():
                        if attr is self:
                            break
                    else:
                        continue
                    return getattr(super(cls, instance), name)
                else:
                    raise TypeError

If you don't want to go the descriptor route, it's also possible to do something like

def deferring(function):
    def wrapped(self, *args, **kwargs):
        if hasattr(self, '_set_up'):
            return function(self, *args, **kwargs)
        else:
            for cls in type(self).__mro__:
                for name, attr in vars(cls).items():
                    if attr is function:
                        break
                else:
                    continue
                return getattr(super(cls, self), name)(*args, **kwargs)
            else:
                raise TypeError
    return wrapped
0

Call self.msg_str = "B" before super(B, self).__init__().

0

This is not good practice and there's likely better way to solve your use-case. But in general, you can modify the instance __class__ at runtime, which will modify the mro of the instance (not the class).

For example:

class A:
  def f(self):
    print('a')

class B:
  def f(self):
    print('b')

b = B()
b.f()  # b
b.__class__ = A
b.f()  # a
assert type(b) is A

With your use-case, it would look like:

class A:
    def __init__(self):
        self.hello()

    def hello(self):
        print("A hello")

class B(A):
    def __init__(self):
        parent_init = super().__init__
        # While A.__init__ is called, inheritance is removed
        self.__class__ = A
        parent_init()
        self.__class__ = B  # Restore inheritance

        self.msg_str = "B"

    def hello(self):
        print("%s hello" % self.msg_str)

a = A()  # A hello
b = B()  # A hello  (called in A.__init__)
b.hello()  # B hello

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.