0

Python basic types do not count inheritance as I expected. For example:

class MyUnicode(unicode):
    pass

mu = MyUnicode('xxx')

>>> type(mu)
<class '__main__.MyUnicode'> # ok

>>> type(mu + 'x')
<type 'unicode'> # why not <class '__main__.MyUnicode'> ?

>>> type(mu.strip())
<type 'unicode'> # why not <class '__main__.MyUnicode'> ?

Strings are immutable, then those two methods must return new objects. But why developers hardcoded the unicode return type inside those methods instead of using sub-class? Does it prevent some potential drawbacks, I'm not aware of?

1
  • I remember this has been asked before, but I cannot find the reference at the moment. Python's behaviour used to be rather inconsistent in the past -- strings and integers would return the base class, while e.g. sets would return your derived class. If I remember correctly, current Python 3 always returns the built-in base types and not the derived types. If you search for the commits in CPython that changed this behaviour you'll probably find the explanation why they did this. Jul 3, 2013 at 14:15

4 Answers 4

2

Personally I would like to see how can you implement unicode to work the way you want, without relying on some implementation details of the subclass.

If you'll try (it's a pseudocode for obvious reasons):

def __add__(self, other):
    return type(self)(concatenate(self.string, other.string))

You are forcing the class to have a single-arg constructor. why? In addition, you have mu + u'x' and u'x' + mu being of different types, and it also guaranteed to be less efficient.

There is no reason to do additional work only for some unknown unlikely subclass to behave in some specific way. I don't think unicode is designed to be subclassed; If you do subclass it, and want a different behavior than you base class, go ahead and override the relevant methods yourself.

1
  • Why not to expect that subclass has the same signature? I'm sure than you use cls in your code, not harcoded names. By hardcoding names you make a bigger assumption (about your code future using) than by relying on signature. Jul 13, 2013 at 15:34
1

You didn't overrride .strip and __add__, which by default still return unicode objects and not instances of you class. Here is the source for the strip functionality (2.7.3). As for why the developers decided to return a Py_UNICODE object from a unicode method instead of checking for a subclass first and returning that? I think the idea here is you are given just enough to do it yourself.

1
  • 1
    This is exactly what OP said. But why those methods are like that? You didn't explain
    – JBernardo
    Jul 3, 2013 at 14:09
0

Because Python tends to favor simplicity, it is usually not aware of subclasses when dealing with built-in types.

On the main implementation (written in C), it is easier to cast something to Py_UCS4 or PyListObject or other type defined in C than to fetch dynamically a type and call the constructor (that might not be compatible with the super class constructor and crash).

The rule of thumb is to replace all the methods you need if you want to fully replace a built-in class, but I advise against.

Many times, it is simpler to just make it an attribute of a class and add methods as you need:

class MyUnicode(object):
    def __init__(self, u):
        self.u = unicode(u)
1
  • I don't believe that reason is simplicity. There are definitely hidden roots for such behavior. I just can't see it. You are correct that I could wrap unicode object instead of subclassing and hook getattribute etc. I just have read sentence somewhere in the docs that subclassing base classes is good idea... Not so good, as it turned out. Jul 13, 2013 at 15:29
0

After multiple comments from OP, I've decided to rewrite my answer from scratch.

Possible workaround:

class MyUnicodeMetaClass(type):

    autocast_methods = ('__add__', '__radd__', 'format')

    def __init__(cls, name, bases, attrs):   
        super(MyUnicodeMetaClass, cls).__init__(name, bases, attrs)
        for method_name in MyUnicodeMetaClass.autocast_methods:
            try:
                setattr(cls, method_name, cls.autocast_creator(method_name))
            except AttributeError:
                if method_name.startswith('__r'):
                    setattr(cls, method_name, cls.autocast_reverse(method_name))
                else:
                    raise

    def autocast_creator(cls, method_name):
        method = unicode().__getattribute__(method_name)
        def autocast_method(self, *args, **kwargs):
            method = unicode(self).__getattribute__(method_name)
            return cls(method(*args, **kwargs))
        return autocast_method

    def autocast_reverse(cls, method_name):
        method_name = method_name.replace('__r', '__', 1)
        def autocast_method(self, *args, **kwargs):
            method = unicode(args[0]).__getattribute__(method_name)
            return cls(method(self, *args[1:], **kwargs))
        return autocast_method


class MyUnicode(unicode):
    __metaclass__ = MyUnicodeMetaClass

a = MyUnicode(u'aaa {0}')
print a, type(a)
# aaa {0} <class '__main__.MyUnicode'>
b = a + u'bbb'
print b, type(b)
# aaa {0}bbb <class '__main__.MyUnicode'>
c = u'ddd' + a
print c, type(c)
# cccaaa {0} <class '__main__.MyUnicode'>
d = a.format(115)
print d, type(d)
# aaa 115 <class '__main__.MyUnicode'>

It might require extending but the base skeleton is ready.
What is happening here?
1. Metaclass is used to alter creation of MyUnicode class.
2. Simple autocast_creator is used to populate MyUnicode class with methods that should return MyUnicode instead of unicode.
3. A little more sophisticated autocast_reverse is used to provide reverse methods as well (like __radd__ which is needed when first operand is unicode and second one is MyUnicode)
This way, you don't have to manually override all the methods - just list them in autocast_methods tuple.

Background information:

Inheritance:
Object Oriented Programming is intended to reflect real word as well as possible.
Inheritance is no exception here.

In real world a group of elephants is always a group of animals. No doubts here.
But a group of animals might be a group of elephants but it can also be a group of various animals and there can even be no elephant in this group.
This is because any group of elephants is a special kind of group of animals.
So it can be reflected in computer programme by defining ElephantGroup as a subclass of AnimalGroup.
How can ElephantGroup extend AnimalGroup? For example by defining new field ivory_weight and new method toot().
Consider such a simple operation ElephantGroup() + AnimalGroup().
What is the class of expected result? AnimalGroup - no magic here and rats, dolphins etc. won't become elephants. Rats and dolphins don't provide ivory and they can't toot, so forcing them to do so is not an expected behavior.

Let's get back to MyUnicode and unicode.
Machines aren't intelligent in the meaning presented in Matrix or Terminator.
The Python interpreter doesn't understand what is the purpose of MyUnicode.
Consider a class extending unicode or str (doesn't really matter here) that is named EmailAddress and is intended to hold e-mail address. No surprise :)
And we have a code snippet now:

a = EmailAddress(u'[email protected]')
b = u'/!\n'
c = a + b
d = b + a

Still expecting c and d to be instances of EmailAddress? (Or MyUnicode?)
If you have answered yes then please tell me:
1. What if EmailAddress.__init__(...) contains well crafted logic checking if an argument is likely to be a valid e-mail address? And it raises exception if it isn't...
2. How can interpreter be aware that it can safely initialize MyUnicode instance with any unicode instance without executing __init__? Please also remember this is Python and __init__ can even be changed dynamically at runtime.
Remember - we can always cast instance to any of it's ancestors. The reverse operation can't be done implicitly. It would be a mess if the interpreter would implicitly cast object instances to objects subclasses.
The strip() method follows the same rules - unicode characters are removed from the original and a reference to new unicode instance is returned (unless exact same unicode exists - in this case the reference to existing one is returned).

Referring to instance class:

In one of your comments you said cls refers to the class of instance that started the execution chain...
cls is a naming convention used in metaclasses, classmethods and in __new__() method to indicate that we don't have an instance - we only have a class.
In fact we can't access instance in any of these cases - __new__() however should return new instance in most cases.

I guess you were thinking of identifier.__class__ attribute. It has nothing to do with execution chain either. It points to the actual class of instance referred by identifier.
Why do you expect methods of unicode and str to use it to create subclasses?
Casting something implicitly to it's subclass is not an expected behavior - I know, one of the operands in your code is MyUnicode but the other one is unicode - even in strip(), the default argument is a unicode containg whitespace characters.

Some unicode implementation details:

Python unicode and string types are immutable and unique (explanation is coming). Immutable means that any modyfying operation on them returns other instance of unicode or string respectively.
Other instance would mean new instance but as I said these types are unique.
What it means? See the code:

a = u'aaa'
b = u'aaa'

What happened here?
There was a new instance of unicode created to initialize a.
The unicode object to initialize b was found so no new instance was created.
Instead a reference counter to unicode object holding u'aaa' was incremented.

Now when we know it, consider this code:

a = u'aaa'
b = MyUnicode(u'aa')
c = b + u'a'

What exactly is stored in c variable? A reference to unicode object - the same object which is referenced by a.
Why changing c won't affect a? Because unicode is immutable and underlying object remains unchanged.
If the next line would be c = c + u'b', then c would get reference to new/other instance and object referenced by a would have it's reference counter decreased.

Conclusions:

The Python unicode and str classes are consistent and predictive.
There are some types that are hard to derive from, due to optimization, special purposes or implementation details.
unicode and str are rather not intended to be subclassed altough it can be achieved for instance with metaclass as in my snippet.

As always I'm looking for any constructive criticism and comments.

Good luck!

6
  • I definitely can override all required methods (so-o-o many) but when developers decide to add one more I will be forced to change my old code... Not a friendly behavior at all. I expected from Python that it will use derived class whenever possible like we do in own code. Keeping initializator's call signature(especially of a base-type class!) out of modifications is usual practice to rely on. Jul 13, 2013 at 15:24
  • @IvanKleshnin, where do you see initializator's call signature modification in my code? I said that you are free to modify MyUnicode __init__ to accept more arguments than unicode. I didn't said you have to. I think you misunderstand inheritance. Abstract example: AppleJuice inherits from FruitJuice. Would you really like the interpreter to assume that AppleJuice + FruitJuice means more AppleJuice? NO! Interpreter can't assume that base class can always be safely casted to descendant class. Jul 15, 2013 at 8:21
  • I think you are wrong. AppleJuice & FruitJuice always mean AppleJuice. Jul 16, 2013 at 16:44
  • Thats why we use cls variable instead of class name. Jul 16, 2013 at 16:52
  • You are surprising me... FruitJuice can be OrangeJuice for example. Still thinking that AppleJuice + FruitJuice is always AppleJuice? Interpreter can't figure out what kind of juice FruitJuice is unless explicitly said. All new style classes inherit from object - this doesn't mean any object can be casted to any of it's descendants. Jul 16, 2013 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.