1415
votes

What are the lesser-known but useful features of the Python programming language?

  • Try to limit answers to Python core.
  • One feature per answer.
  • Give an example and short description of the feature, not just a link to documentation.
  • Label the feature using a title as the first line.

Quick links to answers:

0

191 Answers 191

1 2 3
4
5
7
11
votes

A slight misfeature of python. The normal fast way to join a list of strings together is,

''.join(list_of_strings)
7
  • 20
    there are very good reasons that this is a method of string instead of a method of list. this allows the same function to join any iterable, instead of duplicating join for every iterable type. Jan 2, 2009 at 18:37
  • Yes I know why it does - but would anyone discover this if they hadn't been told? Jan 2, 2009 at 21:28
  • Discover? It's pretty hard to remember too, and I've used python since before there were methods om strings.
    – kaleissin
    Jan 20, 2009 at 13:48
  • 10
    If this is too ugly for you to cope with, you can write the very same thing as str.join('',list_of_strings) but other pythonistas may scorn you for trying to write java. Nov 17, 2009 at 19:04
  • @TokenMacGuy: the reason why ''.join([...]) is preferred is because many people often mixes up the order of the arguments in string.join(..., ...); by putting ''.join() things become clearer
    – Lie Ryan
    Oct 30, 2010 at 22:32
11
votes

Creating enums

In Python, you can do this to quickly create an enumeration:

>>> FOO, BAR, BAZ = range(3)
>>> FOO
0

But the "enums" don't have to have integer values. You can even do this:

class Colors(object):
    RED, GREEN, BLUE, YELLOW = (255,0,0), (0,255,0), (0,0,255), (0,255,255)

#now Colors.RED is a 3-tuple that returns the 24-bit 8bpp RGB 
#value for saturated red
11
votes

The Object Data Model

You can override any operator in the language for your own classes. See this page for a complete list. Some examples:

  • You can override any operator (* + - / // % ^ == < > <= >= . etc.). All this is done by overriding __mul__, __add__, etc. in your objects. You can even override things like __rmul__ to handle separately your_object*something_else and something_else*your_object. . is attribute access (a.b), and can be overridden to handle any arbitrary b by using __getattr__. Also included here is a(…) using __call__.

  • You can create your own slice syntax (a[stuff]), which can be very complicated and quite different from the standard syntax used in lists (numpy has a good example of the power of this in their arrays) using any combination of ,, :, and that you like, using Slice objects.

  • Handle specially what happens with many keywords in the language. Included are del, in, import, and not.

  • Handle what happens when many built in functions are called with your object. The standard __int__, __str__, etc. go here, but so do __len__, __reversed__, __abs__, and the three argument __pow__ (for modular exponentiation).

1
  • For in you have to override __contains__.
    – asmeurer
    Dec 28, 2010 at 21:20
10
votes

"Unpacking" to function parameters

def foo(a, b, c):
        print a, b, c

bar = (3, 14, 15)
foo(*bar)

When executed prints:

3 14 15
1
  • 1
    This is the canonical alternative to the old "apply()" built-in.
    – Jim Dennis
    Jun 25, 2010 at 23:11
10
votes

The reversed() builtin. It makes iterating much cleaner in many cases.

quick example:

for i in reversed([1, 2, 3]):
    print(i)

produces:

3
2
1

However, reversed() also works with arbitrary iterators, such as lines in a file, or generator expressions.

10
votes

The Zen of Python

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
4
  • Hidden? OTOH, This is one of the selling points of Python. May 27, 2010 at 19:19
  • I like the syntax coloring, esp. for Dutch.
    – asmeurer
    Dec 31, 2010 at 4:42
  • Duplicate of a previous answer
    – Bite code
    Dec 24, 2011 at 15:23
  • Duplicate of a previous answer
    – warvariuc
    Mar 4, 2012 at 19:39
10
votes

Changing function label at run time:

>>> class foo:
...   def normal_call(self): print "normal_call"
...   def call(self): 
...     print "first_call"
...     self.call = self.normal_call

>>> y = foo()
>>> y.call()
first_call
>>> y.call()
normal_call
>>> y.call()
normal_call
...
10
votes

string-escape and unicode-escape encodings

Lets say you have a string from outer source, that contains \n, \t and so on. How to transform them into new-line or tab? Just decode string using string-escape encoding!

>>> print s
Hello\nStack\toverflow
>>> print s.decode('string-escape')
Hello
Stack   overflow

Another problem. You have normal string with unicode literals like \u01245. How to make it work? Just decode string using unicode-escape encoding!

>>> s = '\u041f\u0440\u0438\u0432\u0456\u0442, \u0441\u0432\u0456\u0442!'
>>> print s
\u041f\u0440\u0438\u0432\u0456\u0442, \u0441\u0432\u0456\u0442!
>>> print unicode(s)
\u041f\u0440\u0438\u0432\u0456\u0442, \u0441\u0432\u0456\u0442!
>>> print unicode(s, 'unicode-escape')
Привіт, світ!
9
votes

unzip un-needed in Python

Someone blogged about Python not having an unzip function to go with zip(). unzip is straight-forward to calculate because:

>>> t1 = (0,1,2,3)
>>> t2 = (7,6,5,4)
>>> [t1,t2] == zip(*zip(t1,t2))
True

On reflection though, I'd rather have an explicit unzip().

3
  • 8
    def unzip(x): return zip(*x) Done!
    – bukzor
    Jun 27, 2010 at 17:07
  • The solution is slightly subtle (I can understand the point of view of anyone who asks for it), but I can also see why it would be redundant Sep 22, 2010 at 4:47
  • +1. I was going to add this, but it seems I was beat to it.
    – asmeurer
    Dec 28, 2010 at 5:57
9
votes

Creating dictionary of two sequences that have related data

In [15]: t1 = (1, 2, 3)

In [16]: t2 = (4, 5, 6)

In [17]: dict (zip(t1,t2))
Out[17]: {1: 4, 2: 5, 3: 6}
9
votes

Top Secret Attributes

>>> class A(object): pass
>>> a = A()
>>> setattr(a, "can't touch this", 123)
>>> dir(a)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', "can't touch this"]
>>> a.can't touch this # duh
  File "<stdin>", line 1
    a.can't touch this
                     ^
SyntaxError: EOL while scanning string literal
>>> getattr(a, "can't touch this")
123
>>> setattr(a, "__class__.__name__", ":O")
>>> a.__class__.__name__
'A'
>>> getattr(a, "__class__.__name__")
':O'
1
  • 6
    AHHHH! Bad, bad, bad!
    – asmeurer
    Dec 28, 2010 at 6:03
9
votes

namedtuple is a tuple

>>> node = namedtuple('node', "a b")
>>> node(1,2) + node(5,6)
(1, 2, 5, 6)
>>> (node(1,2), node(5,6))
(node(a=1, b=2), node(a=5, b=6))
>>> 

Some more experiments to respond to comments:

>>> from collections import namedtuple
>>> from operator import *
>>> mytuple = namedtuple('A', "a b")
>>> yourtuple = namedtuple('Z', "x y")
>>> mytuple(1,2) + yourtuple(5,6)
(1, 2, 5, 6)
>>> q = [mytuple(1,2), yourtuple(5,6)]
>>> q
[A(a=1, b=2), Z(x=5, y=6)]
>>> reduce(operator.__add__, q)
(1, 2, 5, 6)

So, namedtuple is an interesting subtype of tuple.

9
  • At this point, you've lost all context. If you don't need the context, or the data isn't structured in a particular way, why a tuple at all? Surely you're just using it as a list? Jan 9, 2011 at 1:01
  • @Samir Talwar The question/answer is about hidden features. Did you know about this one? I'm not defending one design or the other, but just pointing out what is there. When I first tried to use named tuples, I thought they woulnd't match as tuples do, but... Let me expand the example to show you.
    – Apalala
    Jan 9, 2011 at 1:25
  • @Apalala: I had assumed it, but never checked. You're right: it is an interesting and hidden feature. I guess useful is a different thing. Jan 9, 2011 at 16:34
  • 5
    Also fun is that you can feed the result of a namedtuple call directly into a class definition, as in class rectangle(namedtuple("rectangle", "width height")): in order to add custom methods
    – Ben Blank
    Jan 10, 2011 at 19:29
  • @Samir Talwar I use namedtuples as the representation for parse trees, and their behavior was useful in merging siblings so they looked more like lists. Imagine the typical grammar productions for a list...
    – Apalala
    Jan 11, 2011 at 1:10
9
votes

Dynamically added attributes

This might be useful if you think about adding some attributes to your classes just by calling them. This can be done by overriding the __getattribute__ member function which is called when the dot operand is used. So, let's see a dummy class for example:

class Dummy(object):
    def __getattribute__(self, name):
        f = lambda: 'Hello with %s'%name
        return f

When you instantiate a Dummy object and do a method call you’ll get the following:

>>> d = Dummy()
>>> d.b()
'Hello with b'

Finally, you can even set the attribute to your class so it can be dynamically defined. This could be useful if you work with Python web frameworks and want to do queries by parsing the attribute's name.

I have a gist at github with this simple code and its equivalent on Ruby made by a friend.

Take care!

0
9
votes

Flattening a list with sum().

The sum() built-in function can be used to __add__ lists together, providing a handy way to flatten a list of lists:

Python 2.7.1 (r271:86832, May 27 2011, 21:41:45) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> l = [[1, 2, 3], [4, 5], [6], [7, 8, 9]]
>>> sum(l, [])
[1, 2, 3, 4, 5, 6, 7, 8, 9]
0
9
votes

The Borg Pattern

This is a killer from Alex Martelli. All instances of Borg share state. This removes the need to employ the singleton pattern (instances don't matter when state is shared) and is rather elegant (but is more complicated with new classes).

The value of foo can be reassigned in any instance and all will be updated, you can even reassign the entire dict. Borg is the perfect name, read more here.

class Borg:
    __shared_state = {'foo': 'bar'}
    def __init__(self):
        self.__dict__ = self.__shared_state
    # rest of your class here

This is perfect for sharing an eventlet.GreenPool to control concurrency.

8
votes

pdb — The Python Debugger

As a programmer, one of the first things that you need for serious program development is a debugger. Python has one built-in which is available as a module called pdb (for "Python DeBugger", naturally!).

http://docs.python.org/library/pdb.html

8
votes

threading.enumerate() gives access to all Thread objects in the system and sys._current_frames() returns the current stack frames of all threads in the system, so combine these two and you get Java style stack dumps:

def dumpstacks(signal, frame):
    id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
    code = []
    for threadId, stack in sys._current_frames().items():
        code.append("\n# Thread: %s(%d)" % (id2name[threadId], threadId))
        for filename, lineno, name, line in traceback.extract_stack(stack):
            code.append('File: "%s", line %d, in %s' % (filename, lineno, name))
            if line:
                code.append("  %s" % (line.strip()))
    print "\n".join(code)

import signal
signal.signal(signal.SIGQUIT, dumpstacks)

Do this at the beginning of a multi-threaded python program and you get access to current state of threads at any time by sending a SIGQUIT. You may also choose signal.SIGUSR1 or signal.SIGUSR2.

See

7
votes

inspect module is also a cool feature.

7
votes

Reloading modules enables a "live-coding" style. But class instances don't update. Here's why, and how to get around it. Remember, everything, yes, everything is an object.

>>> from a_package import a_module
>>> cls = a_module.SomeClass
>>> obj = cls()
>>> obj.method()
(old method output)

Now you change the method in a_module.py and want to update your object.

>>> reload(a_module)
>>> a_module.SomeClass is cls
False # Because it just got freshly created by reload.
>>> obj.method()
(old method output)

Here's one way to update it (but consider it running with scissors):

>>> obj.__class__ is cls
True # it's the old class object
>>> obj.__class__ = a_module.SomeClass # pick up the new class
>>> obj.method()
(new method output)

This is "running with scissors" because the object's internal state may be different than what the new class expects. This works for really simple cases, but beyond that, pickle is your friend. It's still helpful to understand why this works, though.

1
  • 1
    +1 for suggesting pickle (or cPickle). It was really helpful for me, some weeks ago. Aug 20, 2010 at 20:51
7
votes

Backslashes inside raw strings can still escape quotes. See this:

>>> print repr(r"aaa\"bbb")
'aaa\\"bbb'

Note that both the backslash and the double-quote are present in the final string.

As consequence, you can't end a raw string with a backslash:

>>> print repr(r"C:\")
SyntaxError: EOL while scanning string literal
>>> print repr(r"C:\"")
'C:\\"'

This happens because raw strings were implemented to help writing regular expressions, and not to write Windows paths. Read a long discussion about this at Gotcha — backslashes in Windows filenames.

3
  • 2
    Note that the backslash is still part of the string afterwards... So one might not regard this as regular escaping.
    – huin
    Aug 21, 2010 at 7:18
  • You're probably better off just using single quotes ' for the outer string.
    – asmeurer
    Dec 28, 2010 at 5:59
  • Or just use (forward) slashes, as the Windows API will translate them automatically, then you can finally forget about DOS-style paths. (Though you must use backslashes for "\\server\share\path\file" style resources) Dec 4, 2011 at 8:45
7
votes

Operators can be called as functions:

from operator import add
print reduce(add, [1,2,3,4,5,6])
3
  • ? what did you think operators are?
    – Ant
    Jan 23, 2011 at 18:00
  • sorry, i dont get your point..what do you think that we think operators are?
    – Ant
    Jan 23, 2011 at 19:18
  • @Ant, if you were already aware of operators being functions, you can disregard this tip. Not all languages implement operators as functions, so a person coming from another language might not have known this. Jan 25, 2011 at 20:59
7
votes

infinite recursion in list

>>> a = [1,2]
>>> a.append(a)
>>> a
[1, 2, [...]]
>>> a[2]
[1, 2, [...]]
>>> a[2][2][2][2][2][2][2][2][2] == a
True
1
  • i don't think it's a Python feature. nor it's hidden. where this can be used?
    – warvariuc
    Jun 30, 2011 at 8:31
7
votes

...that dict.get() has a default value of None, thereby avoiding KeyErrors:

In [1]: test = { 1 : 'a' }

In [2]: test[2]
---------------------------------------------------------------------------
<type 'exceptions.KeyError'>              Traceback (most recent call last)

&lt;ipython console&gt; in <module>()

<type 'exceptions.KeyError'>: 2

In [3]: test.get( 2 )

In [4]: test.get( 1 )
Out[4]: 'a'

In [5]: test.get( 2 ) == None
Out[5]: True

and even to specify this 'at the scene':

In [6]: test.get( 2, 'Some' ) == 'Some'
Out[6]: True

And you can use setdefault() to have a value set and returned if it doesn't exist:

>>> a = {}
>>> b = a.setdefault('foo', 'bar')
>>> a
{'foo': 'bar'}
>>> b
'bar
0
6
votes

Builtin methods or functions don't implement the descriptor protocol which makes it impossible to do stuff like this:

>>> class C(object):
...  id = id
... 
>>> C().id()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: id() takes exactly one argument (0 given)

However you can create a small bind descriptor that makes this possible:

>>> from types import MethodType
>>> class bind(object):
...  def __init__(self, callable):
...   self.callable = callable
...  def __get__(self, obj, type=None):
...   if obj is None:
...    return self
...   return MethodType(self.callable, obj, type)
... 
>>> class C(object):
...  id = bind(id)
... 
>>> C().id()
7414064
2
  • 1
    It's simpler and easier to do this as a property, in this case: class C(object): id = property(id)
    – Pi Delport
    Oct 2, 2010 at 22:56
  • lambda is also a good alternative: class C(object): id = lambda s, *a, **kw: id(*a, **kw); and a better version of bind: def bind(callable): return lambda s, *a, **kw: callable(*a, **kw)
    – Lie Ryan
    Oct 30, 2010 at 22:40
6
votes

Nested Function Parameter Re-binding

def create_printers(n):
    for i in xrange(n):
        def printer(i=i): # Doesn't work without the i=i
            print i
        yield printer
3
  • it works without it, but differently. :-) Dec 13, 2009 at 15:35
  • No, it doesn't work without it. Omit the i=i and see the difference between map(apply, create_printers(10)) and map(apply, list(apply_printers(10))), where converting to a list consumes the generator and now all ten printer functions have i bound to the same value: 9, where calling them one at a time calls them before the next iteration of the generator changes the int i is bound to in the outer scope.
    – ironfroggy
    Dec 15, 2009 at 2:16
  • 1
    I think what @kaizer.se is saying is that when you omit the i=i the i in the printer function references the i from the for loop rather than the local i that is created when a new printer function is created with the i=i keyword arg. So it still does work (it yields functions, each with access to a closure) but it doesn't work in the way you'd expect without explicitly creating a local variable. Mar 9, 2011 at 22:57
6
votes

You can override the mro of a class with a metaclass

>>> class A(object):
...     def a_method(self):
...         print("A")
... 
>>> class B(object):
...     def b_method(self):
...         print("B")
... 
>>> class MROMagicMeta(type):
...     def mro(cls):
...         return (cls, B, object)
... 
>>> class C(A, metaclass=MROMagicMeta):
...     def c_method(self):
...         print("C")
... 
>>> cls = C()
>>> cls.c_method()
C
>>> cls.a_method()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
AttributeError: 'C' object has no attribute 'a_method'
>>> cls.b_method()
B
>>> type(cls).__bases__
(<class '__main__.A'>,)
>>> type(cls).__mro__
(<class '__main__.C'>, <class '__main__.B'>, <class 'object'>)

It's probably hidden for a good reason. :)

2
  • 2
    That's playing with fire, and asking for ethernal damnation. Better have good reason ;)
    – gorsky
    Dec 18, 2009 at 10:20
  • Does not work with python 2.x. Use __metaclass__ = MROMagicMeta instead. Jul 15, 2011 at 10:35
6
votes

Objects of small intgers (-5 .. 256) never created twice:


>>> a1 = -5; b1 = 256
>>> a2 = -5; b2 = 256
>>> id(a1) == id(a2), id(b1) == id(b2)
(True, True)
>>>
>>> c1 = -6; d1 = 257
>>> c2 = -6; d2 = 257
>>> id(c1) == id(c2), id(d1) == id(d2)
(False, False)
>>>

Edit: List objects never destroyed (only objects in lists). Python has array in which it keeps up to 80 empty lists. When you destroy list object - python puts it to that array and when you create new list - python gets last puted list from this array:


>>> a = [1,2,3]; a_id = id(a)
>>> b = [1,2,3]; b_id = id(b)
>>> del a; del b
>>> c = [1,2,3]; id(c) == b_id
True
>>> d = [1,2,3]; id(d) == a_id
True
>>>

2
  • 5
    This feature is implementation dependent, so you shouldn't rely on it. Oct 27, 2009 at 15:36
  • As Denis said, do not rely on this behavior. It doesn't work, for example, in PyPy, and your code will break miserably in that if you try to use it.
    – asmeurer
    Dec 28, 2010 at 5:53
6
votes

You can decorate functions with classes - replacing the function with a class instance:

class countCalls(object):
    """ decorator replaces a function with a "countCalls" instance
    which behaves like the original function, but keeps track of calls

    >>> @countCalls
    ... def doNothing():
    ...     pass
    >>> doNothing()
    >>> doNothing()
    >>> print doNothing.timesCalled
    2
    """
    def __init__ (self, functionToTrack):
        self.functionToTrack = functionToTrack
        self.timesCalled = 0
    def __call__ (self, *args, **kwargs):
        self.timesCalled += 1
        return self.functionToTrack(*args, **kwargs)
6
votes

Manipulating Recursion Limit

Getting or setting the maximum depth of recursion with sys.getrecursionlimit() & sys.setrecursionlimit().

We can limit it to prevent a stack overflow caused by infinite recursion.

1 2 3
4
5
7

Not the answer you're looking for? Browse other questions tagged or ask your own question.