455

Is there a way to have a defaultdict(defaultdict(int)) in order to make the following code work?

for x in stuff:
    d[x.a][x.b] += x.c_int

d needs to be built ad-hoc, depending on x.a and x.b elements.

I could use:

for x in stuff:
    d[x.a,x.b] += x.c_int

but then I wouldn't be able to use:

d.keys()
d[x.a].keys()
0

6 Answers 6

807

Yes like this:

defaultdict(lambda: defaultdict(int))

The argument of a defaultdict (in this case is lambda: defaultdict(int)) will be called when you try to access a key that doesn't exist. The return value of it will be set as the new value of this key, which means in our case the value of d[Key_doesnt_exist] will be defaultdict(int).

If you try to access a key from this last defaultdict i.e. d[Key_doesnt_exist][Key_doesnt_exist] it will return 0, which is the return value of the argument of the last defaultdict i.e. int().

11
  • 8
    it works great! could you explain the rational behind this syntax? Oct 12, 2011 at 8:25
  • 46
    @Jonathan: Yes sure, the argument of a defaultdict (in this case is lambda : defaultdict(int)) will be called when you try to access a key that don't exist and the return value of it will be set as the new value of this key which mean in our case the value of d[Key_dont_exist] will be defaultdict(int), and if you try to access a key from this last defaultdict i.e. d[Key_dont_exist][Key_dont_exist] it will return 0 which is the return value of the argument of the last defaultdict i.e. int(), Hope this was helpful.
    – mouad
    Oct 12, 2011 at 14:25
  • 34
    The argument to defaultdict should be a function. defaultdict(int) is a dictionary, while lambda: defaultdict(int) is function that returns a dictionary.
    – has2k1
    Sep 22, 2012 at 4:45
  • 36
    @has2k1 That is incorrect. The argument to defaultdict needs to be a callable. A lambda is a callable.
    – Niels Bom
    Jan 22, 2013 at 15:11
  • 4
    @RickyLevi, if you want to have that working you can just say: defaultdict(lambda: defaultdict(lambda: defaultdict(int)))
    – darophi
    Mar 26, 2019 at 15:21
58

The parameter to the defaultdict constructor is the function which will be called for building new elements. So let's use a lambda !

>>> from collections import defaultdict
>>> d = defaultdict(lambda : defaultdict(int))
>>> print d[0]
defaultdict(<type 'int'>, {})
>>> print d[0]["x"]
0

Since Python 2.7, there's an even better solution using Counter:

>>> from collections import Counter
>>> c = Counter()
>>> c["goodbye"]+=1
>>> c["and thank you"]=42
>>> c["for the fish"]-=5
>>> c
Counter({'and thank you': 42, 'goodbye': 1, 'for the fish': -5})

Some bonus features

>>> c.most_common()[:2]
[('and thank you', 42), ('goodbye', 1)]

For more information see PyMOTW - Collections - Container data types and Python Documentation - collections

3
  • 8
    Just to complete the circle here, you would want to use d = defaultdict(lambda : Counter()) rather than d = defaultdict(lambda : defaultdict(int)) to specifically address the problem as originally posed.
    – gumption
    Jun 20, 2014 at 18:50
  • 4
    @gumption you can just use d = defaultdict(Counter()) no need for a lambda in this case
    – Deb
    Aug 4, 2017 at 8:53
  • 9
    @Deb you have a slight error- remove the inner parentheses so you pass a callable instead of a Counter object. That is: d = defaultdict(Counter) Aug 11, 2018 at 7:00
41

Previous answers have addressed how to make a two-levels or n-levels defaultdict. In some cases you want an infinite one:

def ddict():
    return defaultdict(ddict)

Usage:

>>> d = ddict()
>>> d[1]['a'][True] = 0.5
>>> d[1]['b'] = 3
>>> import pprint; pprint.pprint(d)
defaultdict(<function ddict at 0x7fcac68bf048>,
            {1: defaultdict(<function ddict at 0x7fcac68bf048>,
                            {'a': defaultdict(<function ddict at 0x7fcac68bf048>,
                                              {True: 0.5}),
                             'b': 3})})
3
  • 10
    I love this. It's devilishly simple, but incredibly useful. Thanks!
    – rosstex
    May 9, 2020 at 2:22
  • How can this be used for the following use case: the nth level needs to be an int or a list Sep 24, 2020 at 18:55
  • 1
    @MonsieurBeilto "previous answers have addressed how to make a two-levels or n-levels defaultdict" → stackoverflow.com/a/54266841/695591
    – Clément
    Sep 28, 2020 at 16:41
34

I find it slightly more elegant to use partial:

import functools
dd_int = functools.partial(defaultdict, int)
defaultdict(dd_int)

Of course, this is the same as a lambda.

3
  • 3
    Partial is also better than lambda here because it can be applied recursively :) see my answer below for a generic nested defaultdict factory method.
    – Campi
    Jan 19, 2019 at 12:01
  • @Campi you don't need partial for recursive applications, AFAICT
    – Clément
    Nov 25, 2019 at 20:00
  • 1
    This is also better because it can be pickled (lambdas don't play nice with pickling + multiprocessing)
    – Alex Sax
    May 18, 2022 at 2:36
17

For reference, it's possible to implement a generic nested defaultdict factory method through:

from collections import defaultdict
from functools import partial
from itertools import repeat


def nested_defaultdict(default_factory, depth=1):
    result = partial(defaultdict, default_factory)
    for _ in repeat(None, depth - 1):
        result = partial(defaultdict, result)
    return result()

The depth defines the number of nested dictionary before the type defined in default_factory is used. For example:

my_dict = nested_defaultdict(list, 3)
my_dict['a']['b']['c'].append('e')
4
  • Can you give a usage example? Not working the way I expected this to. ndd = nested_defaultdict(dict) .... ndd['a']['b']['c']['d'] = 'e' throws KeyError: 'b'
    – David Marx
    Feb 14, 2019 at 20:14
  • Hey David, you need to define the depth of your dictionary, in your example 3 (as you defined the default_factory to be a dictionary too. nested_defaultdict(dict, 3) will work for you.
    – Campi
    Feb 16, 2019 at 10:01
  • This was super helpful, thanks! One thing I noticed is that this creates a default_dict at depth=0, which may not always be desired if the depth is unknown at the time of calling. Easily fixable by adding a line if not depth: return default_factory(), at the top of the function, though there's probably a more elegant solution.
    – Brendan
    Oct 30, 2019 at 23:44
  • Why not for _ in range(depth - 1):?
    – Clément
    Aug 4, 2022 at 23:49
7

Others have answered correctly your question of how to get the following to work:

for x in stuff:
    d[x.a][x.b] += x.c_int

An alternative would be to use tuples for keys:

d = defaultdict(int)
for x in stuff:
    d[x.a,x.b] += x.c_int
    # ^^^^^^^ tuple key

The nice thing about this approach is that it is simple and can be easily expanded. If you need a mapping three levels deep, just use a three item tuple for the key.

2
  • 4
    This solution means it isn't simple to get all of d[x.a], as you need to introspect every key to see if it has x.a as the first element of the tuple. Feb 18, 2011 at 3:02
  • 5
    If you wanted nesting 3 levels deep, then just define it as 3 levels: d = defaultdict(lambda: defaultdict( lambda: defaultdict(int))) Feb 18, 2011 at 3:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.