398

I'm trying to convert a longish hollow "data" class into a named tuple. My class currently looks like this:

class Node(object):
    def __init__(self, val, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

After conversion to namedtuple it looks like:

from collections import namedtuple
Node = namedtuple('Node', 'val left right')

But there is a problem here. My original class allowed me to pass in just a value and took care of the default by using default values for the named/keyword arguments. Something like:

class BinaryTree(object):
    def __init__(self, val):
        self.root = Node(val)

But this doesn't work in the case of my refactored named tuple since it expects me to pass all the fields. I can of course replace the occurrences of Node(val) to Node(val, None, None) but it isn't to my liking.

So does there exist a good trick which can make my re-write successful without adding a lot of code complexity (metaprogramming) or should I just swallow the pill and go ahead with the "search and replace"? :)

6
  • 3
    Why do you want to make this conversion? I like your original Node class just the way it is. Why convert to named tuple?
    – steveha
    Jul 5, 2012 at 19:29
  • 44
    I wanted to make this conversion because the current Node and other classes are simple data-holder value objects with a bunch of different fields (Node is just one of them). These class declarations are nothing much more than line noise IMHO hence wanted to trim them out. Why maintain something which isn't required? :)
    – sasuke
    Jul 5, 2012 at 19:33
  • You don't have any method functions on your classes at all? You don't, for example, have a .debug_print() method that walks the tree and prints it?
    – steveha
    Jul 5, 2012 at 20:09
  • 4
    Sure I do, but that's for the BinaryTree class. Node and other data holders don't require such special methods esp given that named tuples have a decent __str__ and __repr__ representation. :)
    – sasuke
    Jul 5, 2012 at 21:11
  • Okay, seems reasonable. And I think Ignacio Vazquez-Abrams has given you the answer: use a function that does the default values for your node.
    – steveha
    Jul 6, 2012 at 0:27

23 Answers 23

717

Python 3.7

Use the defaults parameter.

>>> from collections import namedtuple
>>> fields = ('val', 'left', 'right')
>>> Node = namedtuple('Node', fields, defaults=(None,) * len(fields))
>>> Node()
Node(val=None, left=None, right=None)

Or better yet, use the new dataclasses library, which is much nicer than namedtuple.

>>> from dataclasses import dataclass
>>> from typing import Any
>>> @dataclass
... class Node:
...     val: Any = None
...     left: 'Node' = None
...     right: 'Node' = None
>>> Node()
Node(val=None, left=None, right=None)

Before Python 3.7

Set Node.__new__.__defaults__ to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.__defaults__ = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Before Python 2.6

Set Node.__new__.func_defaults to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.func_defaults = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Order

In all versions of Python, if you set fewer default values than exist in the namedtuple, the defaults are applied to the rightmost parameters. This allows you to keep some arguments as required arguments.

>>> Node.__new__.__defaults__ = (1,2)
>>> Node()
Traceback (most recent call last):
  ...
TypeError: __new__() missing 1 required positional argument: 'val'
>>> Node(3)
Node(val=3, left=1, right=2)

Wrapper for Python 2.6 to 3.6

Here's a wrapper for you, which even lets you (optionally) set the default values to something other than None. This does not support required arguments.

import collections
def namedtuple_with_defaults(typename, field_names, default_values=()):
    T = collections.namedtuple(typename, field_names)
    T.__new__.__defaults__ = (None,) * len(T._fields)
    if isinstance(default_values, collections.Mapping):
        prototype = T(**default_values)
    else:
        prototype = T(*default_values)
    T.__new__.__defaults__ = tuple(prototype)
    return T

Example:

>>> Node = namedtuple_with_defaults('Node', 'val left right')
>>> Node()
Node(val=None, left=None, right=None)
>>> Node = namedtuple_with_defaults('Node', 'val left right', [1, 2, 3])
>>> Node()
Node(val=1, left=2, right=3)
>>> Node = namedtuple_with_defaults('Node', 'val left right', {'right':7})
>>> Node()
Node(val=None, left=None, right=7)
>>> Node(4)
Node(val=4, left=None, right=7)
19
  • 28
    Let's see...your one-liner: a) is the shortest/simplest answer, b) preserves space efficiency, c) doesn't break isinstance ...all pros, no cons...too bad you were a little late to the party. This is the best answer.
    – Gerrat
    May 2, 2014 at 13:39
  • 1
    One problem with the wrapper version: unlike the builtin collections.namedtuple, this version is not pickleable/multiprocess serializable if the def() is included in a different module. Jun 22, 2014 at 11:03
  • 3
    I have given this answer an upvote as it is preferable to my own. Its a pity however that my own answer keeps getting upvoted :|
    – Justin Fay
    Mar 16, 2015 at 21:51
  • 4
    @ishaaq, the problem is that (None) is not a tuple, it's None. If you use (None,) instead, it should work fine. Aug 7, 2015 at 18:34
  • 4
    Excellent! You can generalize the setting-of-defaults with: Node.__new__.__defaults__= (None,) * len(Node._fields)
    – ankostis
    Aug 31, 2015 at 9:37
156
+50

I subclassed namedtuple and overrode the __new__ method:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

This preserves an intuitive type hierarchy, which the creation of a factory function disguised as a class does not.

7
  • 9
    This might need slots and fields properties in order to maintain the space efficiency of a named tuple.
    – Pepijn
    Apr 7, 2014 at 9:16
  • For some reason, __new__ is not being called when _replace is used.
    – user3892448
    Sep 9, 2014 at 9:49
  • 1
    Please take a look at @marc-lodato answer below which IMHO is a better solution than this.
    – Justin Fay
    Feb 26, 2015 at 12:53
  • 2
    but @marc-lodato's answer doesn't provide the ability for a subclass to have different defaults
    – Jason S
    May 4, 2017 at 23:23
  • 1
    @JasonS, i suspect that for a subclass to have different defaults could violate the LSP. However, a subclass may very well want to have more defaults. In any case, it would be for the subclass to use justinfay's method, and the base class would be fine with Marc's method.
    – Alexey
    Feb 25, 2018 at 9:01
112

With typing.NamedTuple in Python 3.6.1+ you can provide both a default value and a type annotation to a NamedTuple field. Use typing.Any if you only need the former:

from typing import Any, NamedTuple


class Node(NamedTuple):
    val: Any
    left: 'Node' = None
    right: 'Node' = None

Usage:

>>> Node(1)
Node(val=1, left=None, right=None)
>>> n = Node(1)
>>> Node(2, left=n)
Node(val=2, left=Node(val=1, left=None, right=None), right=None)

Also, in case you need both default values and optional mutability, Python 3.7 is going to have data classes (PEP 557) that can in some (many?) cases replace namedtuples.


Sidenote: one quirk of the current specification of annotations (expressions after : for parameters and variables and after -> for functions) in Python is that they are evaluated at definition time*. So, since "class names become defined once the entire body of the class has been executed", the annotations for 'Node' in the class fields above must be strings to avoid NameError.

This kind of type hints is called "forward reference" ([1], [2]), and with PEP 563 Python 3.7+ is going to have a __future__ import (to be enabled by default in 4.0) that will allow to use forward references without quotes, postponing their evaluation.

* AFAICT only local variable annotations are not evaluated at runtime. (source: PEP 526)

4
  • 4
    This seems like the cleanest solution for 3.6.1+ users. Note that this example is (slightly) confusing as the type hint for the fields left and right (i.e. Node) is the same type as the class being defined and therefore must be written as strings.
    – 101
    Feb 8, 2018 at 23:54
  • 1
    @101, thank you, I've added a note about this to the answer.
    – monk-time
    Feb 10, 2018 at 13:53
  • 2
    What's the analog for the idiom my_list: List[T] = None self.my_list = my_list if my_list is not None else []? Can we not use default parameters like this?
    – weberc2
    Apr 19, 2018 at 21:29
  • @weberc2 Great question! I'm not sure if this workaround for mutable def. values is possible with typing.NamedTuple. But with data classes you can use Field objects with a default_factory attr. for this, replacing your idiom with my_list: List[T] = field(default_factory=list).
    – monk-time
    Apr 20, 2018 at 7:28
105

Wrap it in a function.

NodeT = namedtuple('Node', 'val left right')

def Node(val, left=None, right=None):
  return NodeT(val, left, right)
3
  • 20
    This is clever, and can be a good option, but can also cause problems by breaking isinstance(Node('val'), Node): it will now raise an exception, rather than returning True. While a bit more verbose, @justinfay's answer (below) preserves type hierarchy information properly, so is probably a better approach if others are going to interact with Node instances. Jul 7, 2013 at 22:04
  • 7
    I like the brevity of this answer. Perhaps the concern in the comment above can be addressed by naming the function def make_node(...): rather than pretending it is a class definition. In that way users are not tempted to check for type polymorphism on the function but use the tuple definition itself. Mar 23, 2016 at 12:14
  • See my answer for a variation of this that doesn't suffer from misleading people to use isinstance incorrectly. Jul 12, 2016 at 15:09
22

This is an example straight from the docs:

Default values can be implemented by using _replace() to customize a prototype instance:

>>> Account = namedtuple('Account', 'owner balance transaction_count')
>>> default_account = Account('<owner name>', 0.0, 0)
>>> johns_account = default_account._replace(owner='John')
>>> janes_account = default_account._replace(owner='Jane')

So, the OP's example would be:

from collections import namedtuple
Node = namedtuple('Node', 'val left right')
default_node = Node(None, None, None)
example = default_node._replace(val="whut")

However, I like some of the other answers given here better. I just wanted to add this for completeness.

5
  • 2
    +1. It's very strange that they decided to go with an _ method (which basically means a private one) for something like replace which seems pretty useful..
    – sasuke
    Aug 22, 2015 at 18:50
  • @sasuke - I was wondering that too. It's already a little odd that you define the elements with a space separated string instead of *args. It may just be that it was added to the language before a lot of those things were standardized. Aug 24, 2015 at 12:43
  • 12
    The _ prefix is to avoid colliding with the names of the user defined tuple fields (relevant doc quote: "Any valid Python identifier may be used for a fieldname except for names starting with an underscore."). As for the space separated string, I think that's just to save a few keystrokes (and you can pass a sequence of strings if you prefer). Sep 9, 2015 at 18:22
  • 1
    Ah, yeah, I forgot you access the elements of the named tuple as attributes, so the _ makes a lot of sense then. Sep 9, 2015 at 18:25
  • 2
    Your solution is simple and the best. The rest is IMHO rather ugly. I would do only one small change. Instead of default_node I would prefer node_default because it makes better experience with IntelliSense. In case you start typing node you received everything you need :) Jan 1, 2016 at 18:40
20

I'm not sure if there's an easy way with just the built-in namedtuple. There's a nice module called recordtype that has this functionality:

>>> from recordtype import recordtype
>>> Node = recordtype('Node', [('val', None), ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)
4
  • 2
    Ah, not possible to use a third party package though recordtype certainly looks interesting for future work. +1
    – sasuke
    Jul 5, 2012 at 19:31
  • The module is quite small and only a single file so you could always just add it to your project.
    – jterrace
    Jul 5, 2012 at 19:42
  • Agreed pure python would be nice, but I don't think there is one :(
    – jterrace
    Jul 5, 2012 at 19:44
  • 3
    Just to note that recordtype is mutable whereas namedtuple is not. This might matter if you want the object to be hashable (which I guess you don't, since it started out as a class).
    – bavaza
    Sep 3, 2013 at 7:27
17

In python3.7+ there's a brand new defaults= keyword argument.

defaults can be None or an iterable of default values. Since fields with a default value must come after any fields without a default, the defaults are applied to the rightmost parameters. For example, if the fieldnames are ['x', 'y', 'z'] and the defaults are (1, 2), then x will be a required argument, y will default to 1, and z will default to 2.

Example usage:

$ ./python
Python 3.7.0b1+ (heads/3.7:4d65430, Feb  1 2018, 09:28:35) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import namedtuple
>>> nt = namedtuple('nt', ('a', 'b', 'c'), defaults=(1, 2))
>>> nt(0)
nt(a=0, b=1, c=2)
>>> nt(0, 3)  
nt(a=0, b=3, c=2)
>>> nt(0, c=3)
nt(a=0, b=1, c=3)
0
15

Here is a more compact version inspired by justinfay's answer:

from collections import namedtuple
from functools import partial

Node = namedtuple('Node', ('val left right'))
Node.__new__ = partial(Node.__new__, left=None, right=None)
1
  • 7
    Beware that Node(1, 2) doesn't work with this recipe, but works in @justinfay's answer. Otherwise, it's quite nifty (+1).
    – jorgeca
    Feb 20, 2014 at 1:46
6

Short, simple, and doesn't lead people to use isinstance improperly:

class Node(namedtuple('Node', ('val', 'left', 'right'))):
    @classmethod
    def make(cls, val, left=None, right=None):
        return cls(val, left, right)

# Example
x = Node.make(3)
x._replace(right=Node.make(4))
6

Python 3.7: introduction of defaults param in namedtuple definition.

Example as shown in the documentation:

>>> Account = namedtuple('Account', ['type', 'balance'], defaults=[0])
>>> Account._fields_defaults
{'balance': 0}
>>> Account('premium')
Account(type='premium', balance=0)

Read more here.

5

A slightly extended example to initialize all missing arguments with None:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        # initialize missing kwargs with None
        all_kwargs = {key: kwargs.get(key) for key in cls._fields}
        return super(Node, cls).__new__(cls, *args, **all_kwargs)
4

You can also use this:

import inspect

def namedtuple_with_defaults(type, default_value=None, **kwargs):
    args_list = inspect.getargspec(type.__new__).args[1:]
    params = dict([(x, default_value) for x in args_list])
    params.update(kwargs)

    return type(**params)

This basically gives you the possibility to construct any named tuple with a default value and override just the parameters you need, for example:

import collections

Point = collections.namedtuple("Point", ["x", "y"])
namedtuple_with_defaults(Point)
>>> Point(x=None, y=None)

namedtuple_with_defaults(Point, x=1)
>>> Point(x=1, y=None)
4

I find this version easier to read:

from collections import namedtuple

def my_tuple(**kwargs):
    defaults = {
        'a': 2.0,
        'b': True,
        'c': "hello",
    }
    default_tuple = namedtuple('MY_TUPLE', ' '.join(defaults.keys()))(*defaults.values())
    return default_tuple._replace(**kwargs)

This is not as efficient as it requires creation of the object twice but you could change that by defining the default duple inside the module and just having the function do the replace line.

4

Combining approaches of @Denis and @Mark:

from collections import namedtuple
import inspect

class Node(namedtuple('Node', 'left right val')):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        args_list = inspect.getargspec(super(Node, cls).__new__).args[len(args)+1:]
        params = {key: kwargs.get(key) for key in args_list + kwargs.keys()}
        return super(Node, cls).__new__(cls, *args, **params) 

That should support creating the tuple with positional arguments and also with mixed cases. Test cases:

>>> print Node()
Node(left=None, right=None, val=None)

>>> print Node(1,2,3)
Node(left=1, right=2, val=3)

>>> print Node(1, right=2)
Node(left=1, right=2, val=None)

>>> print Node(1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2)
Node(left=1, right=2, val=None)

but also support TypeError:

>>> Node(1, left=2)
TypeError: __new__() got multiple values for keyword argument 'left'
4

Since you are using namedtuple as a data class, you should be aware that python 3.7 will introduce a @dataclass decorator for this very purpose -- and of course it has default values.

An example from the docs:

@dataclass
class C:
    a: int       # 'a' has no default value
    b: int = 0   # assign a default value for 'b'

Much cleaner, readable and usable than hacking namedtuple. It is not hard to predict that usage of namedtuples will drop with the adoption of 3.7.

2

Inspired by this answer to a different question, here is my proposed solution based on a metaclass and using super (to handle future subcalssing correctly). It is quite similar to justinfay's answer.

from collections import namedtuple

NodeTuple = namedtuple("NodeTuple", ("val", "left", "right"))

class NodeMeta(type):
    def __call__(cls, val, left=None, right=None):
        return super(NodeMeta, cls).__call__(val, left, right)

class Node(NodeTuple, metaclass=NodeMeta):
    __slots__ = ()

Then:

>>> Node(1, Node(2, Node(4)),(Node(3, None, Node(5))))
Node(val=1, left=Node(val=2, left=Node(val=4, left=None, right=None), right=None), right=Node(val=3, left=None, right=Node(val=5, left=None, right=None)))
2

The answer by jterrace to use recordtype is great, but the author of the library recommends to use his namedlist project, which provides both mutable (namedlist) and immutable (namedtuple) implementations.

from namedlist import namedtuple
>>> Node = namedtuple('Node', ['val', ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)
1

Here's a short, simple generic answer with a nice syntax for a named tuple with default arguments:

import collections

def dnamedtuple(typename, field_names, **defaults):
    fields = sorted(field_names.split(), key=lambda x: x in defaults)
    T = collections.namedtuple(typename, ' '.join(fields))
    T.__new__.__defaults__ = tuple(defaults[field] for field in fields[-len(defaults):])
    return T

Usage:

Test = dnamedtuple('Test', 'one two three', two=2)
Test(1, 3)  # Test(one=1, three=3, two=2)

Minified:

def dnamedtuple(tp, fs, **df):
    fs = sorted(fs.split(), key=df.__contains__)
    T = collections.namedtuple(tp, ' '.join(fs))
    T.__new__.__defaults__ = tuple(df[i] for i in fs[-len(df):])
    return T
0

Using the NamedTuple class from my Advanced Enum (aenum) library, and using the class syntax, this is quite simple:

from aenum import NamedTuple

class Node(NamedTuple):
    val = 0
    left = 1, 'previous Node', None
    right = 2, 'next Node', None

The one potential drawback is the requirement for a __doc__ string for any attribute with a default value (it's optional for simple attributes). In use it looks like:

>>> Node()
Traceback (most recent call last):
  ...
TypeError: values not provided for field(s): val

>>> Node(3)
Node(val=3, left=None, right=None)

The advantages this has over justinfay's answer:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

is simplicity, as well as being metaclass based instead of exec based.

0

Another solution:

import collections


def defaultargs(func, defaults):
    def wrapper(*args, **kwargs):
        for key, value in (x for x in defaults[len(args):] if len(x) == 2):
            kwargs.setdefault(key, value)
        return func(*args, **kwargs)
    return wrapper


def namedtuple(name, fields):
    NamedTuple = collections.namedtuple(name, [x[0] for x in fields])
    NamedTuple.__new__ = defaultargs(NamedTuple.__new__, [(NamedTuple,)] + fields)
    return NamedTuple

Usage:

>>> Node = namedtuple('Node', [
...     ('val',),
...     ('left', None),
...     ('right', None),
... ])
__main__.Node

>>> Node(1)
Node(val=1, left=None, right=None)

>>> Node(1, 2, right=3)
Node(val=1, left=2, right=3)
0

If you want to retain the possibility of using type annotation, unfortunately the very nice solution by @mark-lodato isn't usable (it fails for me on setting __defaults__). An alternative is using attrs:

import attr

 
@attr.s
class Node(object):
    val: str = attr.ib()
    left: 'Node' = attr.ib(None)
    right: 'Node' = attr.ib(None)

This has:

  • type annotations
  • nice __str__ and __repr__
  • customizable, since it's a real class
  • same implementation wit all Python versions
0

1. Using NamedTuple >= Python 3.6

Since Python 3.7+ you can use NamedTuple from typing module that support defaults.

https://docs.python.org/3/library/typing.html#typing.NamedTuple

from typing import NamedTuple

class Employee(NamedTuple):
    name: str
    id: int = 3

employee = Employee('Guido')
assert employee.id == 3

Note: Although NamedTuple appears in the class statement as a superclass, it’s actually not. typing.NamedTuple uses the advanced functionality of a metaclass to customize the creation of the user’s class.

issubclass(Employee, typing.NamedTuple)
# return False
issubclass(Employee, tuple)
# return True

2. Using dataclass >= Python 3.7

from dataclasses import dataclass

@dataclass(frozen=True)
class Employee:
    name: str
    id: int = 3

employee = Employee('Guido')
assert employee.id == 3

frozen=True makes dataclass immutable.

-1

Here's a less flexible, but more concise version of Mark Lodato's wrapper: It takes the fields and defaults as a dictionary.

import collections
def namedtuple_with_defaults(typename, fields_dict):
    T = collections.namedtuple(typename, ' '.join(fields_dict.keys()))
    T.__new__.__defaults__ = tuple(fields_dict.values())
    return T

Example:

In[1]: fields = {'val': 1, 'left': 2, 'right':3}

In[2]: Node = namedtuple_with_defaults('Node', fields)

In[3]: Node()
Out[3]: Node(val=1, left=2, right=3)

In[4]: Node(4,5,6)
Out[4]: Node(val=4, left=5, right=6)

In[5]: Node(val=10)
Out[5]: Node(val=10, left=2, right=3)
1
  • 5
    dict has no guarantee of ordering. Dec 7, 2016 at 16:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.