256

I am trying to create a JSON string representation of a class instance and having difficulty. Let's say the class is built like this:

class testclass:
    value1 = "a"
    value2 = "b"

A call to the json.dumps is made like this:

t = testclass()
json.dumps(t)

It is failing and telling me that the testclass is not JSON serializable.

TypeError: <__main__.testclass object at 0x000000000227A400> is not JSON serializable

I have also tried using the pickle module :

t = testclass()
print(pickle.dumps(t, pickle.HIGHEST_PROTOCOL))

And it gives class instance information but not a serialized content of the class instance.

b'\x80\x03c__main__\ntestclass\nq\x00)\x81q\x01}q\x02b.'

What am I doing wrong?

4
  • 50
    Use one line, s = json.dumps(obj, default=lambda x: x.__dict__), to serialize object's instance variables (self.value1, self.value2, ...). Its the simplest and the most straight forward way. It will serialize nested object structures. The default function is called when any given object is not directly serializable. You can also look at my answer below. I found the popular answers unnecessarily complex, which were probably true quite a long time back.
    – codeman48
    Dec 13, 2017 at 7:25
  • 2
    Your testclass has no __init__() method, so all instances will share the same two class attributes (value1 and value2) defined in the class statement. Do you understand the difference between a class and an instance of one?
    – martineau
    Jan 30, 2018 at 2:44
  • 2
    There is a python library for this github.com/jsonpickle/jsonpickle (commenting since answer is too below in the thread and wont be reachable.) May 10, 2019 at 4:54

17 Answers 17

305

The basic problem is that the JSON encoder json.dumps() only knows how to serialize a limited set of object types by default, all built-in types. List here: https://docs.python.org/3.3/library/json.html#encoders-and-decoders

One good solution would be to make your class inherit from JSONEncoder and then implement the JSONEncoder.default() function, and make that function emit the correct JSON for your class.

A simple solution would be to call json.dumps() on the .__dict__ member of that instance. That is a standard Python dict and if your class is simple it will be JSON serializable.

class Foo(object):
    def __init__(self):
        self.x = 1
        self.y = 2

foo = Foo()
s = json.dumps(foo) # raises TypeError with "is not JSON serializable"

s = json.dumps(foo.__dict__) # s set to: {"x":1, "y":2}

The above approach is discussed in this blog posting:

    Serializing arbitrary Python objects to JSON using _dict_

And, of course, Python offers a built-in function that accesses .__dict__ for you, called vars().

So the above example can also be done as:

s = json.dumps(vars(foo)) # s set to: {"x":1, "y":2}
9
  • 4
    I tried this. The end result of a call to json.dumps(t.__dict__) is just {}.
    – ferhan
    Apr 20, 2012 at 19:11
  • 9
    That is because your class doesn't have a .__init__() method function, so your class instance has an empty dictionary. In other words, {} is the correct result for your example code.
    – steveha
    Apr 20, 2012 at 19:37
  • 4
    Thanks. This does the trick. I added a simple init with no parameters and now calling the json.dumps(t.__dict__) returns proper data in the format of: {"value2": "345", "value1": "123"} I had seen posts like this before, wasn't sure whether I needed a custom serializer for members, needing init wasn't mentioned explicitly or I missed it. Thank you.
    – ferhan
    Apr 20, 2012 at 19:56
  • 4
    This work for a single class but not with related classes objets Dec 18, 2015 at 14:54
  • 4
    @NwawelAIroume: True. If you have an object which e.g. is containing multiple objects in a list the error is still is not JSON serializable
    – gies0r
    Mar 5, 2017 at 17:47
88

There's one way that works great for me that you can try out:

json.dumps() can take an optional parameter default where you can specify a custom serializer function for unknown types, which in my case looks like

def serialize(obj):
    """JSON serializer for objects not serializable by default json code"""

    if isinstance(obj, date):
        serial = obj.isoformat()
        return serial

    if isinstance(obj, time):
        serial = obj.isoformat()
        return serial

    return obj.__dict__

First two ifs are for date and time serialization and then there is a obj.__dict__ returned for any other object.

the final call looks like:

json.dumps(myObj, default=serialize)

It's especially good when you are serializing a collection and you don't want to call __dict__ explicitly for every object. Here it's done for you automatically.

So far worked so good for me, looking forward for your thoughts.

4
  • 1
    I get NameError: name 'serialize' is not defined. Any tips? Apr 10, 2019 at 16:50
  • Very nice. Just for classes that have slots: try: dict = obj.__dict__ except AttributeError: dict = {s: getattr(obj, s) for s in obj.__slots__ if hasattr(obj, s)} return dict
    – Fantastory
    Jun 28, 2019 at 9:20
  • @Fantastory Why try-except instead of just checking with hasattr(obj, "__dict__")? Much cleaner and professional, IMHO. Feb 5 at 18:38
  • Is not try the python way?
    – Fantastory
    Feb 6 at 19:34
79

You can specify the default named parameter in the json.dumps() function:

json.dumps(obj, default=lambda x: x.__dict__)

Explanation:

Form the docs (2.7, 3.6):

``default(obj)`` is a function that should return a serializable version
of obj or raise TypeError. The default simply raises TypeError.

(Works on Python 2.7 and Python 3.x)

Note: In this case you need instance variables and not class variables, as the example in the question tries to do. (I am assuming the asker meant class instance to be an object of a class)

I learned this first from @phihag's answer here. Found it to be the simplest and cleanest way to do the job.

2
  • 11
    This worked for me, but because of datetime.date members I changed it slightly: default=lambda x: getattr(x, '__dict__', str(x)) Nov 20, 2018 at 5:30
  • @Dakota nice work-around; datetime.date is a C implementation hence it has no __dict__ attribute. IMHO for uniformity's sake, datetime.date should be having it...
    – codeman48
    Nov 20, 2018 at 10:32
31

Using jsonpickle

import jsonpickle

object = YourClass()
json_object = jsonpickle.encode(object)
1
  • jsonpickle worked fine until I ran into Metaclasses #105. Enum(s) went through a weird serialisation
    – Nithish
    Jul 20 at 11:15
25

I just do:

data=json.dumps(myobject.__dict__)

This is not the full answer, and if you have some sort of complicated object class you certainly will not get everything. However I use this for some of my simple objects.

One that it works really well on is the "options" class that you get from the OptionParser module. Here it is along with the JSON request itself.

  def executeJson(self, url, options):
        data=json.dumps(options.__dict__)
        if options.verbose:
            print data
        headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
        return requests.post(url, data, headers=headers)
2
  • You might want to remove self, if you are not using this inside a class.
    – SpiRail
    Jul 10, 2013 at 14:37
  • 4
    That will work okay, as long as the object isn't composed of other objects.
    – Haroldo_OK
    May 27, 2014 at 11:56
8

JSON is not really meant for serializing arbitrary Python objects. It's great for serializing dict objects, but the pickle module is really what you should be using in general. Output from pickle is not really human-readable, but it should unpickle just fine. If you insist on using JSON, you could check out the jsonpickle module, which is an interesting hybrid approach.

https://github.com/jsonpickle/jsonpickle

3
  • 11
    The main problem I see with pickle is that it's a Python-specific format, while JSON is a platform-independant format. JSON is specially useful if you're writing either a web application or a backend for some mobile application. That having been said, thanks for pointing out to jsonpickle.
    – Haroldo_OK
    May 27, 2014 at 11:50
  • @Haroldo_OK Doesn't jsonpickle still export to JSON, just not very human readable?
    – Caelum
    Nov 24, 2015 at 10:51
  • The question isn't about serialization, it is about serializing to JSON. There are a lot of reasons for doing that and pickle would be a horrible tool for most of them.
    – Tarynn
    Jan 21, 2022 at 21:02
8

Python3.x

The best aproach I could reach with my knowledge was this.
Note that this code treat set() too.
This approach is generic just needing the extension of class (in the second example).
Note that I'm just doing it to files, but it's easy to modify the behavior to your taste.

However this is a CoDec.

With a little more work you can construct your class in other ways. I assume a default constructor to instance it, then I update the class dict.

import json
import collections


class JsonClassSerializable(json.JSONEncoder):

    REGISTERED_CLASS = {}

    def register(ctype):
        JsonClassSerializable.REGISTERED_CLASS[ctype.__name__] = ctype

    def default(self, obj):
        if isinstance(obj, collections.Set):
            return dict(_set_object=list(obj))
        if isinstance(obj, JsonClassSerializable):
            jclass = {}
            jclass["name"] = type(obj).__name__
            jclass["dict"] = obj.__dict__
            return dict(_class_object=jclass)
        else:
            return json.JSONEncoder.default(self, obj)

    def json_to_class(self, dct):
        if '_set_object' in dct:
            return set(dct['_set_object'])
        elif '_class_object' in dct:
            cclass = dct['_class_object']
            cclass_name = cclass["name"]
            if cclass_name not in self.REGISTERED_CLASS:
                raise RuntimeError(
                    "Class {} not registered in JSON Parser"
                    .format(cclass["name"])
                )
            instance = self.REGISTERED_CLASS[cclass_name]()
            instance.__dict__ = cclass["dict"]
            return instance
        return dct

    def encode_(self, file):
        with open(file, 'w') as outfile:
            json.dump(
                self.__dict__, outfile,
                cls=JsonClassSerializable,
                indent=4,
                sort_keys=True
            )

    def decode_(self, file):
        try:
            with open(file, 'r') as infile:
                self.__dict__ = json.load(
                    infile,
                    object_hook=self.json_to_class
                )
        except FileNotFoundError:
            print("Persistence load failed "
                  "'{}' do not exists".format(file)
                  )


class C(JsonClassSerializable):

    def __init__(self):
        self.mill = "s"


JsonClassSerializable.register(C)


class B(JsonClassSerializable):

    def __init__(self):
        self.a = 1230
        self.c = C()


JsonClassSerializable.register(B)


class A(JsonClassSerializable):

    def __init__(self):
        self.a = 1
        self.b = {1, 2}
        self.c = B()

JsonClassSerializable.register(A)

A().encode_("test")
b = A()
b.decode_("test")
print(b.a)
print(b.b)
print(b.c.a)

Edit

With some more of research I found a way to generalize without the need of the SUPERCLASS register method call, using a metaclass

import json
import collections

REGISTERED_CLASS = {}

class MetaSerializable(type):

    def __call__(cls, *args, **kwargs):
        if cls.__name__ not in REGISTERED_CLASS:
            REGISTERED_CLASS[cls.__name__] = cls
        return super(MetaSerializable, cls).__call__(*args, **kwargs)


class JsonClassSerializable(json.JSONEncoder, metaclass=MetaSerializable):

    def default(self, obj):
        if isinstance(obj, collections.Set):
            return dict(_set_object=list(obj))
        if isinstance(obj, JsonClassSerializable):
            jclass = {}
            jclass["name"] = type(obj).__name__
            jclass["dict"] = obj.__dict__
            return dict(_class_object=jclass)
        else:
            return json.JSONEncoder.default(self, obj)

    def json_to_class(self, dct):
        if '_set_object' in dct:
            return set(dct['_set_object'])
        elif '_class_object' in dct:
            cclass = dct['_class_object']
            cclass_name = cclass["name"]
            if cclass_name not in REGISTERED_CLASS:
                raise RuntimeError(
                    "Class {} not registered in JSON Parser"
                    .format(cclass["name"])
                )
            instance = REGISTERED_CLASS[cclass_name]()
            instance.__dict__ = cclass["dict"]
            return instance
        return dct

    def encode_(self, file):
        with open(file, 'w') as outfile:
            json.dump(
                self.__dict__, outfile,
                cls=JsonClassSerializable,
                indent=4,
                sort_keys=True
            )

    def decode_(self, file):
        try:
            with open(file, 'r') as infile:
                self.__dict__ = json.load(
                    infile,
                    object_hook=self.json_to_class
                )
        except FileNotFoundError:
            print("Persistence load failed "
                  "'{}' do not exists".format(file)
                  )


class C(JsonClassSerializable):

    def __init__(self):
        self.mill = "s"


class B(JsonClassSerializable):

    def __init__(self):
        self.a = 1230
        self.c = C()


class A(JsonClassSerializable):

    def __init__(self):
        self.a = 1
        self.b = {1, 2}
        self.c = B()


A().encode_("test")
b = A()
b.decode_("test")
print(b.a)
# 1
print(b.b)
# {1, 2}
print(b.c.a)
# 1230
print(b.c.c.mill)
# s
1
  • I love it. This is amazing implementation. I have just 1 question if I may, why did you use call method and not new method in the meta class? I ask, because the call method gets called every time a new instance is created, so the code tries to register the class for every new instance. Not a big deal, and as I wrote I love this implementation it seems really great :-)
    – SagiZiv
    Aug 27, 2021 at 15:38
6

This can be easily handled with pydantic, as it already has this functionality built-in.

Option 1: normal way

from pydantic import BaseModel

class testclass(BaseModel):
    value1: str = "a"
    value2: str = "b"

test = testclass()

>>> print(test.json(indent=4))
{
    "value1": "a",
    "value2": "b"
}

Option 2: using pydantic's dataclass

import json
from pydantic.dataclasses import dataclass
from pydantic.json import pydantic_encoder

@dataclass
class testclass:
    value1: str = "a"
    value2: str = "b"

test = testclass()
>>> print(json.dumps(test, indent=4, default=pydantic_encoder))
{
    "value1": "a",
    "value2": "b"
}
5

I believe instead of inheritance as suggested in accepted answer, it's better to use polymorphism. Otherwise you have to have a big if else statement to customize encoding of every object. That means create a generic default encoder for JSON as:

def jsonDefEncoder(obj):
   if hasattr(obj, 'jsonEnc'):
      return obj.jsonEnc()
   else: #some default behavior
      return obj.__dict__

and then have a jsonEnc() function in each class you want to serialize. e.g.

class A(object):
   def __init__(self,lengthInFeet):
      self.lengthInFeet=lengthInFeet
   def jsonEnc(self):
      return {'lengthInMeters': lengthInFeet * 0.3 } # each foot is 0.3 meter

Then you call json.dumps(classInstance,default=jsonDefEncoder)

1
  • This is the correct answer. The class knows how to serialize itself. It is too bad that you can't just pass the method name directly to json.dumps
    – Tarynn
    Jan 21, 2022 at 21:01
5

Here are two simple functions for serialization of any non-sophisticated classes, nothing fancy as explained before.

I use this for configuration type stuff because I can add new members to the classes with no code adjustments.

import json

class SimpleClass:
    def __init__(self, a=None, b=None, c=None):
        self.a = a
        self.b = b
        self.c = c

def serialize_json(instance=None, path=None):
    dt = {}
    dt.update(vars(instance))

    with open(path, "w") as file:
        json.dump(dt, file)

def deserialize_json(cls=None, path=None):
    def read_json(_path):
        with open(_path, "r") as file:
            return json.load(file)

    data = read_json(path)

    instance = object.__new__(cls)

    for key, value in data.items():
        setattr(instance, key, value)

    return instance

# Usage: Create class and serialize under Windows file system.
write_settings = SimpleClass(a=1, b=2, c=3)
serialize_json(write_settings, r"c:\temp\test.json")

# Read back and rehydrate.
read_settings = deserialize_json(SimpleClass, r"c:\temp\test.json")

# results are the same.
print(vars(write_settings))
print(vars(read_settings))

# output:
# {'c': 3, 'b': 2, 'a': 1}
# {'c': 3, 'b': 2, 'a': 1}
2
  • Hey @GBGOLC, Thanks for that awesome piece of code! I've tried to implement it with json.loads and dumps since there aren't any files where I'm using the code. I'm struggling to decode the Vehicle class in my example, which refers to wheels. Here's how I've changed it. Help much appreciated.
    – Tom
    Apr 28, 2021 at 20:04
  • 1
    Ok, I solved my own problem: since decode_ changes the instance it is passed, there was no return value to pass to a new instance. Your code was fine and only my way of calling decode_ needed changing. Thanks again for the great code!
    – Tom
    Apr 29, 2021 at 8:23
3

Use arbitrary, extensible object, and then serialize it to JSON:

import json

class Object(object):
    pass

response = Object()
response.debug = []
response.result = Object()

# Any manipulations with the object:
response.debug.append("Debug string here")
response.result.body = "404 Not Found"
response.result.code = 404

# Proper JSON output, with nice formatting:
print(json.dumps(response, indent=4, default=lambda x: x.__dict__))
2

There are some good answers on how to get started on doing this. But there are some things to keep in mind:

  • What if the instance is nested inside a large data structure?
  • What if also want the class name?
  • What if you want to deserialize the instance?
  • What if you're using __slots__ instead of __dict__?
  • What if you just don't want to do it yourself?

json-tricks is a library (that I made and others contributed to) which has been able to do this for quite a while. For example:

class MyTestCls:
    def __init__(self, **kwargs):
        for k, v in kwargs.items():
            setattr(self, k, v)

cls_instance = MyTestCls(s='ub', dct={'7': 7})

json = dumps(cls_instance, indent=4)
instance = loads(json)

You'll get your instance back. Here the json looks like this:

{
    "__instance_type__": [
        "json_tricks.test_class",
        "MyTestCls"
    ],
    "attributes": {
        "s": "ub",
        "dct": {
            "7": 7
        }
    }
}

If you like to make your own solution, you might look at the source of json-tricks so as not to forget some special cases (like __slots__).

It also does other types like numpy arrays, datetimes, complex numbers; it also allows for comments.

2

An approach which I have been using in my Flask app to serialize Class instance to JSON response.

Github project for reference

from json import JSONEncoder
import json
from typing import List

class ResponseEncoder(JSONEncoder):
    def default(self, o):
        return o.__dict__

class ListResponse:
    def __init__(self, data: List):
        self.data = data
        self.count = len(data)

class A:
    def __init__(self, message: str):
        self.message = message

class B:
    def __init__(self, record: A):
        self.record = record

class C:
    def __init__(self, data: B):
        self.data = data

Now create an instance of A, B, C then encode.

data_a = A('Test Data')
data_b = B(data_a)
data_c = C(data_b)

response = ResponseEncoder().encode(data_c)
json_response = json.loads(response)

Output

{
    "data": {
        "record": {
            "message": "Test Data"
        }
    }
}

For list type response

records = ['One', 'Two', 'Three']
list_response = ListResponse(records)
response = ResponseEncoder().encode(list_response)
json_response = json.loads(response)

Output

{
    "data": [
        "One",
        "Two",
        "Three"
    ],
    "count": 3
}
2

You could use Jsonic for to serialize pretty much anything to JSON:

https://github.com/OrrBin/Jsonic

Example:

class TestClass:
def __init__(self):
    self.x = 1
    self.y = 2

instance = TestClass()
s = serialize(instance): # instance s set to: {"x":1, "y":2}
d = deserialize(s) # d is a new class instance of TestClass

Jsonic has some nice features like declaring class attributes transient and type safe deserialization.

(A few years late with the answer, but i think it might help others)

1

There's another really simple and elegant approach that can be applied here which is to just subclass 'dict' since it is serializable by default.

from json import dumps

class Response(dict):
    def __init__(self, status_code, body):
        super().__init__(
            status_code = status_code,
            body = body
        )

r = Response()
dumps(r)
1
  • This leads to some weird outcomes, which are probably OK, just confusing: r = Response(200, 'blah'); r['a'] = 1; r.b = 2 results in r == {'status_code': 200, 'body': 'blah', 'a': 1} and r.__dict__ == {'b': 2}. I would love to know where dict actually stores it's keys and values.
    – naught101
    Feb 8, 2021 at 4:02
1

I made a function for this, it works pretty well:

def serialize(x,*args,**kwargs):
    kwargs.setdefault('default',lambda x:getattr(x,'__dict__',dict((k,getattr(x,k) if not callable(getattr(x,k)) else repr(getattr(x,k))) for k in dir(x) if not (k.startswith('__') or isinstance(getattr(x,k),x.__class__)))))
    return json.dumps(x,*args,**kwargs)
0

You can try objprint, which is a light-weighted library to print Python objects, and it supports json output.

pip install objprint
from objprint import objjson
t = testclass()
json_obj = objjson(t)
print(json.dumps(json_obj))

objjson basically converts an arbitrary object to a jsonifiable object, with a special key .type for it's original Python type, if it's not builtin type like dict, list etc.

If you simply want to print it, you can use op which is normally used to print objects in a human-readable format.

from objprint import op
t = testclass()
op(t, format="json", indent=2)

# If you want to dump to a file
with open("my_obj.json", "w") as f:
    # This is the same usage as print
    op(t, format="json", file=f)

Not the answer you're looking for? Browse other questions tagged or ask your own question.