154

Is it possible to serialize and deserialize a class in C++?

I've been using Java for 3 years now, and serialization / deserialization is fairly trivial in that language. Does C++ have similar features? Are there native libraries that handle serialization?

An example would be helpful.

3
  • 2
    not sure what you mean by "native", do you mean native C++ (like Boost.Serialization)? Do you mean using only the C++ Standard Library? Do you mean something else?
    – jwfearn
    Oct 24, 2008 at 18:54
  • 1
    i mean "not a external software-library". And sorry my english is not very well :S. I'm from Argentina
    – Agusti-N
    Oct 24, 2008 at 19:04
  • 4
    There is not native way to serialize an object (you can still dump the binary data from a POD, but you won't get what you want). Still, Boost, while not an "internal library", is the first external library you should consider to add to your compiler. Boost is of STL quality (i.e. Top Gun C++)
    – paercebal
    Oct 24, 2008 at 21:28

14 Answers 14

102

The Boost::serialization library handles this rather elegantly. I've used it in several projects. There's an example program, showing how to use it, here.

The only native way to do it is to use streams. That's essentially all the Boost::serialization library does, it extends the stream method by setting up a framework to write objects to a text-like format and read them from the same format.

For built-in types, or your own types with operator<< and operator>> properly defined, that's fairly simple; see the C++ FAQ for more information.

9
  • It seems to me that boost::serialization requires the caller to keep track of the order in which objects are written and read. Is that correct? So if there is a change in the order in which two fields are written between versions of a program then we have an incompatibility. Is this right? Jul 2, 2013 at 8:19
  • 1
    That would probably be due to the serializing functions, not the Boost::serialization code itself.
    – Head Geek
    Sep 7, 2013 at 14:39
  • 1
    @0xDEADBEEF: That probably happend when using a binary_(i|o)archive, which introduces other "problems" like endian-ness. Try text_(i|o)archive, it's more platform agnostic.
    – Ela782
    Dec 26, 2014 at 22:23
  • 5
    A specifc framwork/library solution should not be the accepted answer.
    – Andrea
    Mar 1, 2016 at 12:49
  • 6
    @Andrea: The Boost library is a special case. Until C++11 was finalized, it was all but impossible to write modern C++ code without it, so it was closer to a secondary STL than a separate library.
    – Head Geek
    Mar 1, 2016 at 13:56
63

I realize this is an old post but it's one of the first that comes up when searching for c++ serialization.

I encourage anyone who has access to C++11 to take a look at cereal, a C++11 header-only library for serialization that supports binary, JSON, and XML out of the box. cereal was designed to be easy to extend and use and has a similar syntax to boost::serialization.

2
  • 5
    The good thing about cereal is that unlike boost, it has minimal metadata (almost none). boost::serialization becomes really annoying when every time you open an archive, it writes its lib version to the stream, which makes appending to a file impossible.
    – Terry Shi
    Aug 5, 2014 at 5:52
  • @CyberSnoopy - there is a flag for suppressing this feature when an archive is created - of course you have to remember it when reading the archive as well. Nov 30, 2016 at 20:37
17

Boost is a good suggestion. But if you would like to roll your own, it's not so hard.

Basically you just need a way to build up a graph of objects and then output them to some structured storage format (JSON, XML, YAML, whatever). Building up the graph is as simple as utilizing a marking recursive decent object algorithm and then outputting all the marked objects.

I wrote an article describing a rudimentary (but still powerful) serialization system. You may find it interesting: Using SQLite as an On-disk File Format, Part 2.

0
15

I recommend Google protocol buffers. I had the chance to test drive the library on a new project and it's remarkably easy to use. The library is heavily optimized for performance.

Protobuf is different than other serialization solutions mentioned here in the sense that it does not serialize your objects, but rather generates code for objects that are serialization according to your specification.

2
15

Boost::serialization is a great option, but I've encountered a new project: Cereal which I find much more elegant! I highly suggest investigating it.

14

As far as "built-in" libraries go, the << and >> have been reserved specifically for serialization.

You should override << to output your object to some serialization context (usually an iostream) and >> to read data back from that context. Each object is responsible for outputting its aggregated child objects.

This method works fine so long as your object graph contains no cycles.

If it does, then you will have to use a library to deal with those cycles.

5
  • 3
    Surely, that can't be right... the implemented << operators are used to print human-readable text representations of objects, which is very often not what you want for serialization.
    – einpoklum
    May 7, 2014 at 6:54
  • 1
    @einpoklum Instead of defining << for the generic ostream, try defining it for a file stream. May 8, 2015 at 18:31
  • 1
    @Carcigenicate: A log file which takes human-readable text is a file stream.
    – einpoklum
    May 9, 2015 at 7:15
  • 1
    @einpoklum I'm not quite sure what you mean. Frank's right though, those operators can be used to serialize. I just defined them to serialize/deserialize a vector. May 9, 2015 at 13:07
  • 2
    I think the catch is here “You should override << to output your object to some serialization context … Each object is responsible for outputting its…” — the question is about how to avoid having to write that out laboriously for each object: how much can the language or libraries help? Jun 7, 2018 at 23:17
4

You can check the amef protocol, an example of C++ encoding in amef would be like,

    //Create a new AMEF object
    AMEFObject *object = new AMEFObject();

    //Add a child string object
    object->addPacket("This is the Automated Message Exchange Format Object property!!","adasd");   

    //Add a child integer object
    object->addPacket(21213);

    //Add a child boolean object
    object->addPacket(true);

    AMEFObject *object2 = new AMEFObject();
    string j = "This is the property of a nested Automated Message Exchange Format Object";
    object2->addPacket(j);
    object2->addPacket(134123);
    object2->addPacket(false);

    //Add a child character object
    object2->addPacket('d');

    //Add a child AMEF Object
    object->addPacket(object2);

    //Encode the AMEF obejct
    string str = new AMEFEncoder()->encode(object,false);

Decoding in java would be like,

    string arr = amef encoded byte array value;
    AMEFDecoder decoder = new AMEFDecoder()
    AMEFObject object1 = AMEFDecoder.decode(arr,true);

The Protocol implementation has codecs for both C++ and Java, the interesting part is it can retain object class representation in the form of name value pairs, I required a similar protocol in my last project, when i incidentally stumbled upon this protocol, i had actually modified the base library according to my requirements. Hope this helps you.

3

I recommend using boost serialization as described by other posters. Here is a good detailed tutorial on how to use it which complements the boost tutorials nicely: http://www.ocoudert.com/blog/2011/07/09/a-practical-guide-to-c-serialization/

3

Sweet Persist is another one.

It is possible to serialize to and from streams in XML, JSON, Lua, and binary formats.

1
2

I suggest looking into Abstract factories which is often used as a basis for serialization

I have answered in another SO question about C++ factories. Please see there if a flexible factory is of interest. I try to describe an old way from ET++ to use macros which has worked great for me.

ET++ was a project to port old MacApp to C++ and X11. In the effort of it Eric Gamma etc started to think about Design Patterns. ET++ contained automatic ways for serialization and introspection at runtime.

1

If you want simple and best performance and don't care about backward data compatibility, try HPS, it's lightweight, much faster than Boost, etc, and much easier to use than Protobuf, etc.

Example:

std::vector<int> data({22, 333, -4444});
std::string serialized = hps::serialize_to_string(data);
auto parsed = hps::parse_from_string<std::vector<int>>(serialized);
1
  • This looks great, but unmaintained. Much more-convenient than Google FlatBuffers that requires code duplication (an IDL) and separate compiler. Dec 17, 2020 at 2:48
1

I'm using the following template to implement serialization:

template <class T, class Mode = void> struct Serializer
{
    template <class OutputCharIterator>
    static void serializeImpl(const T &object, OutputCharIterator &&it)
    {
        object.template serializeThis<Mode>(it);
    }

    template <class InputCharIterator>
    static T deserializeImpl(InputCharIterator &&it, InputCharIterator &&end)
    {
        return T::template deserializeFrom<Mode>(it, end);
    }
};

template <class Mode = void, class T, class OutputCharIterator>
void serialize(const T &object, OutputCharIterator &&it)
{
    Serializer<T, Mode>::serializeImpl(object, it);
}

template <class T, class Mode = void, class InputCharIterator>
T deserialize(InputCharIterator &&it, InputCharIterator &&end)
{
    return Serializer<T, Mode>::deserializeImpl(it, end);
}

template <class Mode = void, class T, class InputCharIterator>
void deserialize(T &result, InputCharIterator &&it, InputCharIterator &&end)
{
    result = Serializer<T, Mode>::deserializeImpl(it, end);
}

Here T is the type you want to serialize Mode is a dummy type to differentiate between different kinds of serialization, eg. the same integer can be serialized as little endian, big endian, varint, etc.

By default the Serializer delegates the task to the object being serialized. For built in types you should make a template specialization of the Serializer.

Convenience function templates are also provided.

For example little endian serialization of unsigned integers:

struct LittleEndianMode
{
};

template <class T>
struct Serializer<
    T, std::enable_if_t<std::is_unsigned<T>::value, LittleEndianMode>>
{
    template <class InputCharIterator>
    static T deserializeImpl(InputCharIterator &&it, InputCharIterator &&end)
    {
        T res = 0;

        for (size_t i = 0; i < sizeof(T); i++)
        {
            if (it == end) break;
            res |= static_cast<T>(*it) << (CHAR_BIT * i);
            it++;
        }

        return res;
    }

    template <class OutputCharIterator>
    static void serializeImpl(T number, OutputCharIterator &&it)
    {
        for (size_t i = 0; i < sizeof(T); i++)
        {
            *it = (number >> (CHAR_BIT * i)) & 0xFF;
            it++;
        }
    }
};

Then to serialize:

std::vector<char> serialized;
uint32_t val = 42;
serialize<LittleEndianMode>(val, std::back_inserter(serialized));

To deserialize:

uint32_t val;
deserialize(val, serialized.begin(), serialized.end());

Due to the abstract iterator logic, it should work with any iterator (eg. stream iterators), pointer, etc.

1
  • While I do like the automated boost library's its nice to know some easy to read template code to show how you can build it quick and simply. Jan 27, 2022 at 21:05
0

Here is a simple serializer library I knocked up. It's header only, c11 and has examples for serializing basic types. Here's one for a map to class.

https://github.com/goblinhack/simple-c-plus-plus-serializer

#include "c_plus_plus_serializer.h"

class Custom {
public:
    int a;
    std::string b;
    std::vector c;

    friend std::ostream& operator<<(std::ostream &out, 
                                    Bits my)
    {
        out << bits(my.t.a) << bits(my.t.b) << bits(my.t.c);
        return (out);
    }

    friend std::istream& operator>>(std::istream &in, 
                                    Bits my)
    {
        in >> bits(my.t.a) >> bits(my.t.b) >> bits(my.t.c);
        return (in);
    }

    friend std::ostream& operator<<(std::ostream &out, 
                                    class Custom &my)
    {
        out << "a:" << my.a << " b:" << my.b;

        out << " c:[" << my.c.size() << " elems]:";
        for (auto v : my.c) {
            out << v << " ";
        }
        out << std::endl;

        return (out);
    }
};

static void save_map_key_string_value_custom (const std::string filename)
{
    std::cout << "save to " << filename << std::endl;
    std::ofstream out(filename, std::ios::binary );

    std::map< std::string, class Custom > m;

    auto c1 = Custom();
    c1.a = 1;
    c1.b = "hello";
    std::initializer_list L1 = {"vec-elem1", "vec-elem2"};
    std::vector l1(L1);
    c1.c = l1;

    auto c2 = Custom();
    c2.a = 2;
    c2.b = "there";
    std::initializer_list L2 = {"vec-elem3", "vec-elem4"};
    std::vector l2(L2);
    c2.c = l2;

    m.insert(std::make_pair(std::string("key1"), c1));
    m.insert(std::make_pair(std::string("key2"), c2));

    out << bits(m);
}

static void load_map_key_string_value_custom (const std::string filename)
{
    std::cout << "read from " << filename << std::endl;
    std::ifstream in(filename);

    std::map< std::string, class Custom > m;

    in >> bits(m);
    std::cout << std::endl;

    std::cout << "m = " << m.size() << " list-elems { " << std::endl;
    for (auto i : m) {
        std::cout << "    [" << i.first << "] = " << i.second;
    }
    std::cout << "}" << std::endl;
}

void map_custom_class_example (void)
{
    std::cout << "map key string, value class" << std::endl;
    std::cout << "============================" << std::endl;
    save_map_key_string_value_custom(std::string("map_of_custom_class.bin"));
    load_map_key_string_value_custom(std::string("map_of_custom_class.bin"));
    std::cout << std::endl;
}

Output:

map key string, value class
============================
save to map_of_custom_class.bin
read from map_of_custom_class.bin

m = 2 list-elems {
    [key1] = a:1 b:hello c:[2 elems]:vec-elem1 vec-elem2
    [key2] = a:2 b:there c:[2 elems]:vec-elem3 vec-elem4
}
0

C++14 (C++17 recommended) boost prf

  • no macros
  • no code to be explicitly written

demo

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.