75

I usually use stringstream to write into in-memory string. Is there a way to write to a char buffer in binary mode? Consider the following code:

stringstream s;
s << 1 << 2 << 3;
const char* ch = s.str().c_str();

The memory at ch will look like this: 0x313233 - the ASCII codes of the characters 1, 2 and 3. I'm looking for a way to write the binary values themselves. That is, I want 0x010203 in the memory. The problem is that I want to be able to write a function

void f(ostream& os)
{
    os << 1 << 2 << 3;
}

And decide outside what kind of stream to use. Something like this:

mycharstream c;
c << 1 << 2 << 3; // c.data == 0x313233;
mybinstream b;
b << 1 << 2 << 3; // b.data == 0x010203;

Any ideas?

4
  • 1
    That's hex, not binary. Why can't you write 0x01, 0x02, etc., though... those are actual ASCII characters, after all.
    – jrockway
    Oct 13, 2009 at 10:03
  • 3
    He wants the contents of memory (the actual bytes) to be 0x010203 (66051 decimal), not the string "0x010203".
    – KeithB
    Oct 13, 2009 at 11:18
  • 1
    I've modified the question. Hope it's more clear now.
    – FireAphis
    Oct 13, 2009 at 14:20
  • 3
    Excellent question. Too bad it is impossible to give a good answer, because this is a design bug in the standard libraries.
    – isekaijin
    Jan 12, 2013 at 3:50

7 Answers 7

47

To read and write binary data to streams, including stringstreams, use the read() and write() member functions. So

unsigned char a(1), b(2), c(3), d(4);
std::stringstream s;
s.write(reinterpret_cast<const char*>(&a), sizeof(unsigned char));
s.write(reinterpret_cast<const char*>(&b), sizeof(unsigned char));
s.write(reinterpret_cast<const char*>(&c), sizeof(unsigned char));
s.write(reinterpret_cast<const char*>(&d), sizeof(unsigned char));

s.read(reinterpret_cast<char*>(&v), sizeof(unsigned int)); 
std::cout << std::hex << v << "\n";

This gives 0x4030201 on my system.

Edit: To make this work transparently with the insertion and extraction operators (<< and >>), your best bet it to create a derived streambuf that does the right thing, and pass that to whatever streams you want to use.

4
  • It definitely answers the first part of the question, but is there a way to make the insertion look always the same (i.e. s << a) but the inner data representation differ depending on the type of the stream?
    – FireAphis
    Oct 13, 2009 at 14:17
  • Your own streambuf can't do this; formatting is done in the (non-virtual) istream and ostream methods and the result of that is what the streambuf sees.
    – Roger Pate
    Jul 17, 2010 at 0:30
  • The question actually shows the in-memory result 0x010203 whereas this will likely produce 0x00000001 0x00000002 0x00000003 (assuming sizeof(int)==4).
    – MSalters
    Jan 13, 2017 at 13:17
  • @MSalters You're right, apparently 6 year younger me was an idiot.
    – KeithB
    Jan 13, 2017 at 17:55
12

You can do this sort of thing with templates. E.g:

//struct to hold the value:
template<typename T> struct bits_t { T t; }; //no constructor necessary
//functions to infer type, construct bits_t with a member initialization list
//use a reference to avoid copying. The non-const version lets us extract too
template<typename T> bits_t<T&> bits(T &t) { return bits_t<T&>{t}; }
template<typename T> bits_t<const T&> bits(const T& t) { return bits_t<const T&>{t}; }
//insertion operator to call ::write() on whatever type of stream
template<typename S, typename T>
S& operator<<(S &s, bits_t<T> b) {
    return s.write((char*)&b.t, sizeof(T));
}
//extraction operator to call ::read(), require a non-const reference here
template<typename S, typename T>
S& operator>>(S& s, bits_t<T&> b) {
    return s.read((char*)&b.t, sizeof(T));
}

It could use some cleanup, but it's functional. E.g:

//writing
std::ofstream f = /*open a file*/;
int a = 5, b = -1, c = 123456;
f << bits(a) << bits(b) << bits(c);

//reading
std::ifstream f2 = /*open a file*/;
int a, b, c;
f >> bits(a) >> bits(b) >> bits(c);
2
  • 1
    I prefer this answer because it's not confusing and it can also wrap other things like vector<float>
    – 陈家胜
    Apr 20, 2018 at 13:26
  • Hey, @SamuelPowell I like this approach so much, I took it a bit further, and wrote more serializers on top of this approach. I like this as it has such low complexity compared to other C++ serializers. If interested look at github.com/goblinhack/simple-c-plus-plus-serializer - would be interested in your comments. I found I had to remove the stream type in the templates due to (I think) operator overloading issues. Anyway, it works well for many types.
    – Goblinhack
    Sep 8, 2019 at 2:44
5

Well, just use characters, not integers.

s << char(1) << char(2) << char(3);
3

overloading some unusual operators works rather well. Here below I choosed to overload <= because it has the same left-to-right associativity as << and has somehow a close look-and-feel ...

#include <iostream>
#include <stdint.h>
#include <arpa/inet.h>

using namespace std;

ostream & operator<= (ostream& cout, string const& s) {
    return cout.write (s.c_str(), s.size());
}
ostream & operator<= (ostream& cout, const char *s) {
    return cout << s;
}
ostream & operator<= (ostream&, int16_t const& i) {
    return cout.write ((const char *)&i, 2);
}
ostream & operator<= (ostream&, int32_t const& i) {
    return cout.write ((const char *)&i, 4);
}
ostream & operator<= (ostream&, uint16_t const& i) {
    return cout.write ((const char *)&i, 2);
}
ostream & operator<= (ostream&, uint32_t const& i) {
    return cout.write ((const char *)&i, 4);
}

int main() {
    string s("some binary data follow : ");

    cout <= s <= " (machine ordered) : " <= (uint32_t)0x31323334 <= "\n"
         <= s <= " (network ordered) : " <= htonl(0x31323334) ;
    cout << endl;

    return 0;
}

There are several drawbacks :

  • the new meaning of <= may confuse readers or lead to unexpected results :

    cout <= 31 <= 32;
    

    won't give the same result as

    cout <= (31 <= 32);
    
  • the endianess isn't clearly mentionned at reading the code, as illustrated in the above example.

  • it cannot mix simply with << because it doesn't belong to the same group of precedence. I usually use parenthesis to clarify such as :

    ( cout <= htonl(a) <= htonl(b) ) << endl;
    
1
  • 5
    That's a cool proof of concept, but note that C++'s overloaded operators are considered evil because they allow for this. The non-obvious overload of << is justified only because it's a standard overload. No new hacky overloads should be invented and the overloading itself should be used with a great care.
    – cubuspl42
    Jul 10, 2014 at 0:04
2

For this use case I implemented myself a "raw shift operator":

template <typename T, class... StreamArgs>
inline std::basic_ostream<StreamArgs...> &
operator <= (std::basic_ostream<StreamArgs...> & out, T const & data) {
        out.write(reinterpret_cast<char const *>(&data), sizeof(T));
        return out;
}

Put it somewhere convenient and use it like this:

std::cout <= 1337 <= 1337ULL <= 1337. <= 1337.f;

Advantages:

  • chainable
  • automatic sizeof()
  • takes arrays and struct/class instances, too

Disadvantages:

  • unsafe for non-POD objects: leaks pointers and padding
  • output is platform specific: padding, endianess, integer types
0
#include <sstream>

class bostringstream {
public:
  bostringstream() : oss() {}

  template <typename T, typename std::enable_if<std::is_fundamental<T>::value,
                                                bool>::type = true>
  bostringstream& operator<<(const T& v) {
    oss.write((char*)&v, sizeof(T));
    return *this;
  }

  template <typename T, typename std::enable_if<
                            std::is_fundamental<typename T::value_type>::value,
                            bool>::type = true>
  bostringstream& operator<<(const T& v) {
    oss.write((char*)v.data(), v.size() * sizeof(typename T::value_type));
    return *this;
  }

  template <typename _InputIterator>
  bostringstream& write(_InputIterator first, _InputIterator last) {
    char* data = (char*)&(*first);
    auto n = std::distance(first, last);
    oss.write(data, n * sizeof(*first));
    return *this;
  }

  template <typename T, typename std::enable_if<std::is_fundamental<T>::value,
                                                bool>::type = true>
  bostringstream& write(const T* v, std::streamsize count) {
    oss.write((char*)v, sizeof(T) * count);
    return *this;
  }

  auto rdbuf() const { return oss.rdbuf(); }

  auto str() const { return oss.str(); }

  std::size_t size() { return oss.tellp(); }

protected:
  std::ostringstream oss;
};

Example:

#include <array>
#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include "bsstream.hpp"

int main(int argc, char **argv) {

  int i = 1;
  float j = 1.1;
  double k = 1.2;
  std::vector<int> ii{1,2};
  std::vector<double> jj{1.2,2.2};
  std::string kk = "abcd";
  std::array<int, 2> ll{3,4};
  int l[] = {1,2};

  bostringstream of;
  of << i << j <<k;
  of <<ii << jj << kk << ll;
  of.write(l, 2);

  std::ofstream oof("foo.bin", std::ios::binary);
  oof << of.str();
  oof.close();

}

Not an elegant solution but works and flexible

0

I really like Han Luo's approach and have verified it to work great! If changing the oss member variable to use std::stringstream (vs. ostringstream) this class could also be used for extraction with overloaded stream extraction operators like this:

    template <typename T, typename std::enable_if<std::is_fundamental<T>::value, bool>::type = true>
    bostringstream& operator>> (T& v)
    {
        char buffer[sizeof(T)];
        oss.read(buffer, sizeof(T));
        v = *(T*)buffer;
        return *this;
    }

The example templates support integral types and if adding a new template for std::is_compound it's possible to also support types like std::map. For things like std::vector, that are "is_fundemental" I'd suggest pushing the size to the stream first, so on the extraction side it can be pulled to know how many elements to pull afterwards. This approach can work with the common std::vector and std::map types quite nicely.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.