60

I have a char* and the data length that I'm receiving from a library, and I need to pass the data to a function that takes an istream.

I know I can create a stringstream but that will copy all the data. And also, the data will surely have 0s since it's a zip file, and creating a stringstream will take the data until the first 0 I think.

Is there any way to create an istream from a char* and it's size without copying all the data?

4
  • "will take the data until the first 0 I think." Why do you think that?
    – FailedDev
    Oct 16, 2011 at 2:15
  • 3
    @FailedDev: Because constructing an std::string from a char* will stop at the first 0. However, you can get around this with the iterator range constructor, as in: std::string(buffer, buffer + buffer_size), this doesn't get around the copying problem though. Oct 16, 2011 at 2:24
  • 1
    @FailedDev: That's the character '0', usually equivalent to it's ASCII value 48. It's not the integer value 0, which would be represented in a string by '\0'. Your example, revised: ideone.com/UpPSf Oct 16, 2011 at 2:33
  • @BenjaminLindley I didn't realize the op was talking about null terminating chars..
    – FailedDev
    Oct 16, 2011 at 2:38

7 Answers 7

83

Here's a non-deprecated method found on the web, has you derive your own std::streambuf class, but easy and seems to work:

#include <iostream>
#include <istream>
#include <streambuf>
#include <string>

struct membuf : std::streambuf
{
    membuf(char* begin, char* end) {
        this->setg(begin, begin, end);
    }
};

int main()
{
    char buffer[] = "I'm a buffer with embedded nulls\0and line\n feeds";

    membuf sbuf(buffer, buffer + sizeof(buffer));
    std::istream in(&sbuf);
    std::string line;
    while (std::getline(in, line)) {
        std::cout << "line: " << line << "\n";
    }
    return 0;
}

Which outputs:

line: I'm a buffer with embedded nullsand line
line:  feeds
6
  • 2
    Nice, simple solution (I am not the downvoter). But be aware that tellg and seekg on the iostream will not work unless you implement seekoff and seekpos in the streambuf. You might or might not want to provide setbuf as well.
    – Nemo
    Oct 16, 2011 at 19:24
  • 3
    There's an excellent blog post here (see Example 2) that explains this in detail. Jul 11, 2013 at 18:40
  • Is there a similar approach to get a char* from an ostream?
    – rmp251
    Mar 9, 2014 at 2:51
  • 1
    See stackoverflow.com/questions/13059091/… for a slightly improved version that works with const Feb 19, 2015 at 1:21
  • what about output buffer?
    – g10guang
    Jul 4, 2018 at 6:08
14

A non deprecated solution using Boost:

#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/device/array.hpp>
using namespace boost::iostreams;

basic_array_source<char> input_source(my_ptr_to_char, byte_count);
stream<basic_array_source<char> > input_stream(input_source);

or even simpler:

#include <boost/interprocess/streams/bufferstream.hpp>
using namespace boost::interprocess;

bufferstream input_stream(my_ptr_to_char, byte_count);
11

The only (simple) portable way includes making the copy:

std::istringstream ss(std::string(buf,len));

In fact, this is likely to copy the data twice, once to create the string and once to create the istringstream. (Maybe C++11 can avoid one of the copies via a move constructor; I am not sure.)

However, if you are lucky, your C++ implementation will let you do this:

std::istringstream ss;
ss.rdbuf()->pubsetbuf(buf,len);

Under GNU C++ (and, I believe, some other implementations), this will create the stringstream without copying the data. But this is "implementation-defined" behavior according to the spec. (See also this question.)

By including the len parameter, you ensure that both of these will have no problem with null characters.

The only portable way to do what you want is to implement your own subclass of stringbuf and use it to initialize the stringstream. Not for the faint of heart.

1
  • 1
    So istringstream copies the input string instead of its reference? P.S. if you implement your own streambuf then you don't need istringstream; istream will work. Oct 16, 2011 at 2:46
8

I needed a solution that supports tellg and seekg and didn't require boost.

char_array_buffer from A beginner's guide to writing a custom stream buffer (std::streambuf) gave a got starting point.

byte_array_buffer.h:

#include <cstdio>
#include <string>
#include <list>
#include <fstream>
#include <iostream>

//
// http://www.mr-edd.co.uk/blog/beginners_guide_streambuf
//

class byte_array_buffer : public std::streambuf
{
public:
    byte_array_buffer(const uint8_t *begin, const size_t size);

private:
    int_type underflow();
    int_type uflow();
    int_type pbackfail(int_type ch);
    std::streamsize showmanyc();
    std::streampos seekoff ( std::streamoff off, std::ios_base::seekdir way,
                            std::ios_base::openmode which = std::ios_base::in | std::ios_base::out );
    std::streampos seekpos ( std::streampos sp,
                            std::ios_base::openmode which = std::ios_base::in | std::ios_base::out);

    // copy ctor and assignment not implemented;
    // copying not allowed
    byte_array_buffer(const byte_array_buffer &);
    byte_array_buffer &operator= (const byte_array_buffer &);

private:
    const uint8_t * const begin_;
    const uint8_t * const end_;
    const uint8_t * current_;
};

byte_array_buffer.cpp:

#include "byte_array_buffer.h"

#include <cassert>


byte_array_buffer::byte_array_buffer(const uint8_t *begin, const size_t size) :
begin_(begin),
end_(begin + size),
current_(begin_)
{
    assert(std::less_equal<const uint8_t *>()(begin_, end_));
}

byte_array_buffer::int_type byte_array_buffer::underflow()
{
    if (current_ == end_)
        return traits_type::eof();

    return traits_type::to_int_type(*current_);
}

byte_array_buffer::int_type byte_array_buffer::uflow()
{
    if (current_ == end_)
        return traits_type::eof();

    return traits_type::to_int_type(*current_++);
}

byte_array_buffer::int_type byte_array_buffer::pbackfail(int_type ch)
{
    if (current_ == begin_ || (ch != traits_type::eof() && ch != current_[-1]))
        return traits_type::eof();

    return traits_type::to_int_type(*--current_);
}

std::streamsize byte_array_buffer::showmanyc()
{
    assert(std::less_equal<const uint8_t *>()(current_, end_));
    return end_ - current_;
}


std::streampos byte_array_buffer::seekoff ( std::streamoff off, std::ios_base::seekdir way,
                                           std::ios_base::openmode which )
{
    if (way == std::ios_base::beg)
    {
        current_ = begin_ + off;
    }
    else if (way == std::ios_base::cur)
    {
        current_ += off;
    }
    else if (way == std::ios_base::end)
    {
        current_ = end_ + off;
    }

    if (current_ < begin_ || current_ > end_)
        return -1;


    return current_ - begin_;
}

std::streampos byte_array_buffer::seekpos ( std::streampos sp,
                                           std::ios_base::openmode which )
{
    current_ = begin_ + sp;

    if (current_ < begin_ || current_ > end_)
        return -1;

    return current_ - begin_;
}
7
  • That seems to be wrong (in seekof()): else if (way == std::ios_base::end) { current_ = end_; } I think it needs to be: else if (way == std::ios_base::end) { current_ = end_ + off; } ` Apr 21, 2016 at 20:12
  • Thx Marco. Could explain this in more detail? (Sorry it's really early in the morning and I haven't touched that code in a year).
    – catlan
    Apr 22, 2016 at 6:35
  • Did that code work for you? I'm trying to do exactly the same thing, the documentation of streambuf to me looks as it might be more complicated than that. In regards to why: parameter specification for seekoff says: off := Offset value, relative to the way parameter. In other words, seekoff( offset, end, ... ) shall position the _current pointer RELATIVE to end, not to end. seekoff( -5, end, ... ) shall position 5 elements before the end, for example ... Apr 24, 2016 at 19:25
  • Yes it is working for me. But it is possible that it is not covering all cases ... a test case would help but I don't have the time to write one.
    – catlan
    Apr 27, 2016 at 7:17
  • @catlan: I think Marco Freudenberger is correct - end means seek relative to the end. I.e. end + offset.
    – Timmmm
    Jul 12, 2019 at 17:32
5

An extension to the accepted answer that supports tellg and seekg:

struct membuf : std::streambuf
{
    membuf(char* begin, char* end)
    {
        this->setg(begin, begin, end);
    }

    pos_type seekoff(off_type off, std::ios_base::seekdir dir, std::ios_base::openmode which = std::ios_base::in) override
    {
        if (dir == std::ios_base::cur)
            gbump(off);
        else if (dir == std::ios_base::end)
            setg(eback(), egptr() + off, egptr());
        else if (dir == std::ios_base::beg)
            setg(eback(), eback() + off, egptr());
        return gptr() - eback();
    }

    pos_type seekpos(pos_type sp, std::ios_base::openmode which) override
    {
        return seekoff(sp - pos_type(off_type(0)), std::ios_base::beg, which);
    }
};

Usage of this class stays the same.

2

Have you tried std::istrstream? http://stdcxx.apache.org/doc/stdlibref/istrstream.html

Technically, I think that it is deprecated, but still part of the standard.

2
  • 1
    It's been deprecated since 1998, and it's highly unlikely to be removed any time soon (since it addresses the OP's common problem), but it causes annoying "deprecation" warnings on pretty much all compilers these days, which is a good enough reason to want something else. Nov 22, 2013 at 19:23
  • With C++23 you may use std::spanstream, which is practically the same but not deprecated
    – i cant
    Oct 28, 2022 at 10:21
-2

Try the Boost.Iostreams array source and sink classes.

http://www.boost.org/doc/libs/1_47_0/libs/iostreams/doc/index.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.