17

I managed to integrate the boost Iostream APIs for reading zipped files. I followed the documentation in boost page and have the following code so-far:

std::stringstream outStr;  
ifstream file("file.gz", ios_base::in | ios_base::binary);  
try {  
    boost::iostreams::filtering_istreambuf in;  
    in.push(boost::iostreams::gzip_decompressor());  
    in.push(file);  
    boost::iostreams::copy(in, outStr);  
}  
catch(const boost::iostreams::gzip_error& exception) {  
    int error = exception.error();  
    if (error == boost::iostreams::gzip::zlib_error) {  
       //check for all error code    
    }   
}  

The code works fine (so please ignore any typos. and errors above :)).

  1. Looks like the above code will read the complete the file and store it in the memory while creating the filtering_istreambuf. Is that true, from my investigation it looks so to me? If the file is read into memory, this code can be an issue for large files (which is what I'm dealing with).
  2. My current code reads the gzipped using gzgets API from zlib line by line. Is there a way to do line by line reading using boost APIs?

1 Answer 1

26

1) Yes, the above code will copy() the entire file into the string buffer outStr. According to the description of copy

The function template copy reads data from a given model of Source and writes it to a given model of Sink until the end of stream is reached.

2) switch from filtering_istreambuf to filtering_istream and std::getline() will work:

#include <iostream>
#include <fstream>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp>
int main()
{
    std::ifstream file("file.gz", std::ios_base::in | std::ios_base::binary);
    try {
        boost::iostreams::filtering_istream in;
        in.push(boost::iostreams::gzip_decompressor());
        in.push(file);
        for(std::string str; std::getline(in, str); )
        {
            std::cout << "Processed line " << str << '\n';
        }
    }
    catch(const boost::iostreams::gzip_error& e) {
         std::cout << e.what() << '\n';
    }
}

(you can std::cout << file.tellg() << '\n'; inside that loop if you want proof. It will increase in sizeable chunks, but it won't be equal the length of the file from the start)

2
  • thanks. Let me try this out. I'm hoping to store < in> as a class member. and have a member function in the class called getline. getline should be able to return line from current file pointer position.
    – cppcoder
    Jun 21, 2011 at 6:17
  • I tried printing the stream position inside the loop and it is not working. file.tellg() returns a constant number if I use std::ifstream and 0 if I use boost::iostream::file_source. It returns -1 if I call it on the in stream. How can I get current position in file and be able to move to that position? Do I have to first dump all of the stream into another stream?
    – pooya13
    Jun 8, 2020 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.