0

I have a program that serializes data with Protobuf in Java, by writing binary data in a byte[] Array and then saving it in ".txt" file. I am receiving that data on the C++ side in a stringstream. Now I want to parse that binary data with C++, but the Protobuf-Parsing-Method "parseFromString()" doesn't work! The fields from my Test Message are not set. I wrote a little test for that and I can show you some code:

Java Serialization

  byte[] s = test.build().toByteArray(); //This is serialized to "C:\test.txt" as binary

C++ Parsing:

Test t1; // My Protobuf Message
std::ifstream myFile("C:\\test.txt");
std::string s;
myFile >> s;

t1.ParseFromString(s);
std::cout << "Decoded: " << t2.s() << std::endl; // Check if parsing was correct

But it just returns: " Decoded: ", as if t2 was empty, but it shouldn't be! How can you parse binary data in C++?

2 Answers 2

5

Your problem is probably here:

myFile >> s;

The >> operator reads a text string, delimited by whitespace. An encoded protobuf is not text. Probably, ParseFromString() is returning false to indicate that it couldn't parse the data, because it is incomplete.

What you want to do is read the whole file. The easiest way to do this in your case is to use ParseFromIstream(&myFile). (And make sure to check that it returns true!)

(Another option would be to check the file size, create an array of that size, use myFile.read(array, size), and then ParseFromArray(array, size), but that's a lot more work that will do the same thing.)

Note that you probably should not use .txt as a file extension for a protobuf as the file does not contain text.

Also note that, confusingly, in C++ you can put binary (non-text) data in an std::string -- it's just a byte container -- but in Java you cannot put binary data in a String. So the C++ ParseFromString() and SerializeAsString() deal in binary data, whereas the Java toString() actually returns a textual representation of the message (meant for debugging purposes) which is not what you want to transmit.

1
  • Perfect answer, that was the problem! Thank you very much for your time :)
    – Overholt
    Sep 12, 2015 at 17:06
0

The problem is that you are mixing API calls. You're serializing with toByteArray API call and deserializing with ParseFromString API call.

You can use one of these pairs, but you can't mix them:

  • ParseFromString and SerializeToString pair
  • SerializeToArray and ParseFromArray pair

I'm programming C++ and python. I attached below example from my code. I think you should be able to understand how it is working from this example. If something is ambiguous please leave me a comment.

Here is my python serialization:

socket.send(request.SerializeToString())

and C++ deserialization:

// Deserialize request.
if (request.ParseFromString(protobuf_request) == false) {
    throw exception();
}

Here is my C++ serialization:

// Serialize response.
assert(response.SerializeToString(&protobuf_response) == true);

and python deserialization:

response.ParseFromString(str(message))

EDIT:

I think you should use:

  • void writeTo(OutputStream output); in Java
  • bool ParseFromIstream(istream* input); in C++
6
  • Hi, Thank you for your complete answer. That should be the problem for sure. I tried the ParseFromArray() API call with a string as data source in C++, but it still isn't able to parse the raw data. Which API parsing call in C++ might be suitable to load the java-generated binary?
    – Overholt
    Sep 11, 2015 at 7:08
  • [Serialization in Java] void writeTo(OutputStream output); Sep 11, 2015 at 8:55
  • [Deserialization in C++] bool ParseFromIstream(istream* input); Sep 11, 2015 at 8:55
  • This answer is wrong. In C++, ParseFromString() and ParseFromArray() parse exactly the same format; the only difference is the former expects the buffer to be contained in an std::string (which in C++ is allowed to contain binary bytes, not just text) while the latter expects a pointer and a size. You can actually call ParseFromArray(str.data(), str.size()) and it will do the right thing! Sep 11, 2015 at 21:24
  • I see. First I will double check it and then I will update my answer accordingly. Thanks! Sep 11, 2015 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.