I'm using protocol Buffer(proto3) from my latest project. But I encounter below problem. An encoded message generated from python and I'm trying to decode it from Scala (I'm using SPARK). You can see the actual message from "value" and its length 227 as described in console.
The error field just typical string type field. I also have tried to decode string field as "UTF-8" but it is not worked. It looks encode/decode problem. Are there some other guys who encounter similar problems?
Console output
...
value:
U
$a5a9c2bb-efd2-4ea3-ae0e-7479925a7807"kafka_producer.py*2016-12-18 10:37:25.075614*
data{"url": "some url 0", "title": "some title", "_id": "5815a37c43cfd44120e50538", "content": "some content", "tags": ["football"]}
value length: 227
Exception in thread "streaming-job-executor-0" java.lang.NoSuchMethodError: com.google.protobuf.CodedInputStream.readStringRequireUtf8()Ljava/lang/String;
at org.libero.messages.Messages$Event.<init>(Messages.java:160)
at org.libero.messages.Messages$Event.<init>(Messages.java:117)
at org.libero.messages.Messages$Event$1.parsePartialFrom(Messages.java:1564)
at org.libero.messages.Messages$Event$1.parsePartialFrom(Messages.java:1559)
at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
at org.libero.messages.Messages$EventDataProduced.<init>(Messages.java:1742)
at org.libero.messages.Messages$EventDataProduced.<init>(Messages.java:1697)
at org.libero.messages.Messages$EventDataProduced$1.parsePartialFrom(Me
...
Code fragments
....
nonEmptyMsgs.foreachRDD { msgsRDD =>
println("Trying to print each RDD: " + msgsRDD)
println("Count: " + msgsRDD.count())
val elems = msgsRDD.collect()
println("Elems: " + elems)
for (v <- elems) {
println("key: " + v._1)
println("value: " + v._2)
println("value length: " + v._2.length())
val bytes = new String(v._2).getBytes()
val event = EventDataProduced.parseFrom(ByteString.copyFrom(bytes))
println("event: " + event)
}
}
...