Questions tagged [scalapb]
ScalaPB is a protocol buffer compiler (protoc) plugin for Scala. It generates Scala case classes, parsers and serializers for protocol buffers.
105
questions
0
votes
2
answers
104
views
Unmarshalling protobufs in Scala
I have previously got Unmarshalling working in Scala using ScalaPB and the following marshallers:
implicit def marshaller[T <: GeneratedMessage]: ToEntityMarshaller[T] = ...
0
votes
0
answers
45
views
Classes code generated from protobuf file not found in Scalapb
I'm new to Scalapb and currently I'm trying to compile a proto file to Scala classes as below
build.sbt (made by George Leung / phiSgr - run normally on his demo project):
// the enterprisePackage ...
0
votes
0
answers
39
views
ScalaPb annotations on sealed trait
I am trying to put an annotation on top of a generated sealed trait using scalaPb this way:
message SealedTrait {
option (scalapb.message).annotations ="@zio.json.jsonDiscriminator("...
0
votes
0
answers
37
views
Is there a way to prevent scalapb auto-generation for certain gRPC endpoints in a service?
We have a large gRPC service that contains many endpoints. When we try to add a new endpoint, compilation of the scala code generated by scalapb fails, because the bindService method in the service ...
1
vote
0
answers
32
views
How to compile the protofile
need a help, I don't have idea about how to compile for proto useing scalapb
syntax = "proto2";
package com.test;
import "ImageElement.proto";
import "Color.proto";
...
0
votes
2
answers
133
views
How to create a jar file for Gatling gRPC Load test with only test class
I am using the code from https://github.com/alexromanov/gatling-grpc-tests-sample
i have added new protobuf under src/main and added a newSimulation.scala under src/test/scala/load. The code works ...
0
votes
1
answer
209
views
Scala Protocol Buffer and Maven issues
My requirment is use Protocol Buffer on the Scala Play! REST API project and I'm having a hard time adding it. I followed the instructions in the documentation but it keeps failing. This is how it ...
0
votes
1
answer
169
views
Map a Scala code generated by the Protocol Buffer Compiler from a .proto file to a simple scala case class
Is there a library out there to map a Scala code generated by the Protocol Buffer Compiler from a .proto file to a simple scala case class. The big file doesnt have a mapper fiunction
Example of ...
0
votes
1
answer
330
views
How to parse arbitrary protobuf message in scala usind scalapb
According to the documentation scalapb is able to parse arbitrary protobuf messages into json format. However, I cannot make heads or tails of their documentation and my code simply does not compile:
/...
0
votes
0
answers
361
views
java.nio.channels.UnsupportedAddressTypeException
I'm building a test gRPC server with Mill using ScalaPBModule. I can successfully build (with server.assembly) and start the server, but when it is hit with a request, the client throws this error:
...
1
vote
1
answer
184
views
Is there a way to get all possible protobuf Enum values using scalapb?
I have a protobuf enum like
enum MyEnum {
A = 0;
B = 1;
C = 2;
}
At runtime, I want to loop over all possible values for the enum like this:
MyEnum().uniqueValues.forEach(println)
How might I do ...
0
votes
1
answer
499
views
sbt-protoc error File does not reside within any path specified using --proto_path (or -I).You must specify a --proto_path which encompasses this file
I've scala grpc protobuf project with below build.sbt.
ThisBuild / version := "0.1.0-SNAPSHOT"
val openRtbCoreVersion = "1.5.5"
val googleCommonProtosVersion = "1.12.0"
...
-1
votes
1
answer
113
views
How to avoid a copy of array when parsing a ByteString with ScalaPB
Given bs, an instance of com.google.protobuf.ByteString.
What is the best way to parse from it
I have tried
Foo.parseFrom(bs.toByteArray)
Foo.parseFrom(bs.newInput)
Surprisingly, a JMH benchmark ...
0
votes
0
answers
296
views
Converting List[Map[String, Any]] to Option[com.google.protobuf.struct.Struct] using Scalapb
I am trying to convert List[Map[String, Any] to Option[com.google.protobuf.struct.Struct] using Scalapb:
Proto:
message IOTDecodeRequest {
string id = 1;
string user_id = 2;
bytes raw_data = 3;
}...
0
votes
0
answers
306
views
sbt scala pb "Protobufs files found, but PB.targets is empty."
I'm having project which converts proto files into scala case classes.
My build.sbt is -
ThisBuild / version := "0.1.0-SNAPSHOT"
lazy val root = (project in file("."))
.settings(...
0
votes
0
answers
335
views
Protobuf: strange transformation int64 to Long
The protobuf-file has the following declaration of method:
message GetRequest {
int64 client_id = 1;
google.protobuf.StringValue name = 2;
}
It generate:
final case class GetRequest(
...
0
votes
1
answer
300
views
ScalaPb sbt: Import "data/common/num.proto" was not found or had errors
My project structure is:
logs
- data
- pubs
- invent.proto
- common
- num.proto
NOTE - The .proto files are not under src/main/protobuf. They're under a directory called logs ...
0
votes
1
answer
642
views
How to share .proto (protobuf) files using a shared scala library using sbt
I have a few apps using shared .proto files. Each app's repo currently contains a copy of the files, which is not ideal and has recently created a problem when they accidentally diverged.
I would like ...
-1
votes
3
answers
1k
views
Import "google/protobuf/wrappers.proto" was not found scala
I have a .proto file which Imports google/protobuf/wrappers.proto
while I run Scalapbc to generate the relevant scala code out of it it gives Import google/protobuf/wrappers.proto not found error.
as ...
0
votes
1
answer
92
views
GRPC - Separate proto serialization cpu stage from IO stage
We have a service which sends grpc requests under heavy load.
Normally, grpc send is considered to be an IO stage, which means it should use an IO thread pool.
We've noticed that the serialization ...
1
vote
0
answers
262
views
java.lang.VerifyError: class frameless.functions.Spark2_4_LambdaVariable overrides final method genCode
I am running a databricks job that uses ProtoSQL.udf.
The data frame has a column that has encoded protoValue. The parse method is used to create a new column which stores json extracted from ...
0
votes
1
answer
2k
views
convert ByteArray to String to ByteArray
I want to convert ByteArray to string and then convert the string to ByteArray,But while converting values changed. someone help to solve this problem.
person.proto:
syntax = "proto3";
...
0
votes
2
answers
1k
views
How to generate any protobuf Enum as String with only one TypeMapper?
For instance, I have some enums in my proto schema:
enum E1 {
UNKNOWN = 0;
OPTION_1 = 1;
OPTION_2 = 2;
OPTION_3 = 3;
}
enum E2 {
UNKNOWN = 0;
ANOTHER_OPTION_1 = 1;
...
0
votes
2
answers
375
views
zio-grpc with Scala 3 (2nd try)
Does anyone have a bare-bones zio-grpc server, with codegen in the project also, working with Scala 3?
I started with the HelloWorld project from their repo and attempted to get it to build with ...
0
votes
0
answers
94
views
Synchronizing clients and server
I'm making a distributed sorting system, where each machine (client) has a specific range of values (partition) to store. I don't have any problem sending and receiving data, but the synchronization ...
2
votes
1
answer
3k
views
object is not a member of package
Im trying to implement the simple server-client application using scalaPB's official example. The scala code is found on their gitHub
However, when I try to run it, I keep getting the error object ...
1
vote
1
answer
332
views
How to describe Option[FiniteDuration] in proto with scalapb
I am trying to use proto3 and scalapb, but I am unable to map FiniteDuration as well as unable to use it as Option. Can anyone please advise on the same
case class Message(id:Int , interval: Option[...
0
votes
2
answers
712
views
how to change the scalaPB default path of protobuf files?
I have a build.sbt for a couple of packages in the same project.
When I do sbt compile, my protobuf files are under [packagename]/src/main/protobuf/*,
so it won't scan these files since they have that ...
0
votes
1
answer
510
views
Scala scalapb GeneratedMessageCompanion- do not conform to trait GeneratedMessageCompanion's
I am trying to generate Scala proto buff classes using Scalapb (https://scalapb.github.io/docs/scalapbc/). I am able to generate scala files but getting below error.
type arguments [com.huawei.utility....
0
votes
1
answer
354
views
ScalaPB: compilePlugin generate inappropriate class
I'm trying to use Protocol Buffers for generate Scala case classes.
Base settings are:
1.In project/plugins.sbt:
addSbtPlugin("com.thesamet" % "sbt-protoc" % "1.0.0")
...
1
vote
1
answer
1k
views
How to generically unpack google.protobuf.Any on Spark with ScalaPB at runtime?
I have the following protobuf setup:
import "google/protobuf/any.proto";
message EntityEnvelope {
string id = 1;
string entity_type = 2;
google.protobuf.Any entity = 3;
}
message ...
0
votes
1
answer
317
views
Using Scalapb with spark structured streaming and java generated protobuf classes
The project I'm working on is in the planning/prototyping phase and we would like to stream our data into spark 3 using protobuf encoded messages in Kafka and structured streaming. We've prototyped ...
0
votes
1
answer
521
views
Set file-level option to scalapb project
I'm using ScalaPB (version 0.11.1) and plugin sbt-protoc (version 1.0.3) to try to compile an old project with ProtocolBuffers in Scala 2.12. Reading the documentation, I want to set the file property ...
0
votes
1
answer
505
views
Spark unmarshal proto bytes into readable format
My Spark application receives binary data from Kafka. The data is sent as a byte proto message to Kafka. The proto message is:
message Any {
string type_url=1;
bytes value=2;
}
With the ScalaPB ...
0
votes
1
answer
441
views
How to define Compile settings in sbt multi-project?
I'm trying to configure a new Scala build using sbt multi-project. I want to use sbt-protoc and in particular ScalaPB which requires this configuration setting:
Compile / PB.targets := Seq(
scalapb....
0
votes
1
answer
231
views
How to generate validators file only for message that contains validate rules?
How to generate validator file only for message that contains validates rules ?
In the example below, actually sbt compilation generates 4 scala classes: one for protobufA, one for protobufB and one ...
0
votes
0
answers
597
views
How to solve problem with compile protobuf in scala project on Windows10?
I have simple code with example protobuf on scala
proto file
syntax = "proto3";
package grpc.example;
message HelloRequest {
string msg = 1;
int32 code = 2;
}
message HelloResponse {
...
2
votes
0
answers
751
views
High latency of grpc service using grpc client in kubernetes
I have services (both writtenin scala) deployed in kubernetes cluster, lets name them as external service and internal service
External service - 5 pods, expose REST API, consume internal service ...
0
votes
1
answer
205
views
How to add "targets in compile" to a SBT project which is defined with lazy val root = project.in(".")
I have a project:
lazy val root = project
.in(file("."))
.settings( ...
I wish to add ScalaPB to the project and I need to add:
PB.targets in Compile := Seq(
scalapb.gen(grpc = true)...
0
votes
1
answer
173
views
How to define and pass implicit encoder of a particular subtype to AvroSchema
Is it possible to define and pass Encoder[E] for any subtype E (e.g. any E that extends GeneratedEnum class, in the code instance of E is Color) to AvroSchema[C] where C is some case class that ...
0
votes
1
answer
235
views
How to integrate sbt-protoc and scalapb with sbt cross build?
I have a library that needs two different versions of "com.thesamet.scalapb" %% "compilerplugin" depending on the Scala version.
In my project/scalapb.sbt I have this code:
def ...
0
votes
1
answer
168
views
SparkSQL with ScalaPB: Error while skipping proto fields while converting from DataFrame to proto dataset
I have the following proto message, that needs to be written through Spark using ScalaPB:
message EnforcementData
{
required int32 id = 1;
required int32 source = 2;
required int32 ...
0
votes
1
answer
291
views
SparkSQL with ScalaPB: Using MapType in output proto format gives scala.MatchError while calling toByteString
The following is my output message format :
message EditorialTextAdEnforcementData {
int32 customerId = 1;
int32 source = 2;
DecisionDetails decisionDetails = 3;
int32 flagsEnforceOption = 4;
...
0
votes
1
answer
248
views
Creating dataframe on proto case class with bcl.DateTime field throws none is not a term exception
I have a case class generated from a .proto file through scalapb, which has a few fields with bcl.DateTime type. The case class definition is as follows:
@SerialVersionUID(0L)
final case class ...
0
votes
0
answers
34
views
Running scalapbc command from a thread pool
I am trying to run the scalapb command from a threadpool with each thread running the scalapbc command;
When I do that, I get an error of the form :
# A fatal error has been detected by the Java ...
0
votes
1
answer
880
views
ScalaPB, proto3: set field to None, no optionalX prefix available
Using the Person and Address message definition in https://scalapb.github.io/generated-code.html
syntax = "proto3";
package trexo.scalapb;
message Person {
string name = 1;
int32 age = 2;
...
1
vote
0
answers
435
views
Unable to call method from scala protobuf jar
I have define scala object(say, MyObject) which extends following
trait GeneratedMessageCompanion[A <: GeneratedMessage with Message[A]]
And when I call parseFrom method on the object, I get ...
0
votes
0
answers
203
views
ScalaPB code generation option for metadata access in method implementation
What option can I set for ScalaPB code generation where the methods for the service implementation have access to metadata?
Default options generate something like:
override def sayHello(req: ...
0
votes
1
answer
752
views
Serializing message with protobuf for akka actor which contains serializable data
I have a persistent actor which can receive one type of command Persist(event) where event is a type of trait Event (there are numerous implementations of it). And on success, this reponds with ...
0
votes
2
answers
909
views
How to write google protobuf messages in java?
I am using google protobuf library to write messages in a file. However these messages do not look binary messages to me. The proto I am using is simple it has Name, Email and Cell No. The file I get ...