Questions tagged [spring-kafka]
The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.
4,291
questions
0
votes
0
answers
10
views
Reading traceID as B3 headers from Kafka payload using spring-kafka
I am using spring-kafka in my consumer application to read the records from kafka. The producer application is sending the B3 headers as follows.
ProducerRecord producerRecord = new ProducerRecord(...
0
votes
0
answers
3
views
KStream Joins Not working in kubernetes env but is working perfectly in local
I have a simple application that would consume two streams from topicA and topicB, convert them to KTables and then join both tables using Foreign Keys. The result of the join is then send to an ...
0
votes
0
answers
8
views
Auth against Azure Active Directory on Azure Event Hubs
I'm new to Azure Active Directory (Microsoft Entra) auth technology, and trying to authenticate this way against a event hub that will receive events from a rest API. Just to have some support, I am ...
0
votes
1
answer
19
views
ClassCast error when using org.springframework.messaging.Message with Protobuf object as kafka message
I am using spring-kafka with producer property as below
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: io.confluent.kafka.serializers.protobuf....
0
votes
1
answer
19
views
How to configure 2 Spring Kafka listeners which listen to the same topic to have their own retry and dead letter topics
In Spring Kafka I have two Kafka listeners listening to the same topic. I'd like both listeners to both have their own retry and dead letter topic so if failure occurs in one listener it doesn't get ...
-1
votes
1
answer
26
views
where should I get missing properties for creating the Kafka producer?
Could you explain where should I get missing properties in the following code?
(smth should be in the lsat line)
KafkaProperties prop = new KafkaProperties();
KafkaProperties.Producer producer = prop....
0
votes
1
answer
16
views
KafkaListener partition mapping within instance for concurrency > 1
I have a simple kafka listener annotated method
and spring.kafka.listener.concurrency = 5
My topic has 100 partitions, and there are 5 instances of the application running.
The topic has partition ...
0
votes
1
answer
27
views
Spring-Kafka "Could not obtain cluster info" when Observation is enabled
Setup:
I'm using Spring boot 3.1.5 (spring-kafka 3.0.12).
Question/Issue:
When I enabled observation for my kafka consumer/producer I am seeing constant errors by the AdminClient attempting to get the ...
0
votes
0
answers
6
views
Schema Registry with Spring cloud stream Binder and Json deserializer
I am new to Kafka and I'm trying to use schema registry and integrate with spring cloud stream binding consumer I have researched a lot and got many references but nothing really worked out. So can ...
0
votes
1
answer
29
views
Spring kafka infinite retries without retryabletopic annotation
We are using spring Kafka 3.x and using DefaultErrorHandler.
In this handler, we check if the exception is retriable then do retries till Integer.maxValue else put a message in DLT.
Whole idea is to ...
0
votes
1
answer
18
views
Multiple intercept method in RecordInterceptor Spring Framework Kafka Listener
I have a class, that implements RecordInterceptor. Reading the documentation from Spring-Kafka there is a method called intercept which takes 2 parameter, the Record and the Consumer. However for some ...
0
votes
0
answers
43
views
Evolving Avro Schema where the Record Name changes and using alias for it does not work
So I have one avro schema Which I want to evolve and using topicRecordName strategy. I am using confluent schema registry. I want to have it backward compatible and It is necessary that I change the ...
0
votes
0
answers
38
views
Kafka Producer with Spring Kafka takes too long to refresh metadata when trying to produce
We have a standard setup with Kafka, where we use @KafkaLisetener to listen to events on a topic and send a response to another topic using KafkaTemplate.
In order to increase throughput, we increased ...
2
votes
1
answer
38
views
Spring Kafka - Delayed messages consumption
I'm using spring-kafka 2.9.12 and I'd like to consume messages from a topic after a certain period of time after the message sending. To do this, I found two ways.
The first approach is as follows, ...
0
votes
1
answer
34
views
how to have repeated configurations in springboot as jar and use as dependency
We have many microservices for multiple tasks and most of them have some common functionality like log configuration, metrics configuration and kafka configuration etc.
We are planning to have a ...