1

Is it possible to read a single message from a Kafka compact topic if I know the message key?
I didn't find anything in the consumer Javadoc which looked like it would work.

1 Answer 1

3

Kafka topics aren't indexed on keys, they are partitioned by them.

Yes, you can read a single message, regardless of compaction, but you can't read a topic starting from a particular key, only an offset.

Knowing a key, you can find the partition (assuming using default partitioner).

But that is the best you can do... You must scan the entire partition to then find that offset for a key


For example, using kafkacat, print a key and the offset

kafkacat -C -b kafka:9092 -t topic-name -p 0 -f "key: %k; offset: %o\n" | grep key-value

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.