Is it possible to read a single message from a Kafka compact topic if I know the message key?
I didn't find anything in the consumer Javadoc which looked like it would work.
1 Answer
Kafka topics aren't indexed on keys, they are partitioned by them.
Yes, you can read a single message, regardless of compaction, but you can't read a topic starting from a particular key, only an offset.
Knowing a key, you can find the partition (assuming using default partitioner).
But that is the best you can do... You must scan the entire partition to then find that offset for a key
For example, using kafkacat, print a key and the offset
kafkacat -C -b kafka:9092 -t topic-name -p 0 -f "key: %k; offset: %o\n" | grep key-value