14

I use Ubuntu server 16.04 to try using Kafka. For the command to start a producer and a consumer console I use the following.

producer console :

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic hello-topic

consumer console :

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic hello-topic

but the command above only subscribes to one topic. How can I subscribe to multiple topics?

2
  • 1
    Did you try the whitelist option of the console consumer?
    – Harald
    Commented Jan 2, 2017 at 8:39
  • @Harald not yet, i don't know there is whitelist Commented Jan 3, 2017 at 9:14

3 Answers 3

25

First you should connect with the option bootstrap-server to the Kafka server itself not the zookeeper server.

For multiple topics you use the whitelist option. This will get interpreted as a regular expression and has to get quoted, see Kafka documentation. So a correct command would be:

kafka-console-consumer.sh --bootstrap-server localhost:9092 --whitelist 'hello-topic|world-topic|another-topic'

Other expressions are also possible, like

kafka-console-consumer.sh --bootstrap-server localhost:9092 --whitelist '.*'

BE AWARE

For convenience we allow the use of ',' instead of '|' to specify a list of topics.

Does not work with Kafka 2.0, perhaps only when mirroring, which I did not try yet.

1
  • i am using kafka 2.6.0 this command is not working for me
    – MiniSu
    Commented Apr 3, 2021 at 15:12
0

Hi The actual problem lies in the syntax, from version to verison it keeps on changing.

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic muleesb

would work for you

Reason

For kafka versions above kafka_2.11-2.1.0: ./kafka-console-consumer.sh --bootstrap-server IP:PORT --from-beginning --topic TOPIC

For Kafka versions kafka_2.11_0.9.0.0 and below: /kafka-console-consumer.sh --zookeeper IP:PORT —-topic TOPIC --from-beginning --whitelist TOPIC

Exactly one of whitelist/blacklist/topic is required

-1

As mentioned by Harald, use whitelist / blacklist option to include / exclude a set of topics for consumption.

sh kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --whitelist Hello,World
3
  • This is not working for me when I use --bootstrap-server kafkaserver:9092. Since the old consumer will be deprecated, is there a way for new consumers to dot that?
    – colin
    Commented Jan 18, 2018 at 5:51
  • @colin-SBI which Kafka version are you using ? Commented Jan 18, 2018 at 6:17
  • kafka_2.11-0.11.0.1
    – colin
    Commented Jan 19, 2018 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.