Skip to main content
Filter by
Sorted by
Tagged with
4 votes
1 answer
7k views

Error: Invalid value "sasl_ssl" for configuration property "security.protocol"

I'm using node-rdkafka to connect to IBM MessageHub with the following options: var options = { // 'debug': 'all', 'metadata.broker.list': brokers, 'security.protocol': 'sasl_ssl', 'ssl.ca....
B.Tong's user avatar
  • 53
1 vote
2 answers
337 views

Message Hub & Confluent Kafka Connect S3

I have requirement to consume messages from IBM MHub topic into IBM Object Storage. I got it working with local Kafka server with Confluent Kafka Connect S3 plugin as standalone worker for sink ...
Jarkko Turpeinen's user avatar
5 votes
1 answer
4k views

How to use kafkacat with message-hub

In trying to use kafkacat with message hub, I've used the following: kafkacat -X client.id=xxxxx \ -X sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="xxxx"...
Zach's user avatar
  • 83
0 votes
1 answer
63 views

Error on creating Message Hub service

I got the following error while creating IBM Message Hub service. Could you please help to check what causes this error? Service broker error: {"description"=>"Error getting account info with org ...
Zhe Sun's user avatar
-1 votes
1 answer
730 views

Configuring Apache NiFi for sending messages to Message Hub

What properties do I need to set to start sending messages from Apache NiFi PublishKafka processor to Message Hub in IBM Cloud? I have entered these values for the reference I am doing this on ...
yash agarwal's user avatar
3 votes
1 answer
2k views

Kafka Streams cast to string issues with KTable when grouping and aggregating

I have a Kafka stream with incoming messages that looks like sensor_code: x, time: 1526978768, address: Y I want to create a KTable that stores each unique address at each sensor code. KTable ...
jawwe's user avatar
  • 658
7 votes
1 answer
1k views

Kafka Streams KTable configuration error on Message Hub

This issue is now solved on Message Hub I am having some trouble creating a KTable in Kafka. I am new to Kafka, which is probably the root of my problem, but I thought I could ask here anyway. I have ...
jawwe's user avatar
  • 658
-2 votes
1 answer
4k views

How to implement camel kafka producer and consumer which is using kafka brokers and user name and password but not ssl certs

How to implement camel kafka producer and consumer which is using kafka brokers and user name and password but not ssl certs. my kafka broker configurations look like this { "kafka_brokers_sasl"...
suri's user avatar
  • 41
2 votes
2 answers
264 views

MirrorMaker Kafka 0.9 connection to Kafka Brokers 0.10 (IBM Message Hub)

I am trying to connect my MirroMaker Kafka 0.9 to the Kafka Brokers 0.10 (IBM Message Hub) without success. The links I have followed are the followings, but they are mostly for Kafka clients 0.10: ...
Edoardo Basili's user avatar
2 votes
1 answer
345 views

Retrieve messages from a topic on a message hub

I am trying to get messages from a topic on a message hub on bluemix using Confluent Kafka Python. My code is found below, but something is not working. The topic and the message hub is up and running,...
jawwe's user avatar
  • 658
12 votes
0 answers
9k views

Failed to send SSL Close message

I have a thread which once in a while is going to list the topics on the Message Hub. But once in a while, I am getting a :Failed to send SSL Close message. Any ideas? KafkaConsumer<String, ...
John Chu's user avatar
  • 121
2 votes
3 answers
9k views

Kafka java.io.EOFException - NetworkReceive.readFromReadableChannel

I'm trying to connect to IBM Message Hub from Apache Spark 2.2.1 Structured Streaming. The connection code is quite basic: import org.apache.spark.sql.functions._ import org.apache.spark.sql....
Chris Snow's user avatar
  • 24.4k
1 vote
1 answer
79 views

Solaris MQ accessing Message Hub in Bluemix via secure gateway

I'm testing to bridge messages from WMQ in Solaris to Message Hub - Bluemix and vice versa using Secure Gateway. Since secure gateway client is not available for Solaris , what is the option for ...
Vignesh's user avatar
  • 949
1 vote
1 answer
79 views

Creating a Message-Hub Bridge for IBM Cloud Object Storage

I'm trying to create a "Bridge" from Message Hub to S3 Object Storage, copying information from the credentials that I created but I always get an error that says "Please trying refreshing the page, ...
Daniel Alvarez-Torres's user avatar
0 votes
1 answer
4k views

Kafka Consumer Not Receiving Messages When Subscribed to 3+ Topics

I'm using node-rdkafka (https://github.com/Blizzard/node-rdkafka) to consume and produce messages from an IBM Bluemix Message Hub. I'm able to consume and produce messages without issue, as long as I ...
alreit's user avatar
  • 322
1 vote
1 answer
342 views

How do you connect NIFI to IBM MessageHub through PublishKafka processor?

I am trying to connect NIFI to IBM MessageHub but I am not getting any connection working. Does anyone have a working example on how to configure it? I get a timeout exception in the bulletinboard. ...
OlaSon's user avatar
  • 13
0 votes
1 answer
898 views

Unrecoverable error with kafka input plugin connecting to IBM Message Hub

I'm trying to use IBM Message Hub from logstash. My logstash.conf: input { kafka { bootstrap_servers => "kafka04-prod01.messagehub.services.eu-de.bluemix.net:9093,kafka05-prod01....
Chris Snow's user avatar
  • 24.4k
0 votes
0 answers
843 views

Failed while executing StreamTask 0_1 due to flush state:

I'm trying to run a basic pipe from one topic to another using Kafka Streams 0.10.2.1 KStream<ByteBuffer, ByteBuffer> stream = builder .stream("transactions_load"); stream....
Chris Snow's user avatar
  • 24.4k
0 votes
0 answers
1k views

DataStage Kafka connector / Message Hub / Authentication via Kerberos

This is not really a programming question - rather one about different implementation options. I'm trying to stream data to a Message Hub on Bluemix using the Kafka connector in DataStage. The ...
Jan's user avatar
  • 1
2 votes
1 answer
261 views

IBM Message Hub - after hours of working fine it fails with SASL Handshake not supported by broker (required by mechanism PLAIN)

The following application is a simple consumer which prints all messages to the console. #!/usr/bin/env python import confluent_kafka consumer = confluent_kafka.Consumer({ 'bootstrap.servers': '...
Christian Will's user avatar
1 vote
1 answer
3k views

I see errors using node-rdkafka but it seems to be working ok

I have a Bluemix Node.js (6.1.0) application that uses node-rdkafka 1.0.3. It seems to be working ok but there are tons of error events like Error: Local: Broker Transport Failure or Error: Local: ...
Daniel Alvarez-Torres's user avatar
3 votes
1 answer
223 views

UI console to browse topics on Message Hub

I have a Message Hub instance on Bluemix, and am able to produce / consume messages off it. I was looking for a quick, reasonable way to browse topics / messages to see what's going on. Something ...
Deven's user avatar
  • 166
0 votes
1 answer
71 views

Does Node Red on Bluemix allow for a connection to MessageHub MQLight topic?

I have created a MessageHub service in Bluemix and bound a Node Red application, also in Bluemix. In MH I added an MQLight topic. When I try to connect the mqlight node provided in Node Red it says ...
Jon Dockter's user avatar
3 votes
2 answers
9k views

Kafka Recover from Commit Failed Exception

I have an issue where my commit fails because poll() is too long (why this happens I don't know, there was no messages and it was simply read/committing on an empty queue, and my poll-interval is set ...
kyl's user avatar
  • 559
0 votes
1 answer
95 views

OpenWhisk messaging package - messageHubProduce unstable

I'm trying to use the "whisk.system/messaging" and trying to use the method messageHubProduce. I created a bind to this package, and tried to use a simple call with postman. Using the documentation, ...
Mike Moreira's user avatar
-2 votes
1 answer
41 views

unable to obtain topic data error in ibm message hub

suddenly getting error on ibm message hub. "unable to get topic data". Everything was working fine till couple of hours ago. Checked: 1. Usage. Everything seems fine.
nova iot's user avatar
0 votes
2 answers
160 views

Connection to IBM message hub using C# and Confluent

I'm trying to create producer and consumer apps using IBM message hub. For the producer i'm using the following code: var config = new Dictionary<string, object> { { "bootstrap....
israel altar's user avatar
  • 1,794
0 votes
1 answer
271 views

IBM message hub communciation with C# confluent api

I have been trying to find documentation to connect to IBM Message Hub through C# Apache Kafka Confluent API, but have not been successful. The official repo on github does not have sample for C#. Has ...
Suchit Patel's user avatar
0 votes
1 answer
5k views

Kafka Error For Topics I'm Not Subscribed To

I saw a kafka warning for topics my current instance is not subscribed to. We use the same kafka broker, and each dev has his/her own topic. However we all use the same default groupID/clientid ...
kyl's user avatar
  • 559
1 vote
1 answer
78 views

Can't Connect to BlueMix MessageHub From Bluemix Liberty

I have a liberty app that connects Bluemix Message Hub. It works fine locally, however it never works when I run the liberty app from Bluemix Liberty. I'm following the example in https://github.com/...
kyl's user avatar
  • 559
17 votes
2 answers
19k views

Consumer Stuck in Re-join

I've read other threads and I've gotten around the problem by using a new group ID, however I'd like to understand what could cause this. I have a topic with 16 partitions, I've set session.timeout....
kyl's user avatar
  • 559
0 votes
1 answer
538 views

Kafka Consumer Re-reading Messages

I've seen an issue where all my messages in my topic gets re-read by my consumer. I only have 1 consumer, and I turn it on/off while I'm developing/testing. I notice that sometimes after days of not ...
kyl's user avatar
  • 559
0 votes
1 answer
74 views

How do I set up a 'long poll' using Bluemix Message Hub in NodeJS

This question is related to why do I need to poll message hub, The documentation linked with that answer shows that Kafka will support a 'long-poll' concept, but there is no clear way in the existing ...
Bob Dill's user avatar
  • 1,010
0 votes
2 answers
89 views

why do I need to poll message hub?

I am looking at the node-js express example for message-hub on blue mix and am puzzled for why I need to poll message hub from my server. I thought that the whole idea behind a pub-sub model is that I ...
Bob Dill's user avatar
  • 1,010
0 votes
1 answer
179 views

How To Authenticate To IBM Message Hub Without Jass Login Module

I have tried setting up a connection to kafka-message-hub on Bluemix using a Liberty for Java app. I followed the docs and the git repository example. It creates/uses a jaas.conf file fine and says it ...
kyl's user avatar
  • 559
0 votes
1 answer
555 views

Kafka Golang Client fails to connect to Message Hub Service

This is a code snippet of what I'm running: err := godotenv.Load() if err != nil { log.Fatal("Error loading .env file") } broker := os.Getenv("BROKER") topic := os.Getenv("TOPIC") username := ...
Stephanie Reaves's user avatar
0 votes
1 answer
3k views

kafka python - Bluemix MessageHub - ConnectionError: socket disconnected

I'm using the kafka python client to push messages to Message Hub, but noticed that after a while of running my app that it would stop sending messages to Message Hub. I then noticed the following in ...
Chris Snow's user avatar
  • 24.4k
1 vote
1 answer
3k views

Apache Kafka System Error Handling

We are trying to implement Kafka as our message broker solution. We are deploying our Spring Boot microservices in IBM BLuemix, whose internal message broker implementation is Kafka version 0.10. ...
Tatha's user avatar
  • 1,293
0 votes
0 answers
72 views

Can we create a listener(trigger) inside a message hub?

I am trying to connect message hub and apache kafka. I have created the trigger in openwhisk, but I want to know if we can create the trigger in message hub itself.
Shreya's user avatar
  • 11
0 votes
1 answer
70 views

How to customize Service credentials in Message Hub

When we create a message hub instance in bluemix, we can see a service credentials tab on the screen. The data is in json format and not editable: "user": "xxxxxxxxxxxxxxxx", "password": "...
Shreya's user avatar
  • 11
1 vote
1 answer
90 views

How does IBM Message Hub count and bill for partitions created

According to Bluemix, IBM Message hub is billed based on (a) "number of partitions created (calculated daily)" and (b) the number of messages sent. Each partition created, it says, costs $10. In our ...
Mike Gorman's user avatar
5 votes
1 answer
3k views

Kafka enable.auto.commit set to false but poll still fetch "next" messages

I want to tell Kafka when my consumer has successfully processed a record so I have turned auto-commit off by settting enable.auto.commit to false. I have two messages on a topic I am subscribed to ...
Mike Gorman's user avatar
0 votes
1 answer
260 views

Why do I get a 403 Unauthorized when I try to create a topic on IBM Message Hub

I created a new instance of IBM Message Hub on Bluemix and can use the Kafka Java client to connect to it (from my laptop) and list topics. What I cannot do is create a topic via the administrative ...
Mike Gorman's user avatar
3 votes
1 answer
319 views

Create topic in Message-Hub via curl command

I am trying to create a topic in Message-Hub using a curl command. I followed this yaml: https://github.com/ibm-messaging/message-hub-docs/blob/master/kafka-administration-api/KafkaTopicManagement....
jd76's user avatar
  • 117
1 vote
2 answers
541 views

Bluemix Message Hub - Failed to initialize SASL authentication yet still seems to work

When I try to use the Bluemix Message Hub with Python, I keep receiving errors and failures. Any ideas why given the below? The errors are "Failed to initialize SASL authentication: SASL Handshake ...
Aaron B's user avatar
  • 173
0 votes
1 answer
149 views

Bluemix message hub ACLs

I am using Bluemix message hub service. I need to provide read only access to specific topics to specific users. Question is how can I define ACLs in Bluemix message hub? Apache kafka does provide the ...
jeet's user avatar
  • 629
1 vote
1 answer
85 views

Speeding up Message Hub Kafka Java Console sample

I've been working with the Message Hub sample code found at this link: https://github.com/ibm-messaging/message-hub-samples In particular, I've been trying to increase the throughput of the producer ...
Taylor Allison's user avatar
2 votes
1 answer
6k views

Why am I getting the Kafka Error Failed to initialize SASL authentication: SASL handshake failed when using Node.js client for Message Hub service ?

I am getting the ERROR Failed to initialize SASL authentication: SASL handshake failed (start (-4)): SASL(-4): no mechanism available: No worthy mechs found when trying to use the Message Hub Bluemix ...
ValerieLampkin's user avatar
1 vote
1 answer
4k views

Kafka Error INVALID_ARG No Such configuration property sasl.mechanisms when using confluent-kafka-python

When trying to use the Message Hub Bluemix service with confluent-kafka-python, I am getting the ERROR Uncaught exception: , KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration ...
ValerieLampkin's user avatar
3 votes
1 answer
4k views

Why Kafka REST Proxy API not Consuming data in JSON Format?

I'm try to access message hub instance by using kafka rest API. when i'm sending post request through Postman Client and giving Content-Type as application/vnd.kafka.json.v1+json it's giving me the ...
satyam bansal's user avatar