97
questions
4
votes
1
answer
7k
views
Error: Invalid value "sasl_ssl" for configuration property "security.protocol"
I'm using node-rdkafka to connect to IBM MessageHub with the following options:
var options = {
// 'debug': 'all',
'metadata.broker.list': brokers,
'security.protocol': 'sasl_ssl',
'ssl.ca....
1
vote
2
answers
337
views
Message Hub & Confluent Kafka Connect S3
I have requirement to consume messages from IBM MHub topic into IBM Object Storage.
I got it working with local Kafka server with Confluent Kafka Connect S3 plugin as standalone worker for sink ...
5
votes
1
answer
4k
views
How to use kafkacat with message-hub
In trying to use kafkacat with message hub, I've used the following:
kafkacat -X client.id=xxxxx \
-X sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="xxxx"...
0
votes
1
answer
63
views
Error on creating Message Hub service
I got the following error while creating IBM Message Hub service. Could you please help to check what causes this error?
Service broker error: {"description"=>"Error getting account info with org ...
-1
votes
1
answer
730
views
Configuring Apache NiFi for sending messages to Message Hub
What properties do I need to set to start sending messages from Apache NiFi PublishKafka processor to Message Hub in IBM Cloud?
I have entered these values for the reference
I am doing this on ...
3
votes
1
answer
2k
views
Kafka Streams cast to string issues with KTable when grouping and aggregating
I have a Kafka stream with incoming messages that looks like
sensor_code: x, time: 1526978768, address: Y
I want to create a KTable that stores each unique address at each sensor code.
KTable
...
7
votes
1
answer
1k
views
Kafka Streams KTable configuration error on Message Hub
This issue is now solved on Message Hub
I am having some trouble creating a KTable in Kafka. I am new to Kafka, which is probably the root of my problem, but I thought I could ask here anyway. I have ...
-2
votes
1
answer
4k
views
How to implement camel kafka producer and consumer which is using kafka brokers and user name and password but not ssl certs
How to implement camel kafka producer and consumer which is using kafka brokers and user name and password but not ssl certs.
my kafka broker configurations look like this
{
"kafka_brokers_sasl"...
2
votes
2
answers
264
views
MirrorMaker Kafka 0.9 connection to Kafka Brokers 0.10 (IBM Message Hub)
I am trying to connect my MirroMaker Kafka 0.9 to the Kafka Brokers 0.10 (IBM Message Hub) without success. The links I have followed are the followings, but they are mostly for Kafka clients 0.10:
...
2
votes
1
answer
345
views
Retrieve messages from a topic on a message hub
I am trying to get messages from a topic on a message hub on bluemix using Confluent Kafka Python. My code is found below, but something is not working. The topic and the message hub is up and running,...
12
votes
0
answers
9k
views
Failed to send SSL Close message
I have a thread which once in a while is going to list the topics on the Message Hub. But once in a while, I am getting a :Failed to send SSL Close message.
Any ideas?
KafkaConsumer<String, ...
2
votes
3
answers
9k
views
Kafka java.io.EOFException - NetworkReceive.readFromReadableChannel
I'm trying to connect to IBM Message Hub from Apache Spark 2.2.1 Structured Streaming.
The connection code is quite basic:
import org.apache.spark.sql.functions._
import org.apache.spark.sql....
1
vote
1
answer
79
views
Solaris MQ accessing Message Hub in Bluemix via secure gateway
I'm testing to bridge messages from WMQ in Solaris to Message Hub - Bluemix and vice versa using Secure Gateway.
Since secure gateway client is not available for Solaris , what is the option for ...
1
vote
1
answer
79
views
Creating a Message-Hub Bridge for IBM Cloud Object Storage
I'm trying to create a "Bridge" from Message Hub to S3 Object Storage, copying information from the credentials that I created but I always get an error that says "Please trying refreshing the page, ...
0
votes
1
answer
4k
views
Kafka Consumer Not Receiving Messages When Subscribed to 3+ Topics
I'm using node-rdkafka (https://github.com/Blizzard/node-rdkafka) to consume and produce messages from an IBM Bluemix Message Hub. I'm able to consume and produce messages without issue, as long as I ...
1
vote
1
answer
342
views
How do you connect NIFI to IBM MessageHub through PublishKafka processor?
I am trying to connect NIFI to IBM MessageHub but I am not getting any connection working.
Does anyone have a working example on how to configure it?
I get a timeout exception in the bulletinboard. ...
0
votes
1
answer
898
views
Unrecoverable error with kafka input plugin connecting to IBM Message Hub
I'm trying to use IBM Message Hub from logstash. My logstash.conf:
input {
kafka {
bootstrap_servers => "kafka04-prod01.messagehub.services.eu-de.bluemix.net:9093,kafka05-prod01....
0
votes
0
answers
843
views
Failed while executing StreamTask 0_1 due to flush state:
I'm trying to run a basic pipe from one topic to another using Kafka Streams 0.10.2.1
KStream<ByteBuffer, ByteBuffer> stream = builder
.stream("transactions_load");
stream....
0
votes
0
answers
1k
views
DataStage Kafka connector / Message Hub / Authentication via Kerberos
This is not really a programming question - rather one about different implementation options.
I'm trying to stream data to a Message Hub on Bluemix using the Kafka connector in DataStage. The ...
2
votes
1
answer
261
views
IBM Message Hub - after hours of working fine it fails with SASL Handshake not supported by broker (required by mechanism PLAIN)
The following application is a simple consumer which prints all messages to the console.
#!/usr/bin/env python
import confluent_kafka
consumer = confluent_kafka.Consumer({
'bootstrap.servers': '...
1
vote
1
answer
3k
views
I see errors using node-rdkafka but it seems to be working ok
I have a Bluemix Node.js (6.1.0) application that uses node-rdkafka 1.0.3. It seems to be working ok but there are tons of error events like Error: Local: Broker Transport Failure or Error: Local: ...
3
votes
1
answer
223
views
UI console to browse topics on Message Hub
I have a Message Hub instance on Bluemix, and am able to produce / consume messages off it. I was looking for a quick, reasonable way to browse topics / messages to see what's going on. Something ...
0
votes
1
answer
71
views
Does Node Red on Bluemix allow for a connection to MessageHub MQLight topic?
I have created a MessageHub service in Bluemix and bound a Node Red application, also in Bluemix. In MH I added an MQLight topic. When I try to connect the mqlight node provided in Node Red it says ...
3
votes
2
answers
9k
views
Kafka Recover from Commit Failed Exception
I have an issue where my commit fails because poll() is too long (why this happens I don't know, there was no messages and it was simply read/committing on an empty queue, and my poll-interval is set ...
0
votes
1
answer
95
views
OpenWhisk messaging package - messageHubProduce unstable
I'm trying to use the "whisk.system/messaging" and trying to use the method messageHubProduce.
I created a bind to this package, and tried to use a simple call with postman.
Using the documentation, ...
-2
votes
1
answer
41
views
unable to obtain topic data error in ibm message hub
suddenly getting error on ibm message hub. "unable to get topic data". Everything was working fine till couple of hours ago.
Checked:
1. Usage. Everything seems fine.
0
votes
2
answers
160
views
Connection to IBM message hub using C# and Confluent
I'm trying to create producer and consumer apps using IBM message hub. For the producer i'm using the following code:
var config = new Dictionary<string, object> {
{ "bootstrap....
0
votes
1
answer
271
views
IBM message hub communciation with C# confluent api
I have been trying to find documentation to connect to IBM Message Hub through C# Apache Kafka Confluent API, but have not been successful. The official repo on github does not have sample for C#. Has ...
0
votes
1
answer
5k
views
Kafka Error For Topics I'm Not Subscribed To
I saw a kafka warning for topics my current instance is not subscribed to. We use the same kafka broker, and each dev has his/her own topic. However we all use the same default groupID/clientid ...
1
vote
1
answer
78
views
Can't Connect to BlueMix MessageHub From Bluemix Liberty
I have a liberty app that connects Bluemix Message Hub. It works fine locally, however it never works when I run the liberty app from Bluemix Liberty. I'm following the example in https://github.com/...
17
votes
2
answers
19k
views
Consumer Stuck in Re-join
I've read other threads and I've gotten around the problem by using a new group ID, however I'd like to understand what could cause this.
I have a topic with 16 partitions, I've set session.timeout....
0
votes
1
answer
538
views
Kafka Consumer Re-reading Messages
I've seen an issue where all my messages in my topic gets re-read by my consumer. I only have 1 consumer, and I turn it on/off while I'm developing/testing. I notice that sometimes after days of not ...
0
votes
1
answer
74
views
How do I set up a 'long poll' using Bluemix Message Hub in NodeJS
This question is related to why do I need to poll message hub, The documentation linked with that answer shows that Kafka will support a 'long-poll' concept, but there is no clear way in the existing ...
0
votes
2
answers
89
views
why do I need to poll message hub?
I am looking at the node-js express example for message-hub on blue mix and am puzzled for why I need to poll message hub from my server. I thought that the whole idea behind a pub-sub model is that I ...
0
votes
1
answer
179
views
How To Authenticate To IBM Message Hub Without Jass Login Module
I have tried setting up a connection to kafka-message-hub on Bluemix using a Liberty for Java app. I followed the docs and the git repository example. It creates/uses a jaas.conf file fine and says it ...
0
votes
1
answer
555
views
Kafka Golang Client fails to connect to Message Hub Service
This is a code snippet of what I'm running:
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
}
broker := os.Getenv("BROKER")
topic := os.Getenv("TOPIC")
username := ...
0
votes
1
answer
3k
views
kafka python - Bluemix MessageHub - ConnectionError: socket disconnected
I'm using the kafka python client to push messages to Message Hub, but noticed that after a while of running my app that it would stop sending messages to Message Hub.
I then noticed the following in ...
1
vote
1
answer
3k
views
Apache Kafka System Error Handling
We are trying to implement Kafka as our message broker solution. We are deploying our Spring Boot microservices in IBM BLuemix, whose internal message broker implementation is Kafka version 0.10. ...
0
votes
0
answers
72
views
Can we create a listener(trigger) inside a message hub?
I am trying to connect message hub and apache kafka. I have created the trigger in openwhisk, but I want to know if we can create the trigger in message hub itself.
0
votes
1
answer
70
views
How to customize Service credentials in Message Hub
When we create a message hub instance in bluemix, we can see a service credentials tab on the screen. The data is in json format and not editable:
"user": "xxxxxxxxxxxxxxxx",
"password": "...
1
vote
1
answer
90
views
How does IBM Message Hub count and bill for partitions created
According to Bluemix, IBM Message hub is billed based on (a) "number of partitions created (calculated daily)" and (b) the number of messages sent. Each partition created, it says, costs $10.
In our ...
5
votes
1
answer
3k
views
Kafka enable.auto.commit set to false but poll still fetch "next" messages
I want to tell Kafka when my consumer has successfully processed a record so I have turned auto-commit off by settting enable.auto.commit to false. I have two messages on a topic I am subscribed to ...
0
votes
1
answer
260
views
Why do I get a 403 Unauthorized when I try to create a topic on IBM Message Hub
I created a new instance of IBM Message Hub on Bluemix and can use the Kafka Java client to connect to it (from my laptop) and list topics. What I cannot do is create a topic via the administrative ...
3
votes
1
answer
319
views
Create topic in Message-Hub via curl command
I am trying to create a topic in Message-Hub using a curl command.
I followed this yaml: https://github.com/ibm-messaging/message-hub-docs/blob/master/kafka-administration-api/KafkaTopicManagement....
1
vote
2
answers
541
views
Bluemix Message Hub - Failed to initialize SASL authentication yet still seems to work
When I try to use the Bluemix Message Hub with Python, I keep receiving errors and failures. Any ideas why given the below?
The errors are "Failed to initialize SASL authentication: SASL Handshake ...
0
votes
1
answer
149
views
Bluemix message hub ACLs
I am using Bluemix message hub service. I need to provide read only access to specific topics to specific users. Question is how can I define ACLs in Bluemix message hub? Apache kafka does provide the ...
1
vote
1
answer
85
views
Speeding up Message Hub Kafka Java Console sample
I've been working with the Message Hub sample code found at this link: https://github.com/ibm-messaging/message-hub-samples
In particular, I've been trying to increase the throughput of the producer ...
2
votes
1
answer
6k
views
Why am I getting the Kafka Error Failed to initialize SASL authentication: SASL handshake failed when using Node.js client for Message Hub service ?
I am getting the ERROR Failed to initialize SASL authentication: SASL handshake failed (start (-4)): SASL(-4): no mechanism available: No worthy mechs found when trying to use the Message Hub Bluemix ...
1
vote
1
answer
4k
views
Kafka Error INVALID_ARG No Such configuration property sasl.mechanisms when using confluent-kafka-python
When trying to use the Message Hub Bluemix service with confluent-kafka-python, I am getting the ERROR Uncaught exception: , KafkaError{code=_INVALID_ARG,val=-186,str="No such configuration ...
3
votes
1
answer
4k
views
Why Kafka REST Proxy API not Consuming data in JSON Format?
I'm try to access message hub instance by using kafka rest API. when i'm sending post request through Postman Client and giving Content-Type as application/vnd.kafka.json.v1+json it's giving me the ...