2,033
questions
0
votes
0
answers
8
views
How to connect single NodeJS Azure Function Event Hub trigger to connect to multiple event hubs present within the same event hub namespace?
I have a single function in an Azure Function App written in NodeJS which is an Event Hub Trigger and is used to forward logs to datadog. The logs are generated as events to event hub then getting ...
0
votes
1
answer
29
views
Azure Function EventHub output binding to export function response data
I have a Java Azure Function which has an input EventHub binding and an output EventHub binding as follows:
@FunctionName("test-output-function-request")
@EventHubOutput(
name = &...
-1
votes
0
answers
31
views
Trying to override Bean definition not working
I am including a library in my spring boot service. The library is included as a gradle dependency and my spring boot service uses the library to connect and consume msgs in Azure Event Hubs.
The ...
0
votes
1
answer
61
views
Cannot connect to Azure Event Hubs Emulator from C# .NET application
I'm trying to connect from C# .NET application to Azure Event Hubs Emulator, using real Azure Blob on Azure (before I also switch to Azurite). I have the following docker-compose:
eventhubs:
...
0
votes
0
answers
13
views
Azure Event Hub / Confluent SDK
I'm using the Confluent SDK from an existing application (C#), and moving over to the Azure Event Hub Kafka endpoint. I noticed that the Consumer Group, as defined within the constructor parameters, ...
0
votes
0
answers
68
views
I'm getting InvalidPidMappingException without using transactional producers with Spring Boot and Spring Kafka
Repository in GitHub
Hello! I hope you are doing great!
I'm working in a kubernetes environment, with 3 pods using Spring Boot and Kotlin, and we are using Apache Kafka with Azure Event Hubs. In the ...
0
votes
1
answer
41
views
Spring Kafka with Azure Event Hub- could not be established. Broker may not be available
I am trying to send messages using spring cloud kafka stream to Azure Event Hub.
Azure Example
The reference guide by spring (mentioned in azure documentation) is binder
Error:
[AdminClient clientId=...
0
votes
1
answer
46
views
How would I consume Azure Event Hubs without any checkpointing?
I want to publish events through an Azure Event Hub. For example stock quotes. I have an application that is interested in using the last consumed quote per stock symbol. I'm not interested in quotes ...
0
votes
0
answers
47
views
How do I edit Azure Blob Metadata used in Live Service?
To remedy an issue, I've been trying to run a tool to upload new EventHub checkpoint offsets to store in blob metadata. I've also tried manually updating these in the Azure Portal. However, this is ...
0
votes
1
answer
19
views
Azure Event Hubs Schema Registry Error SubCode 40504 "conflicted with another request"
I am occasionally getting the error below when my .NET 8 Azure Functions app is trying to send events to my Event Hubs instance that is using Avro with Schema Registry:
SubCode=40504, Request on ...
0
votes
1
answer
83
views
Is there a way to specify the zone_redundant flag for an Event Hub since this flag has been deprecated in azurerm 4.0?
We have been using Terraform to instantiate an Event Hub with the zone_redundant flag set to true, and have been seeing the message:
The `zone_redundant` property has been deprecated and will be ...
0
votes
1
answer
45
views
How to Copy Data from primary eventhub to secondary
I am exploring about the data transfer from one eventhub to another eventhub but both are paired using Geo-Recovery since it wont replicate the data from primary to secondary want to copy the message ...
0
votes
1
answer
92
views
I'm unsure if I should use Azure Event Hub or Azure Event Grid
I've been digging into both products Azure Event Grid and Azure Event Hub to learn which tool suits better in my use case. I'll share a brief of my use case, hopefully that makes some clarity around ...
0
votes
1
answer
33
views
How does ingress distribution work across partitions in Azure Event Hubs?
I'm trying to understand how ingress distribution works across partitions in Azure Event Hubs, specifically in relation to partition allocation and throughput.
I know that the number of partitions in ...
0
votes
1
answer
175
views
How to consume from EventHubs Emulator with Azure EventProcessor?
Context
I have the local Azure EventHub Emulator running.
I set it up with docker as shown here: https://learn.microsoft.com/en-us/azure/event-hubs/test-locally-with-event-hub-emulator?source=...
0
votes
1
answer
40
views
How to configure MassTransit to get messages from Azure Event Hub, store fault and/or error message in RabbitMQ and correctly consume Fault<T>
I'm a newbie to MassTransit and for now I correctly configure MassTransit to get and consume messages of type "T" from Azure Event Hub, but a cannot view/consume Fault<T> messages when ...
0
votes
1
answer
38
views
Filebeat output using SAS token to azure eventhub
I want to send my filebeat logs to eventhubs using SAS token, earlier we wanted to use Kafka output but we came to know Kafka doesn't support SAS tokens, so now is there some way we can send data ...
1
vote
1
answer
39
views
Event Hubs Consumer Groups and Checkpoints Storage
There are numerous questions/answers here on what the Consumer Group on the Event Hubs is. It looks like the short answer is:
Each posted event is delivered to one of the consumer clients in each ...
0
votes
1
answer
28
views
not able to retrieve the data from Azure Event Hub with Pyspark
I'm trying to retrieve the data from Azure Event Hub using pysprak. the code just keeps running but doesn't display any data
EH_CONN_STR = 'Endpoint=sb://event-hub-18-jul.servicebus.windows.net/;...
0
votes
0
answers
28
views
Receiving Null values from Eventhub streaming
I am new to PySpark, and facing an issue while consuming data from the Azure eventhub. I am unable to deserialize the consumed data. I see only null values upon deserializing data using the schema. ...
0
votes
1
answer
113
views
spring boot auto-configure Azure EventHub handshake issue
I am writting an application using Java Spring Boot auto configure that tries to send a message to Azure EventHub using SPN for authentication without avail as i'm getting a cert issue. My code looks ...
0
votes
0
answers
69
views
Azure Logic App: Azure Event Hub Based Trigger - Reprocess Failed Events
I am looking for the details on the checkpoint mechanism in the Azure Logic App when the "When events are available in Event Hub" trigger is selected.
My simple setup as shown below:
...
-1
votes
1
answer
44
views
Spring Boot Consumer service unable to connect EventHub's consumer group
Have created two consumer groups in Eventhubs service, each consumer group used by different services,
Consumer group 1 name is $Default -> used by Service 1 with 1 instance running in AKS
Consumer ...
0
votes
1
answer
70
views
Authentication Error with Apache Camel Kafka Component Connecting to Azure Event Hub
I am attempting to send messages to an Azure Event Hub (Premium) using Apache Camel 4.4.3 with its Kafka component on Java 17, operating under Windows 10. While I have successfully used other Kafka ...
0
votes
1
answer
62
views
EventHub spark structured streaming using certificate authentication
I want to use structured streaming for EventHub message consumption using pyspark in Azure Synapse and I was pointed to this project https://github.com/alexott/databricks-playground/tree/main/kafka-...
0
votes
0
answers
45
views
How do I make Stream Analytics always send one event at a time with LineSeparated?
I'm facing a problem with Azure Stream Analytics and I need help...
Contextualizing...
I have a stream where I receive events from an Event Hub topic in Stream Analytics. When these events arrive in ...
0
votes
1
answer
127
views
Consume EventHub messages using spark structured streaming job with Service Principal + Certificate Auth
I want to use structured streaming for EventHub message consumption using pyspark in Azure Synapse, I found out this documentation https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-...
0
votes
1
answer
72
views
Sending Blob Data to Event Hub After JSON Parse
I am using an Azure Logic App to get data from an Azure Blob, convert it to JSON, index the JSON to take just the chunk I need, and then send it to EventHub. The data is of the following schema:
{
...
0
votes
1
answer
77
views
Subscribe to EventGrid for Defender For Cloud scan results?
I've managed to create an Azure Blob Storage and set up Defender For Cloud. I can see that on uploading it updates the tags for the blob.
I have also been able to set it up to send notifications to an ...
-1
votes
1
answer
132
views
How to add an inline record field using Avro in Azure schema registry?
the inline record
{
"type": "record",
"name": "Foo",
"namespace": "com.example",
"fields": [
{
"name"...
0
votes
1
answer
87
views
Why does Azure Self-hosted agent receive Connection refused error when connecting with EventHubs?
When using Azure self-hosted agent deployed on Kubernetes I face the following error:
[Error]2024-06-14T11:28:08.719 [EventHubSendFailed], exception: System.Net.Sockets.SocketException (111): ...
0
votes
1
answer
166
views
How to connect my pod on AKS to Azure Event Hub with its kafka interface?
How can I connect a Kafka consumer written in Python with Event Hub on an AKS pod? I've already tried using Workload Identity with a Service Connector (previously I've tried with a connection string ...
0
votes
1
answer
125
views
keda is unable to get eventhub metadata: no storage connection string given
I am currently trying to use Keda to start a job after having received an event from Azure Event Hub. When I deploy my scaledjob, I see the message (with kubectl describe):
Warning KEDAScalerFailed ...
0
votes
1
answer
58
views
How to establish authenticate, authorize from AKS deployed Container to Azure Event Hub
Apologies if my question is trivial. I have limited knowledge with Azure security.
( Identity , principal id, entra id....)
In my container app model, identity was created and used in Event Hub role ...
1
vote
0
answers
138
views
Spring boot - Event hub - non-nullable field authBytes was serialized as null - Standard plan - Kafka
I am getting an error when sending messages from local machine to Event Hub Kafka
The hub is standard tier and see Kafka enabled
So this stackoverflow post is not working for my scenario - ...
0
votes
1
answer
88
views
CompletionException error in KafkaListener while listening KafkaAvro Format events
I am getting this CompletionError while using KafkaListener to listen Avro format events from Azure EventHub.The error logs:
java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: ...
0
votes
1
answer
42
views
Azure Stream Analytcs Reading data from Event Hub
I have the below structure JSON data coming from event hub, and I am trying to process the same using Azure Stream Analytics.
I have tried some of the methods to extract the array elements from that ...
0
votes
1
answer
96
views
I would like to conduct performance testing of Azure Event Hubs and Event Grid using Jmeter
I would like to conduct performance testing of Azure's Event Hubs and Event Grid using Jmeter.
First, I downloaded the Event Hubs sampler plugin and configured it as follows.
I configured it with Test ...
0
votes
1
answer
60
views
What guarantees does Microsoft Graph provide for change notifications?
I am building a system that requires tracking changes of resources in MS Graph. I'm trying to figure out if I really need to depend on delta queries or not. To do this, I need to understand what ...
0
votes
1
answer
79
views
Why my Spring boot EventHubConsumer has Duplicate consumption issue
I have an EventHubConsumer that consumes data from Azure EventHub. So I have Event Grid Topic that is consuming from external Producer. The event grid topic after consuming from Producer it pushes ...
0
votes
1
answer
363
views
EventProcessorClient fails to start processing Eventhub messages - Creation of RequestResponseAmqpLink did not complete in 0 milliseconds
I created an Event hub processor in .net, followed that example:
https://github.com/Azure/azure-sdk-for-net/blob/Azure.Messaging.EventHubs.Processor_5.11.2/sdk/eventhub/Azure.Messaging.EventHubs....
1
vote
1
answer
128
views
Guidance on EventHub data ingestion process
I am sending the events from the application to eventHub.I am able to monitoring the health graph of eventHub due to limited access. there I have noticed that Metric Events received and Events ...
0
votes
1
answer
232
views
How to read from specific event hub partition spark-streaming fabric
I have a event hub with 32 partition .
I need to read from Partition 1 from Event hub using Pyspark .
This is my existing code
# Config
connectionString = "Endpoint=sb://abcd"
eventHubName = ...
0
votes
0
answers
103
views
Event hub No such host is known
I subscribed to an event hub in order to fetch messages from there
code:
BlobContainerClient storageClient = new BlobContainerClient(new Uri(serviceConfig.BlobContainerName), certificate);
...
0
votes
1
answer
319
views
How to Read Compressed Data from Azure Event Hub in Fabric using Py-spark
I am sending compressed data into Event Hub to overcome 1 MB hard limit in Azure Event Hub.
I also have to read this in Py-spark and update delta table.
The compressed data send to Event hub is coming ...
0
votes
0
answers
109
views
Error connecting to Event Hubs from spring applications - reactor.core.Exceptions$ErrorCallbackNotImplemented
I want to connect to event hub from Spring boot
https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/using-event-hubs-in-spring-applications
Use Spring Cloud Azure Event Hubs ...
0
votes
1
answer
105
views
How can I do Device to Device communication using Azure
I'm trying to communicate two devices let us assume Device A and Device B. According to the documentations online and references online we can do the following methods as per my understanding.
Use ...
0
votes
0
answers
62
views
Azure Function Eventhub output binding (typescript) - how to set Key/Headers with programming model v4
Using an output binding for Eventhub it was possible to add furhter message properties like Key/Headers in the v3 programming model, as shown in the examples:
context.bindings.outputKafkaMessage = [&...
0
votes
0
answers
62
views
Consumer on azure bootsrap server and Producer on kafka bootstrapserver possible?
I have a consumer which consumes messages from azure eventhub, i am using spring cloud stream for binding and once messages are consumed i am using kafka producer too publish to a kafka topic. Now the ...
0
votes
1
answer
106
views
From databricks I need to consume messages from my even hub using managed Identity
For databricks managed Identity I have assigned Azure Eventhub sender as well as Receiver Roles.
When I try to consume from eventhub using SASL Plain text its failing due to
java.lang.RuntimeException:...