Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Filter by
Sorted by
Tagged with
0 votes
0 answers
34 views

I want to integrate CoAP with the Azure with IoT hub. Below are the available services a) Event hub b) function appsc) Cumulocity

I want to know whether CoAP is supported by Azure Event Hub. If not, Please help with an architecture which support CoAP without IoT hub. I have designed that CoAP will be ingestes Event hub and then ...
Naveen G's user avatar
0 votes
0 answers
50 views

How can we add more consumer groups to Azure Event Hub? [closed]

Background We are using azure event hubs with pricing tier "Standard" and we just hit the maximum of 20 consumer groups. We have roughly 20 microservices that consume messages from the same ...
lightbricko's user avatar
  • 2,657
1 vote
1 answer
67 views

Azure EventHub Function Trigger Default Retry Mechanism

I am working on an Azure EventHub function trigger to consume messages in batches. I would like to know the following two things. Is there a default retry policy/mechanism in the event hub trigger so ...
diyath.nelaka's user avatar
0 votes
0 answers
25 views

How to create time window based on more than just TIMESTAMP in FROM clause in Stream Analytics?

I have users who can take two actions: Enter the room Leave the room I'm using Azure Stream Analytics and my goal is to create a table of users who entered the room but haven't left it in the next ...
Daniil Podtesov's user avatar
0 votes
0 answers
74 views

Capturing API Management request through diagnosis settings and event hub in Azure

I am using a goat function app AzureGoat. I have integrated my AzureGoat function app into API Management Azure. When I test a GET request from the API management console then I am getting logs in ...
Harry's user avatar
  • 1
0 votes
1 answer
47 views

Python / Azure Event Hubs

I'm putting together a very basic Python script which suppose to send a sample JSON data to Azure Event Hubs. Here is my script: import asyncio from azure.eventhub.aio import EventHubProducerClient ...
Eugene Goldberg's user avatar
0 votes
1 answer
21 views

Node.js / Azure Event Hubs "TypeError: EventHubProducerClient is not a constructor"

I'm trying to put together a very simple node script to send some data to Azure Event Hubs. Here is my script (send.js): const { EventHubProducerClient } = require("@azure/event-hubs"); // ...
Eugene Goldberg's user avatar
1 vote
0 answers
20 views

What is the most common way to ingest JSON file changes from Azure BLOB storage into Logstash?

I have a need to integrate Logstash data ingestion to include JSON files which are stored in an Azure BLOB storage container. These JSON files are being constantly appended to and Logstash is expected ...
Eugene Goldberg's user avatar
0 votes
2 answers
20 views

Azure Eventhub, pausing capture looses events when restarted

Mysql DB with Java front end sending compressed and encrypted messaged to eventhub. All fine here. EventHub Capture is every 5 mins sending Avro with all messages to Storage account. For maintenance ...
user22538275's user avatar
0 votes
1 answer
71 views

Azure Event Hub With Net Core Web API

I am new with Azure event hub.My requirement is that I want to listen from one of the third-party azure event hub(I have only connection-string of third party event hub and my own storage account) ...
Harsh Joshi's user avatar
0 votes
1 answer
35 views

Upgrade Azure.Messaging.EventHubs from 5.5.0 to 5.9.0. When I use 'eventData.Body.Array, eventData.Body.Offset' return error

Upgrade Azure.Messaging.EventHubs from 5.5.0 to 5.9.0. I need to change old Microsoft.Azure.EventHubs.EventData to Azure.Messaging.EventHubs.EventData. But when I use : var decompressed = Decompress(...
yoyo's user avatar
  • 23
1 vote
1 answer
58 views

How to ingest Event Hub event list created by Azure Diagnostics into Azure Data Explorer by subscribing to Event Hub Notifications?

I have an event hub that emits logs from Azure Diagnostic Settings (specifically from a Function App). These logs are in the form of a list and I would like for them to be ingested into Azure Data ...
Rajesh Patel's user avatar
0 votes
1 answer
41 views

Using specific consumer group in event hub in Azure to input the event

I have 2 consumer groups in event hub. It is being used by Synapse to push event and then consumed by Azure function and then AZ function pushes to event hub and gets consumed by Logic App. Name of ...
heman123's user avatar
  • 3,011
0 votes
1 answer
44 views

pyspark batch data process of Huge data

i have huge data which is more than 10 milions this data can be paritioned by a key in the data frame "partition key" I want to publish the data in the group of this partition key ,is there ...
Manoj's user avatar
  • 63
0 votes
1 answer
28 views

Sending event to Event Hub from third party applications

How do I send event from a third-party application like CREATIO to Azure Event Hub? I had set up the event hub and shared access policies as well. Is there any option available in CREATIO to send the ...
Sandeep T's user avatar
  • 421
0 votes
0 answers
33 views

Azure Eventhub Consume Message throttling in metrics

Problem Statement Single consumer is consuming the message from the single topic partition. The consuming topic partition has 8578 messages The observation from Azure Eventhub Metrics, Outgoing ...
Vishnu Kumar K S D's user avatar
0 votes
1 answer
45 views

Count of Availability Zones in Each region in Azure for EventGrid , Event Hub

Any az cli command in general to get to know count (1?2?3?) of availability zones in a particular region of Azure? I am trying to get these details for EventGrid and event hub specifically..
ABC DEF's user avatar
  • 189
0 votes
1 answer
55 views

Clickstream analytics on Azure - End to End

I am trying to create a POC for an end to end Clickstream analytics solution using Azure products. The plan would be: write a simple JS that captures the event I want to send to Azure. This will be ...
Davide's user avatar
  • 1
0 votes
1 answer
53 views

How to replay the messages in azure eventhub

I have a azure eventhub and has multiple consumers for the eventhub. Checkpoint are available for these consumers. I would like to replay the messages for a specific consumer. How can I achieve this? ...
Madhu's user avatar
  • 411
1 vote
0 answers
46 views

Azure Eventhub Consume Message throttling [closed]

Problem Statement Single consumer is consuming the message from the single topic partition. The consuming topic partition has 8578 messages The observation from Azure Eventhub Metrics, Outgoing ...
Vishnu Kumar K S D's user avatar
0 votes
0 answers
83 views

Not able to receive messages from azure event hubs

Getting below exception while connecting eventhub in spring boot application. @SpringBootApplication public class VALogAnalyticsApplication implements CommandLineRunner { private static final Sinks....
Prem Kumar's user avatar
0 votes
1 answer
62 views

Azure Event Grid to Azure Event Hub events delivery failed

We run into a problem with the subject. May be someone knows a solution. Talking to Azure support for 4 days already and the only thing they come up is "it looks like a problem on your end". ...
777's user avatar
  • 1
0 votes
0 answers
54 views

Eventhub consumer skips messages

We're using a python event hub consumer client that reads from a topic containing 32 partitions. We've enabled checkpointing as per the documentation (that seems to work). Events get consumed via a ...
Havnar's user avatar
  • 2,558
0 votes
1 answer
64 views

Extract incremental live data ingestion from salesforce using Change Data Capture

I have successfully copied data from Salesforce to my Azure SQL Database using Azure Data Factory pipelines, and I want to ensure that my pipeline automatically retrieves updated or new entries from ...
Abhishek Chaudhary's user avatar
0 votes
1 answer
178 views

Databricks delta live tables from Azure data stream error - no tables were discovered

Attempted to update an empty pipeline. This error usually means that no tables were discovered in your specified Notebook libraries. Please verify that your Notebook libraries include table ...
Aseem's user avatar
  • 5,896
0 votes
1 answer
51 views

Json payload to string without spaces and newline characters

I'm consuming events from Azure Event Hub, and need to log the event payload as a string. The payload is in JSON format. private readonly EventProcessorClient _eventProcessor; _eventProcessor....
Darkside's user avatar
  • 470
0 votes
1 answer
28 views

Is it possible to set Azure Data Explorer and Event Hub with Cross-tenant data connection using Service Principals?

I want to set up an Event Hub Data connection in an Azure Data Explorer. The Azure Data Explorer is in a different tenant than the Event Hub. Checking the Azure Documentation about Cross-tenant data ...
Alejandro Alvarez's user avatar
0 votes
0 answers
40 views

Azure Event Hub Consumer group event is processed twice by 2 different listeners (Azure function)

We have a consumer group created for an event hub in Azure. Two Azure functions from two different function apps are subscribed to same consumer group. These functions apps are in different regions. ...
Vicky's user avatar
  • 624
0 votes
0 answers
68 views

Azure Event Hub Schema Registry for JSON Serialization always auto generates schema

The auto.register.schemas option for a JSON schema registry group in event hub does not seem to work. Using sample json producer project for azure-schema-registry-for-kafka, I expect a validation ...
bvk's user avatar
  • 395
0 votes
1 answer
63 views

Trouble Configuring Azure Event Grid Topic to Event Hub Event Forwarding

I aim to transmit Events from an Azure Event Grid Topic to an Event Hub. The desired setup involves disabling public network access while allowing an exception for trusted Microsoft services through ...
CUiD8pdQJh's user avatar
0 votes
0 answers
54 views

Migrate Kafka cluster to Azure Eventhubs

My objective is to migrate data from a kafka cluster to Azure eventhub . I tried MirrorMaker2 , it works but it requires namespace level admin privileges to work as MirrorMaker creates some internal ...
redeemed's user avatar
  • 471
1 vote
1 answer
92 views

EventHubTrigger EventData[] binding not working

I am making a C# Isolated worker process Azure Function triggered by an EventHub that will output a list of EventData with headers into an other EventHub. The Function works well when using the string[...
Ceci Semble Absurde.'s user avatar
0 votes
1 answer
54 views

Streaming job to write data from delta table to event hub failing in databricks - Timeout issue

Here is my code to write data from a delta table to event hub (from where consumer team will consume data): import org.apache.spark.eventhubs._ import org.apache.spark.sql.streaming.Trigger._ ...
Apleen Dhaliwal's user avatar
0 votes
1 answer
64 views

Not able to create multiple event-hub in one namespace using ARM template

I am trying to create multiple event hubs in a single namespace but I am not able to do so. I have tried a single event hub which I am able to create. When I am trying to create multiple event hubs ...
Ravi kant Gautam's user avatar
0 votes
1 answer
35 views

How to group messages so that they go to the same partition every time?

We are using java, Akka framework and azure event hub. We are using akka framework's groupedWeightedWithin to group messages based on the total number of bytes or duration. I don't know how to set the ...
Venky's user avatar
  • 177
0 votes
0 answers
106 views

MirrorMaker 2 - Event Hub connection terminated during authentication

The goal We are trying to mirror a topic on a source Kafka (open source variant) to Azure Event Hubs using MirrorMaker 2. I have reviewed the documentation of both Kafka and Event Hub. MirrorMaker 2 ...
endnjsretia's user avatar
0 votes
1 answer
139 views

Event Hub Source Connector throwing "Invalid Offset" error after changing event hub and consumer group

I have an Event Hub Source Connector in my setup that was initially pointing to a single partition Event Hub within a namespace. Recently, we decided to migrate to a new Event Hub with an increased ...
Akhil Kondapaneni's user avatar
0 votes
2 answers
128 views

How to send Json & Avro format events to Azure Event Hub and finally Ingesting them into Azure Data Explorer Table. Using .Net (c#)

My requirement is using Azure Event Hub to capture events in format Json and/or Avro format and finally ingesting those events into ADX. Also don't want to capture these events in to Blob storage etc ...
suresh pasupunuri's user avatar
0 votes
0 answers
48 views

Azure Function app with event hub trigger stops randomly from consuming event hub message

All of sudden, Azure Function app with event hub trigger stopped consuming messages/events from partition. Restarting the function app mitigated the issue. What would be possible reason? It is happen ...
191180rk's user avatar
  • 745
0 votes
1 answer
26 views

Run databricks jar task

To install a jar library on a job cluster, I try to use the jar task, where I select the maven library, but I can not find the main class of the library. does anyone know what the main class for com....
Beesure 's user avatar
0 votes
1 answer
47 views

High Egress using Spark-Kafka Adapter on EventHub

We have an EventHub Namespace with 2 EventHubs, 32 partitions and 20 Throughput Units. One of the Eventhubs is dormant with very little throughput and the other has much higher throughput, accounting ...
TheRealJimShady's user avatar
0 votes
0 answers
70 views

Kafka consumer does not resume processing where it last left off (Azure EventHub)

I'm struggling to configure processing of event messages. My goal: create a job that consumes events. Events are added to the queue continuously and my job is executed every 5 minutes. I want my job ...
Piotrek's user avatar
  • 11k
0 votes
0 answers
71 views

Lots of event hub timeout error in Azure function after migrating to RBAC auth to event hub

I had an Event hub triggered Azure function which access event hub using connection string in app settings. Recently, I updated the code to access event hub using RBAC auth by configuring fully ...
Youxu's user avatar
  • 1,050
0 votes
1 answer
55 views

Not able to push the data into Azure event hub

I am trying to push the data sample data into the Azure event hub but I am not able to do so. import azure.functions as func from azure.eventhub import EventData from azure.eventhub.aio import ...
Ravi kant Gautam's user avatar
0 votes
2 answers
124 views

Azure Function App Invocation within same Event Hub Partition

May I confirm with you, when we have an Event Hub with multiple partitions, is there any possibility that you can have more than one instances of Azure Function that process the event on same ...
LumineLittlelight's user avatar
0 votes
3 answers
580 views

RabbitMQ Shovel to Azure Event Hub or Service Bus

I am hoping someone has got this to work. I am the trying to connect RabbitMQ shovel to Azure Event hub or Service Bus. I found two tutorials on MS, (very short to set up) but both results in the ...
aspen l's user avatar
  • 54
0 votes
1 answer
327 views

Snowflake: Recommendation how to integrate streaming data from Azure Event Hub

I'm trying to figure out how to best integrate streaming data from Azure Event Hub into Snowflake, e.g. with low latency and as cost-efficient as possible. Brainstorming with myself I came up with 2 ...
saso's user avatar
  • 764
0 votes
0 answers
92 views

Azure SQL Managed Instance Drop/Deletion Event logs

We have an Azure SQL Managed Instance having number of databases on it . We have to enable audit logs of SQL Managed Instance from diagnostic settings and also enabled logs by executing queries on MI ...
Nauman Kyani's user avatar
0 votes
1 answer
200 views

Parallel processing in Azure Event Hubs (multiple consumers on same partition)

Azure Event Hubs allows partitions within a consumer group. In my example, i have two consumers that send out notifications with different logics. I need to decide within the consumer which logic to ...
Aitin's user avatar
  • 21
1 vote
1 answer
128 views

timeout in Azure functions App binding with events hub to send data

I have a nodeJS functions app, in which i have a http trigger function with following code. It runs successfully. With all the below code and configuration, i am able to send data to events hub from a ...
user4906240's user avatar

1
2 3 4 5
39