Questions tagged [azure-stream-analytics]
Related to Microsoft Azure Cloud Analytics system
920
questions
0
votes
1
answer
53
views
How do I check write speeds to a container? AZure DLS Gen 2
I'm doing some work to test the write speeds of different output serializations (Avro, Parquet etc) for a streaming use case. I can see the size of the files, but how can I check other metrics like ...
0
votes
1
answer
97
views
Azure Stream Analytics · Counting people in/out from a room
I'm trying to make a query in Stream Analytics in order to send the output to PowerBI.
I need three things:
Counting people (location=Entrada - location=Salida). That is the load factor of the room, ...
0
votes
1
answer
19
views
Azure Stream Analytics Job query: how to case bigint to float?
My input is JSON like this:
{
"ClientStateOrProvince": "Noord-Holland",
"ClientType": "PC",
"IKey": "6ca435d6-f333-4888-...
0
votes
1
answer
70
views
Azure IOT Hub Python SDK application giving error in Stream Analytics "Could not deserialize the input event(s) from resource"
I am trying to send data to IOT Hub and store it in blob storage as well as Power BI streaming dataset using Azure Stream Analytics service.
For sending data, I used Microsoft's Python SDK application ...
0
votes
1
answer
54
views
What is causing malformed level error on parquet file generated via stream analytics?
I have a service bus topic subscription and a logic app that sends the data to event hub, this data is read by stream analytics and a parquet file is generated and stored in data lake. I am then ...
0
votes
1
answer
43
views
How to create time window based on more than just TIMESTAMP in FROM clause in Stream Analytics?
I have users who can take two actions:
Enter the room
Leave the room
I'm using Azure Stream Analytics and my goal is to create a table of users who entered the room but haven't left it in the next ...
0
votes
1
answer
55
views
Are there any examples of replacing Azure Stream Analytics by Azure Functions for distributing data coming from IoT hub into SQL db?
Are there any examples of replacing Azure Stream Analytics by Azure Functions for distributing data coming from IoT hub into Azure SQL database?
According to the question raised here and answers to it ...
0
votes
1
answer
104
views
Is Azure Stream Analytics viable for a use-case of many different sensor types with a single output?
Please me patient with me, I'm a junior developer. I'm working on a project where we're aiming to integrate lots of different LoRaWAN sensors, from multiple vendors. This means we are ingesting many ...
0
votes
0
answers
17
views
Stream analytics Anomaly detection over multiple devices
Is it possible to do anomaly detection in stream analytics when the input contains multiple devices.
I've tried using the partition key but that didn't help.
This is the query I tried.
WITH ...
0
votes
1
answer
46
views
I have a Azure Stream analytics job. I want to make it process output every 15 minutes
I am getting some events from event hub and saving them in ADLS Gen 2 without performing any operation. Just saving the live events in ADLS Gen 2.
I am not doing any kind of sum, average or ...
0
votes
1
answer
78
views
Creating Azure Stream Analytics job using the python SDK gives an error
Creating Azure Stream Analytics job using the python SDK gives an error azure.core.exceptions.HttpResponseError: (BadRequest). The JSON provided in the request body is invalid. The required property '...
0
votes
1
answer
42
views
Hopping Window from DeltaLake to Power BI
A Databricks job writes one delta-parquet file per minute into an Azure Storage Account (ADLS2), partitioned by event_date + event_hour:
This 1-minute frequency is just an average: sometimes 10 files ...
0
votes
1
answer
49
views
Azure Stream Analytics and JSON data
I have an event hub which im ingesting data from to put into a synapse table. The event data has multiple layers of nested JSON, and im trying to manipulate some of this data. e.g
{
"...
0
votes
1
answer
58
views
calculate cummulative sum in azure stream analytics
Is it possible to calculate the cummulative sum over groups in azure stream analytics?
This is the input:
TMS product amount
2023-06-01 10:00:00 P1 10
2023-...
0
votes
0
answers
77
views
Azure streaming analytics
This is more of an architecture question.
I’ve got a current architecture which sits like this
Azure http function -> service bus -> azure bus function -> database -> azure scheduled ...
0
votes
1
answer
78
views
Calculate volume weighted average price of trades based on the last traded 500 units of volume in azure stream analytics
I have streaming data coming from eventhub as follows:
product price quantity
1 55 100
2 44 200
2 43 200
1 60 300
2 55 100
2 ...
0
votes
1
answer
90
views
Azure Stream Analytics- Window Function to aggregate current event record referencing data from 12 AM UTC
I have a scenario where I need to aggregate only the latest events (keys) from the stream job with data from 12 a.m. UTC every day. It looks like I can't go with only hopping and sliding windows as ...
0
votes
0
answers
68
views
Stream Analytics JOIN new data with value from a database and then store in said database
I have a database that is indexed with an ID and a counter and I'm sending the data from an event hub through to stream analytics and then to a SQL database. I want to get the latest counter for a ...
0
votes
1
answer
101
views
Variables in Azure Stream Analytics
I'm trying to create a self incrementing value in my database the increases value after there has been a gap in time greater than a second. The Stream Analytics is currently sending data from an event ...
0
votes
2
answers
266
views
Azure Stream Analytics reports watermark delay
Azure Stream Analytics is used to distribute JSON data incoming from azure IoT hub into tables of SQL database.
Recently there were several large watermark delays which totally ruined the pipeline.
...
0
votes
1
answer
114
views
Adding data from Azure Event Hub to Blob
I've got a number of process that throw data an an Azure Event Hub.
I've a Stream Analytics Job that takes each message and pushes it top a blob in a specified folder.
This all seems fin until I look ...
0
votes
0
answers
64
views
How to use Nuget package in Azure Stream Analytics Application in Visual Studio?
I want to use System.Data.SqlClient package in my C# function to read a value from a table in my sql database so that the same can be used in my sql Query by returning value from my UDF.
https://learn....
0
votes
1
answer
112
views
Azure Stream analytics can not retrieve ML endpoint swagger
I successfully deployed a model on Azure machine learning studio following the official guide
The endpoint and the deployment are successfull, I can invoke it without any problem. I also added the ...
0
votes
1
answer
86
views
How to get dt-subject system property from Azure IoT Hub into Azure Stream Analytics
Background: I am using Azure Stream Analytics to process telemetry events from IoT Hub. The telemetry messages themselves include a dt-subject system property, because the telemetry corresponds to a ...
0
votes
0
answers
65
views
Multiple Application Insights Resources For Single App
We have a current setup within our system as follows:
App service A -> App insights (classic) -> Continuous Export -> Storage Account -> Azure Data Factory -> Azure SQL B
We want to ...
0
votes
0
answers
36
views
Azure Stream analytics - Handle Same second records
I have a lot of telemetry data being sent by various sensors and it often happens that multiple parameters for a single device is changed within the same second (with millisecond differences). I want ...
0
votes
1
answer
108
views
How Azure Stream Analytics find schema of data coming from Event Hub
I am following this tutorial to understand how Stream Analytics work on Azure. https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-real-time-fraud-detection?toc=%2Fazure%...
1
vote
0
answers
105
views
Azure Data explorer sink option not available for stream analytics output in VS 2022 [closed]
I am using the below visual studio version
Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.4.4
I am using VS 2022 to manage my the development part of the stream analytics, and ...
0
votes
0
answers
77
views
send datetime with offset field in Stream Analytics
I'm trying to send a Timestamp field which is ISO 8601 with offset (
"2023-02-01T11:11:12.2220000+03:00" )
Azure doesn't really work with offsets, I first encountered that when sending data ...
0
votes
1
answer
106
views
The message from Azure IoT hub is reaching partially to Stream Analytics and thereby SQL db
The message from Azure IoT edge device installed on Raspberry Pi is sent to the cloud.
In the logs of IoT edge device on Azure Portal the message sent to the cloud is seen (in the troubleshoot section)...
0
votes
1
answer
102
views
Is there a way to send data to a SQL server in a private vnet using stream analytics jobs?
As mentioned in the title, I am have set up a sql server in a private vnet which only can access the azure app service. Is there a way to send data to the SQL server using a Stream Analytics Job while ...
1
vote
0
answers
106
views
Two very similar SQL statement, but one causes my HTTP Request Azure function to trigger and the other doesn't
In Azure Stream Analytics, I'm receiving telemetry from a sensor to the IoT Hub, and I'm outputting that to a C# Azure Function (HTTP Request type). The function sends an email with a warning.
The SQL ...
0
votes
1
answer
95
views
How can I create a Data Explorer Stream Analytics output using ARM templates?
I successfully manually configured a Stream Analytics Job that outputs data into Data Explorer. However, I am unable to set up my infrastructure pipeline using ARM templates. All the documentation ...
0
votes
1
answer
279
views
Azure IoT Hub routing to Event Hub
I created a Event Hub with 32 partitions and doing the message injection over the IoT Hub message routing.
If i connect with Stream Analytics to my Event Hub and look at the input i get this message: &...
0
votes
1
answer
148
views
Azure Event Hub - Stream Analytics architecture
I need some help figuring out how to build/optimize my Azure architecture for the future.
I currently have a test running that looks like this:
I am currently sending some kind of data x1 (700k a day)...
0
votes
1
answer
69
views
Release note or changelog popup after extension upgraded
I developed an extension for VS Code helping users manage their services in Azure. I found that some of our extension users were not aware of the new features after the extension was upgraded. Is ...
0
votes
1
answer
74
views
Pass streams with array and combine records based on specific id
I have a stream from IoT Hub like:
{"timestamp":1669022177222,
"values":[
{"id":"Channel1.Device1.Tag1","v":62799,"q":true,"...
0
votes
1
answer
212
views
Durable Entity function doesn't trigger or receive data from streaming analytics
The durable entity function I've setup as output for a streaming analytics job doesn't trigger or receive any data. I can see incoming input data in the stream analytics job and it outputs to store ...
0
votes
1
answer
330
views
Exceeded the maximum number of allowed receivers. Azure stream analytics job connected to Iot Hub
I have an Azure Stream Analytics (ASA) job which I'm using to route events from my Iot Hub to some function apps and blob storage. I just recently got the following error at the Input Preview when my ...
1
vote
1
answer
86
views
From Azure Stream Analytics to Power Bi Desktop
I have done some research on my own, but cant seem to find a solution. We have an IoT solution in Azure. We are gathering real-time streaming data from sensors to Azure IoT HUB. This data is getting ...
0
votes
0
answers
137
views
Azure - ingesting data into IoT Hub and sending notifications via slack/email for certain messages
I've got data coming into IoTHub and want to filter on them.
Relevant data I want to forward to slack as notification.
I've got the IoT Hub and a slack subscription in place and am having trouble ...
0
votes
2
answers
163
views
Load JSON to SQL database tables
I have JSON files that at the end of the day should be inserted to SQL Database.
To explain what I mean, let's assume the JSON looks like this:{ name : "John", surname : "Smith", ...
0
votes
1
answer
132
views
Using a sliding window in stream analytics causes a type conversion error (bigint somehow becomes a double)
So I'm trying to run the following query in stream analytics:
WITH
InputData AS(
SELECT
tags.[tag0] AS 'shift',
tags.[tag1] AS 'Good Count',
tags.[tag2] AS 'Bad Count'
...
0
votes
1
answer
155
views
Azure Stream Analytics - JSON Array, windowing and previous value
I receive containers of senor data as an input for ASA. Containers look like this.
{
"data": [
{
"sensor_id": 55,
"timestamp": 1663075725000,
"value&...
1
vote
0
answers
179
views
Use variables in azure stream analytics properties
I want to reduce the number of overrides during the deployment of my ASA by using environment variables in my properties.
Expectations
Having variables defined in the asaproj.json or the JobConfig....
0
votes
0
answers
128
views
Create alerts using Azure Stream Analytics, Functions, and Notifications Hub
I am working on a project where I have to create alerts on incoming data in Azure Stream Analytics using Azure Functions and Notifications Hub. I would like to know if there is any way using either of ...
1
vote
1
answer
197
views
Agregate JSON events into an array in Azure Stream Analytics
I'm new to Azure Stream Analytics and query language. I have an ASA job which reads json data coming from my IoT Hub and feeds it to different functions based on one of the values. This is what I have ...
0
votes
1
answer
123
views
Is a Azure Stream Analytics job with sparse/ none output possible?
Not the best question, I'm aware but I couldn't find much information on this and currently lack the time to test it.
Is it in principle possible with Azure Stream Analytics to only forward an output ...
-1
votes
1
answer
150
views
No data getting pushed from Azure stream analytics to Cosmos DB
I have data getting pushed from Azure IoT Hub -> Stream Analytics -> CosmosDB (Core SQL API)
In stream analytics I can see the data is coming. But I am not seeing the data in cosmos db.
I have ...
0
votes
1
answer
58
views
Ideal Azure/Power BI Architecture for Stream Analytics with RLS
We are using Azure Stream Analytics to build out a new IoT product. The data is successfully streaming to Power BI but there is no way to implement Row Level Security so we can display this data back ...