Questions tagged [data-stream]
In Connection-oriented communication, a data stream is a sequence of digitally encoded coherent signals (packets of data or data packets) used to transmit or receive information that is in the process of being transmitted.
205
questions
1
vote
2answers
58 views
How to cast a string in Apache Flink Datastream in Scala?
I'm writting a Scala script for processing a csv file in Apache Flink, using Datastream API.
I need to fix the format to some columns, and then cast them to proper types.
My current code is this:
...
0
votes
0answers
43 views
issues assigning/re-assigning real-time data to variables using asyncio - python
My intent is to assign/re-assign the data to a variable from the stream as it gets updated. As in the script below, a set of operations will execute based on the incoming data. I am relatively new to ...
0
votes
1answer
43 views
Python: dispatched on-line processing of high volume real-time datastreams
I have some high volume datastreams coming in on different websockets (sensor data - several TBs per month), for which
I want to guarantee that all data is stored, even during high load.
So I want to ...
0
votes
1answer
36 views
Fraud Detection DataStream API tutorial questions
I am following the tutorial here.
Q1: Why in the final application do we clear all states and delete timer whenever flagState = true regardless of the current transaction amount? I refer to this part ...
1
vote
0answers
42 views
Highland JS - turn reduce result array back to stream
I'm experimenting with data streams where I would like to aggregate time series data. The reduce works, however I cannot find a solution to turn back the resulted array to another stream.
When I call ...
0
votes
1answer
38 views
Apache Flink: How to update Source Function in Unit Test?
I need my Flink job to read from a local instance of a Source Function and update every time the Source Function instance's data changes within the unit testing code itself rather than a stream.
...
0
votes
0answers
26 views
fs.createReadStream CSV to SQL- Chunks Seem to Stop Progress
I'm very new to all of this. So I hope I can write all the information correctly. I'm having trouble parsing through a LARGE csv file (32,000KB) and inserting it into a SQL type database. This is the ...
0
votes
0answers
16 views
Streaming IOT data from SQL Lite DB to Cloud Service Provider
If we are storing the IOT sensors data to SQL Lite database running within Raspberry Pi, what are some of the approaches to stream this data to any cloud service provider?
So what we want to achieve ...
0
votes
0answers
14 views
Data stream clustering using scikit-multiflow
I have a CSV file data set as follows and I wrote my stream clustering algorithm. I wanna generate stream data to simulate the process. I found scikit-multiflow. I have a question that how can I do ...
0
votes
1answer
160 views
How can i use Debezium connector with Apache Flink
I'm trying to create a table whit flink's table API that uses a Debezium source function, I found an implementation of these functions here https://github.com/ververica/flink-cdc-connectors, and used ...
1
vote
0answers
17 views
How to correctly implement a bi-directional streaming http proxy?
I'm trying to write a .NET Core API Controller action that would basically act like a HTTP proxy:
Data comes in POSTed via Request.Body stream
Then POSTed to another backend service which processes ...
0
votes
1answer
60 views
Streaming data from the R&S rto oscilloscope - UnicodeDecodeError python3.6
I'm trying to get the signal data for a specific channel on the Rhode and Schwarz RTO oscilloscope . I'm using the vxi11 python(3.6) library to communicate with the scope.
On my first try, I was able ...
0
votes
0answers
22 views
Streaming with listen/notify Fails
I'm trying to use channels in postgresql to communicate a python script the values that are being inserted. The main goal is to stream data from postgresql to a python script. I have read that NOTIFY ...
3
votes
3answers
936 views
What's the use cases of Streams and Firehose?
I am working on an application that will read and analyze the logs of payment transactions. I know I will use Kinesis Analytics as per my requirements, which takes the input from the Data Streams and ...
0
votes
1answer
65 views
Having trouble getting readAllBytes() method to work
Other students in my class have gotten this code to work, but I get an error. I am trying to use the readAllBytes() method for an assignment and I can't get it to work. My teacher there is another ...
0
votes
0answers
303 views
pytorch dataloader default_collate argument use with to(device)
Ive been trying to integrate the to(device) inside my dataloader using to(device) as seen in https://github.com/pytorch/pytorch/issues/11372
I defined it on FashionMNIST in the following way:
device = ...
1
vote
1answer
153 views
Mulesoft with Salesforce Streaming API using CDC
I am working on a Mule API flow testing out the Salesforce event streams. I have my connector set up and subscribed to a streaming channel.
This is working just fine when I create / update / delete ...
0
votes
0answers
5k views
Netcat xxx.xxx.x.xx: inverse host lookup failed: h_errno 11004: NO_DATA
I am using netcat on Windows 10 to fetch a data stream from a data acquisition module. The command I'm using is:
nc -u -l -vv -p 5168 > log.txt
But then, looking at the verbose, I got:
listening on ...
0
votes
0answers
12 views
Adding custom DTMF tones
So as far as I know there are 12 DTMF tones that you can send over a call (0-9,*,#). Now I wish to develop an app that can send DTMF tones in a set pattern. But I need more than 12 existing tones. So ...
0
votes
0answers
17 views
Use records from S3 file to hit API endpoint
trying to gather ideas on how to design a system using only AWS resources.
Basically, I have a file in S3 (tsv for the sake of the argument) with a variable number of records (1000s to millions)
For ...
0
votes
1answer
21 views
Could someone provide me with some tips of how I could implement a given algorithm for distributed streaming in Flink?
I'm trying to implement Algorithm 2 (page 4) from this scientific paper (https://www.cse.ust.hk/~yike/pods10-cdsample.pdf) in Flink and I would be glad if someone helped me with some tips on how to ...
0
votes
0answers
18 views
client side storage data stream
I’m relatively new at web development and like to store a data stream at the client PC.
Background:
I have a small sensor with an accelerometer + BLE and need to log the values. I came across the “...
0
votes
1answer
38 views
How to load huge amount of data from spring boot to reactjs?
I have two applications, spring boot backend and react frontend. I need to load a lot of data (lets say 100 000 objects, each 3 Integer fields), and present it on a leaflet map. However i don't know ...
0
votes
0answers
12 views
Listen for messages of an already running process in NodeJs
I'm planning to write a code which will be composed of 3 scripts that will run simultaneously and communicate with each other, one of them will open the other two. This part is simple, I can use spawn ...
3
votes
0answers
294 views
Flutter Stream with Spring WebFlux
I'm stating one study with Spring WebFlux and I'd like to integrate it with Flutter to build one entire Reactive App.
On every site I found about Flutter Stream it shows Flutter + Firebase Stream, but ...
0
votes
0answers
19 views
Real Time Sampling of Data Stream
There is a stream of string data coming into a server. I want the server to pick whether the user who sent that piece of data wins a prize or not at the moment when the server receives the packet by ...
0
votes
0answers
36 views
Airflow pulling data from huge datasources
Very basic question.. Can Airflow be used to connect huge datastores that contain terrabytes of data ? Do the connectors/plugins for various data source perform well (from user's experience) ?
0
votes
0answers
16 views
Instrument to be referenced from another cell in a table
I am trying to fetch data using Thomson Reuters DS via DSWS and needed for instrument to reference to a cell in another SQL table that I have, i.e. "VOD" & "HSBA" to reference cells & to ...
1
vote
2answers
373 views
How to stream data with golang query to Cassandra
I have the following code:
cluster := gocql.NewCluster("our-cass")
cass, err := cluster.CreateSession()
defer cass.Close()
iter := cass.Query(`SELECT * FROM cmuser.users LIMIT 9999999999;`).Iter()
c :...
0
votes
0answers
51 views
Continuous communication between Flask app and Spring Boot app
I am trying to make an application that detects empty parking spots. The object detection is performed by a flask application. This application needs to do the following things:
* When accessing /...
0
votes
0answers
54 views
Is there any Bluetooth module which can support multiple smartphone connections?
We have a requirement of Bluetooth module which can be connected to multiple Bluetooth devices(Android phone) at same time.
All the paired phones connected to the module should receive data at same ...
0
votes
0answers
23 views
Data from the streamer server (Sensor Data from my phone) integrated in Python and in real time
I want to get the Sensor Data from my iPhone and process it on my Laptop.
I can get the Data with the Webbrowser but i should get it with python.
The IPhone App is phyphox https://phyphox.org/forums/...
0
votes
0answers
45 views
Aws C++ SDK Kinesis Data Stream issue when using SetStreamName and SetPartitionKey function in Linux
Issue Description
I have integrated AWS C++ SDK in my existing project. It works perfectly fine in Windows.
I had several compilation issues in Linux but in the end I was able to create a build ...
1
vote
1answer
47 views
kbmmw Client: 64-bit request stream issue
I have developped a client-server application using 64-bit kbmmw package on Delphi 10.3 IDE.
The 64-bit client application send the requests to server by SendRequest method (kbmmwClient class method) ...
0
votes
1answer
292 views
How store streaming data from Amazon Kinesis Data Firehose to s3 bucket
I want to improve my current application. I am using redis using ElastiCache in AWS in order to store some user data from my website.
This solution is not scalable and I want to scale it using ...
0
votes
1answer
33 views
Finishing write stream in external module upon program error
How can I make a write stream in an external module finish writing upon an error?
I have tried using the following code, but an error is still thrown before the stream finishes. I have also tried to ...
0
votes
0answers
93 views
Using socat to send TCP stream to multiple addresses
I am attempting to use socat to connect to an external host and send the TCP stream that I receive to several other hosts on my network.
I am trying this command
sudo socat -v TCP4:$EXTERNALIP:7816,...
0
votes
0answers
7 views
Data Stream creation
I have electrical data stored in my local disk. I have tried pre-processing techniques with stored static data in R. I wanted to try the same in stream data preprocessing. Can you please give idea how ...
0
votes
0answers
10 views
Finding a hash function that calculates average of an item in a stream of tuples
am new to this topic and I am trying to learn so my question might have been a bit confusing. What my question actually is: I have a stream of data coming in the form of tuple [Country, state, person]....
0
votes
1answer
28 views
How can we sum only the most recent values of a quantity computed in each stratum of a datastream using Flink?
I have a datastream (from a csv file) which contains strings and a specific value (a double) in every row. I use keyBy() in Flink to group these values by a specific attribute (country), so I have a ...
0
votes
1answer
61 views
Flink and Cassandra Connection Issue
Has anyone faced any issue with Connecting to Cassandra from a Flink Job when the Connection is made outside the Flink's DataStreams normally?
Session session = clusterBuilder.getCluster()....
0
votes
0answers
59 views
Python script for streaming TCP/IP data and saving it as .txt file
I have a software which has the option of streaming data as TCP/IP data. The used port is specified. Although I am not sure what should be the "hostname"? I want to receive this data using a python ...
0
votes
1answer
13 views
Best way to manage a queue or stack in java for DB Seq objects
In my application there are some data which always reads from Seq. I am thinking of having a data structure which would start getting loaded with DB Seq and whenever I need DB Seq, I can directly read ...
5
votes
4answers
724 views
Flutter DataStream not closing and re-building properly. [Bad state: Stream has already been listened to.]
Okay, so I'm aware that a stream can be manufactured to listen to a stream more than once, using a broadcast system, but that's specifically NOT what I'm trying to do here.
I'm also editing this as ...
-1
votes
1answer
51 views
Best way to push data to connected users from .NET web application [closed]
the scenario is the following :
-We have a MVC web application with an in-memory database providing users with a set of features.
-We created a REST web api in that application that an android ...
-1
votes
1answer
41 views
Java Socket does not error but is not functional(?)
I have a very basic understanding of clients, servers, sockets, input and output streams, etc. related to having servers. However, my socket that I created does not seem to work. I used many unused ip ...
2
votes
1answer
112 views
What is a data stream in DirectX 9?
I am practicing drawing in Direct3D9, and ran into a little problem.
For example, the first parameter in d3ddev->SetStreamSource(1, NULL, 0, sizeof(CUSTOMVERTEX)); is to specify the "data stream", ...
0
votes
1answer
188 views
Apache Flink DataStream - Count of Element in Tumbling Window
I’m very new to Apache Flink and its API.I want to create Java program which will do event time based processing with tumbling windows. I want to count the number of elements in the given window. ...
3
votes
1answer
2k views
What is the difference between AWS Kinesis and EventBridge
I'm an AWS noob, I'm trying to figure out what the difference between Amazon's Kinesis Data Stream and EventBridge products. Can someone explain this for someone not familiar with the AWS tech stack?
1
vote
1answer
93 views
continue an interrupted stream python pycurl
I am implementing a pycurl connection to a live datastream, sometimes the connection to the data stream disconnects and I am unsure as to how to continue the interrupted stream right away. I am not ...