Questions tagged [hazelcast-jet]
Hazelcast Jet is a distributed computing platform for low-latency stream and batch processing. It benefits from integration with Hazelcast IMDG providing fast, scalable Big Data processing that’s lightweight and embeddable.
178
questions
0
votes
1
answer
22
views
Problem trying to configure an interface and connect to the cluster
Hello people and good day, i'm trying to build an interface so, the interface it will be an external client that i need to get access from my linux console and execute the commands, this is the ...
0
votes
2
answers
24
views
Can we read parquet file in Hazelcast jet?
I trying to read parquet file via Hazelcast for that I have written below code which is working fine, but do Hazelcast provide any in-build source to read parquet file?
BatchSource<Object> ...
0
votes
0
answers
28
views
Aggregation and grouping using Hazelcast jet
I am trying to do grouping and aggregation using Hazelcast Jet but it is getting little bit slow as I have to loop through data twice one for creating groupingKey and after that aggregating all data ...
0
votes
0
answers
20
views
How to handle multiple requests to huge csv files?
I am working on on Java web application and in our application user can upload his csv files and than we allow them to create charts, table etc in our dashboard.
But when user uploads huge csv files ...
0
votes
0
answers
40
views
Hazelcast Jet not allowing Tomcat to stop
I am using Hazelcast jet to some aggregation and grouping but after being idle for sometime and when I tried to stop my tomcat it is not allowing to stop my tomcat and I have restart my PC. Below is ...
1
vote
0
answers
31
views
How to change exception from ERROR to WARRNING in Hazelcast
How to change exception level from ERROR to WARNING in hazelcast. How to customise the hazelcast global exception handler?
2022-04-07 15:07:06,094 [ERROR] [hz.laughing_bhabha.partition-operation....
0
votes
0
answers
42
views
Hazelcast SQL Performance Issue
I am trying to query Hazelcast cache Using Hazelcast-SQL which contains a map having around 140 columns.
When we try to query with all columns in select query then it took around 8 sec.
But when we ...
0
votes
1
answer
42
views
Hazelcast getSql().execute() throws Unrecognized client message received with type: 0x210100
I am trying out Hazelcast client-server.
So far, I
Got up hazelcast server members
Create a spring boot application and connected to hazelcast
Inserted/updated/queried Imap in hazelcast instance
Used ...
0
votes
0
answers
88
views
Apache Tomcat server did not shutdowns gracefully
I am trying to shutdown Apache Tomcat gracefully but it is not getting shutdown properly and gets hang may be because of Apache Camel or Hazelcast Jet not sure on context destroy I am shutting down ...
2
votes
2
answers
169
views
Hazelcast SQL interface slow performance HZ 4.2.2 vs HZ 5.0.2
Situation :
We have a product with approx 30 attributes (String, Enum, Double)
values
We have iMap with indexes for all attributes IndexType.HASH for
string value and IndexType.SORTED for double ...
1
vote
1
answer
49
views
How Hazecast BatchStage work with memory?
I am new to Hazelcast jet and I am trying to join to datas but I want to know how it will occupy memory and how much and what it best practice to do it?
Here is my code:
BatchStage<Map<String,...
0
votes
1
answer
33
views
Hazelcast Jet logs not working after configuration
I am new to Hazelcast and I am trying to change the logs location to file using logs4j but it seems to be not working. log file is getting created but there is no entry coming, please help how can I ...
0
votes
2
answers
30
views
Are Hazelcast Jet Reliable Topic Sinks idempotent? (Hazelcast fault-tolerance of a websocket source)
I cannot find this in the Hazelcast Jet 5.0 (or 4.x) documentation, so I hope someone can answer this here - can a reliable topic be used as an idempotent sink, for example to de-duplicate events ...
0
votes
1
answer
68
views
Hazelcast multiple left joins not working for two it is working fine
I am new to Hazelcast Jet and I m tryin to join 4 datas but join is not working as accepted I am not sure what wrong I am doing as for two tables it is working fine. Please help
My Code:
...
1
vote
1
answer
68
views
How to query embedded object field with sql from hazelcast?
I'm trying to get complex java object from hazelcast using sql approach.
My data object contains embedded object like
Person{
...
HumanName name;
...
}
HumanName {
String firstName;
}
...
1
vote
1
answer
44
views
How to Sink output to the json file in Hazelcast Jet?
I am new to Hazelcast Jet and in my application I am using jdbc source, this source returning List of Map which I have to Sink in one of json file but when I use Sinks.json it create json directory ...
1
vote
1
answer
237
views
Java parallelStream java.util.concurrent.RejectedExecutionException: Thread limit exceeded replacing blocked worker
I'm using Hazelcast Jet to do calculations over a large number of elements (~ 480 million).
I'm starting using an IMap Sink containing the starting data. I'm filling the map using
Collections2....
1
vote
2
answers
110
views
How to use Connection Pooling with Hazelcast jet JDBC?
I am trying to use connection pooling with hazelcast jet jdbc but it is not allowing me to do it.
I have Datasource bean from which I am fetching connection but it is not working.
Here is my code:
...
0
votes
1
answer
41
views
Problem with data structures using Hazelcast and Hazelcast Jet
I'm having a few problems with jet and hazelcast, but my question is for logic purposes, I have a class which is going to bring all the data and when we start the client we got a queue and 2 maps, but ...
0
votes
1
answer
51
views
In Hazelcast jet how can we store IList to normal list as I have to sent it in Response?
I am new to Hazelcast jet and in my application on data I am doing some aggregation and getting data but I want to send that in rest response so how can I change it to normal list?
public class ...
0
votes
1
answer
56
views
In Hazelcast jet how to Sink Map object to BatchSource?
I am new to Hazelcast jet and I am not getting how to Sink simple java.util.Map to BatchSource?
I have tried below but not seems to be working.
Map<String, Object> data =new HashMap<String, ...
0
votes
1
answer
80
views
Can't set hazelcast ManagedContext when using Jet.bootstrappedInstance()
I want to run hazelcast-jet embedded in my spring application (ie cluster is in process / NOT remote). I also want my jet jobs to be able to access spring beans from the ApplicationContext (see jet/...
0
votes
1
answer
104
views
Hazelcast Node multiple errors when run multiple nodes
i'm trying to launch several nodes with this programmatic configuration
public class Server {
private Server() {
//TODO: MapStore/MapLoader config here
MapConfig ...
0
votes
0
answers
55
views
Submit mutiple jobs in hazelcast jet
I have two separate jet pipeline. and I have to execute one by one.
public static void main(String[] args) {
JetInstance instance = Jet.newJetInstance();
// this job is for ...
0
votes
2
answers
45
views
StreamSource vs BatchStage in hazelcast jet
I'm using CDC in my application.
CDC always returns the object in the form of streams.StreamSource<ChangeRecord> source.
My Requirement:-
1] Capture the db changes using cdc and store in the map....
2
votes
2
answers
120
views
Hazelcast Jet change data capture
I'm using Hazelcast Change data capture (CDC) in my application. (Reason I'm using CDC because if use jdbc or other alternative feature to load data into cache its taking to much of time). So CDC ...
1
vote
1
answer
103
views
Hazelcast Change Data Capture with Postgres
I'm trying to use CDC for my Postgres Database.
and I have created simple project using Hazelcast Docs example.
https://jet-start.sh/docs/tutorials/cdc-postgres
public static void main(String[] args) {...
1
vote
1
answer
61
views
Hazelcast Jet "mapFn" must be Serializable Error
Good day, i'm having a problem implementing some code source from github to my project so... i'm trying to build a pipeline, and then create StreamStage to read the sources and build a queuesink with ...
0
votes
1
answer
38
views
Hazelcast jet data loading
I'm trying to load database data into hazelcast jet map instance
public void sourceStage(JetInstance jet, Pipeline pipeLine){
SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
...
0
votes
1
answer
30
views
Problem Deserializing Hazelcast Aggregator Response
I have this custom Aggregator written, which aggregation response is a HashMap. I have a problem on client side deserializer fails in the error given below. Any clue what could be wrong?
public class ...
1
vote
0
answers
44
views
Hazelcast jet Python Exception: PythonService initialization failed
While executing python function but it's not working:
Here is my code :
BatchStage<Object> bd1 = AggregateData.aggregate(data, Object);
bd1 = bd1.filter(k -> {
// some ...
2
votes
1
answer
37
views
How to apply python on BatchStage Hazelcast Jet
What I am trying to do is, I am fetching the data from different source and doing grouping and join on them and after that I am getting result in BatchStage and now I have to perform python code on ...
0
votes
1
answer
412
views
Hazelcast adding address to blacklist and client not connecting
I am new to Hazelcast and in my application when I am starting Hazelcast member it is showing two addresses been blacklisted don't know why, what is the best configuration for Hazelcast clustering?
...
0
votes
1
answer
41
views
Convert BatchStage<List<Object>> to BatchStage<Object> in Hazelcast
In my hazelcast jet application I am performing grouping joins after that I am getting result in
BatchStage<List> format but I want to convert it to BatchStage, I have tried flatMap but it ...
0
votes
0
answers
75
views
Hazelcast near statistics returning Null Pointer Exception with Client Configuration
EvictionConfig evictionConfig = new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LFU)
.setMaximumSizePolicy(EvictionConfig.MaxSizePolicy.ENTRY_COUNT)
...
0
votes
1
answer
90
views
Hazelcast jet flatMap says: These transforms have nothing attached to them: [flat-map]
I am trying to filter BatchStage but it is not working it is giving me below error:
java.lang.IllegalArgumentException: "filterFn" must be serializable
at com.hazelcast.jet.impl.util....
2
votes
1
answer
128
views
Perform Left, Right and Inner join in Hazelcast Jet
I am working on Hazelcast jet application and I trying to join two Sources using Left, Right or Inner Join but I am stuck at below:
Here is my code:
BatchStage<Map<String,Object>> batch1= ...
0
votes
1
answer
107
views
How to control number of processors during Hazelcast Imap filter
I am new to Hazelcast. When Imap filter with predicate is triggered on a HazelCast Jet pipeline, in a VDI I see the porcessors count is only 2 and on my machine I see 12 porcessors.
When there is only ...
0
votes
1
answer
107
views
How to read CSV in Hazelcast jet in List<Map> or JsonArray format?
As I am new to hazelcast am I trying few thing but not getting result as I accepted, please help me out.
Here is my below code which I am trying but not getting success.
BatchSource<List> ...
0
votes
2
answers
116
views
Hazelcast Jet with Named Jdbc Query or ResultSet?
I am new to Hazelcast Jet and I am using Spring JdbcTemplate to execute my query, which uses named parameters in the query, but I am not sure how to use it with Hazelcast Jet.
E.g. Hazelcast works in ...
2
votes
2
answers
171
views
wordcount test shows slowness in Flink
i am doing some benchmark comparison between streaming processing frameworks,
I selected WordCount such "Hello world" task (with some twists) in this area, and tested Flink and Hazelcast Jet ...
0
votes
0
answers
83
views
Junit Mockito test cases using Jupiter for Hazelcast Jet Pipiline
I am trying to write the Junit test cases for the below hazelcast Jet Pipeline.
My component:
@Component
public class UserJetJob extends AbstarctJetJob {
private static final String TABLE_NAME = &...
0
votes
1
answer
200
views
failed to map segment from shared object hazelcast mancenter
I have Hazelcast 4.2 runs in a linux server in a standalone mode by running the following files in background:
hazelcast management center : management-center/bin/start.sh
but hazelcast management ...
0
votes
1
answer
90
views
Hazelcast Jet vs Java 8 streams
I'm trying sort List<Map<String,Object>> object based on max date using Hazelcast Jet.
Here is my java 8 code that works:
public static List<Map<String, Object>> extractDate1(...
0
votes
1
answer
196
views
Serialization in the Hazelcast IMap
I'm trying to load data into IMap inside jet pipeLine stage I'm getting the error
Here is my code
public static Pipeline pipeLineStage(JetInstance jet) {
Pipeline pipeLine = Pipeline....
0
votes
1
answer
135
views
Serialization in the hazelcast jet
I'm getting the following error when I tried to processing the data from hazelcast jet.
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at com.example.LearnJet.joins....
0
votes
1
answer
120
views
Left join in the hazelcast jet
I'm using the Hazelcast Jet in My application.
// value1
BatchStageWithKey<Map<String, Object>, Object> indianCustomerJson = value
.filter(model -> model....
1
vote
1
answer
94
views
Why is Hazelcast IMap journal not producing all expected events?
The minimal working example below fast produces events which then update an IMap. The IMap in turn produces update events from its journal.
public class FastIMapExample {
private static final int ...
0
votes
0
answers
35
views
Does Hazelcast Jet allow to query accumulator values from rollingAggregate?
The minimal working example below presents some (simplified) processing where:
each event is processed individually (no windowing),
each event belongs to a certain group,
each event updates a group ...
1
vote
0
answers
41
views
Does Hazelcast Jet mapWithUpdating preserve Data Locality
The minimal working example below presents some (simplified) processing where:
each event is processed individually (no windowing),
each event belongs to a certain group,
each event updates a group ...