Skip to main content
Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

Spring-batch - read and write in the same table in batches (pagination)

How can I use (or is there a way) to use spring-batch to read and write in the same table in batches? For context: I have a table that has millions of registers. From time to time many of those ...
Mariza's user avatar
  • 11
0 votes
1 answer
28 views

org.springframework.batch.item.WriterNotOpenException: Writer must be open before it can be written to - in Spring batch Multithreading?

I've used FlatFileItemReader with the Classifier and ClassifierCompositeItemWriter in my project where we read Tab delimited file and convert it into CSV as a output file by performing some validation,...
PAA's user avatar
  • 11.7k
-1 votes
1 answer
41 views

Database not initialized when executing end-to-end tests

I'm trying to build an end-to-end test on a spring-batch project, however, when trying to run the execution of a single step, the database appears to be not initialized. The contextLoads test runs ...
Allan Vital's user avatar
0 votes
1 answer
25 views

spring batch job using less than 1 cpu despite thread pool of size 20

I have below step configured that is running super slow. On our own servers it runs in less than 20 minutes but in the cloud it takes 10+ hours. We have cpu request set to 4 and limit of 20 (this is ...
futevolei's user avatar
  • 190
0 votes
0 answers
35 views

Spring Batch - could not serialize access due to read/write dependencies among transactions

I am facing some issues with Spring Batch metadata tables upon transactions. org.springframework.dao.CannotAcquireLockException: PreparedStatementCallback; SQL [INSERT INTO ...
Thomson Ignesious's user avatar
0 votes
1 answer
24 views

Spring Batch upgrade - No Scope registered for scope "Step"

I have a spring batch that does not use spring boot. I am upgrading it to Spring batch version 5 The batch has a reader bean which is stepscopped because I need to access the jobparameters in it. With ...
cucumber's user avatar
-1 votes
1 answer
23 views

How to stop the Spring batch job gracefully post app crash or DB restart

I am using spring boot application and spring batch for background job. When I schedule to job, it is running. Suddenly application restart or DB restart is happening due to some reason. Now, Spring ...
Galet's user avatar
  • 6,169
0 votes
0 answers
17 views

Why caching in springboot not working in multi threaded environment such as spring batch application

I am working on spring batch application and implemented multi-threading (ThreadPoolTaskExecutor) along with using Partition interface provided grid size 12 and number of threads also 12. For ...
pawan dhami's user avatar
-1 votes
0 answers
29 views

Unexpected Processor Retry Behavior and Fault Tolerance in Spring Batch

@Bean public Step step1(DataSourceTransactionManager dataSourceTransactionManager) throws Exception { return new StepBuilder(STEP_NAME, jobRepository) .<StartTask, ...
239010391's user avatar
2 votes
0 answers
101 views

How to stop the Spring batch gracefully in Spring boot application

I am using Spring batch for my spring boot application. After I schedule the job in the background, i want to stop the job. I am using following to stop the job jobOperator.stop(jobId) Job is going ...
Galet's user avatar
  • 6,169
0 votes
1 answer
33 views

How can I use XStream to start parsing from a given XML Path

I have an XML file with the following shape: <file> <items> <item name="item1"> <labels> <label code="item 2 label 1" /> <...
Arno 2501's user avatar
  • 9,289
0 votes
0 answers
14 views

How should long running steps be used in Spring Batch with transaction management?

I'm getting started with Spring Batch and am trying to handle the following scenario: I have a SQL database with database records in a table. For each of these records, I would like to make an HTTP ...
Knox's user avatar
  • 1,300
-1 votes
1 answer
34 views

How do I setup KafkaItemReaderBuilder to read all partitions?

I currently use the partitions method with the list of partitions to be consumed : partitions(0, 1, 2) If a partition 3 is added, it will not consume it. How do I tell KafkaItemReaderBuilder I want to ...
Sybuser's user avatar
  • 1,338
0 votes
0 answers
20 views

Not able to consume the message from Kafka topic in latest Spring batch application( Spring Batch 5.x)

We developed a spring boot application using Spring Batch 4 and Spring Cloud Stream 3 earlier. Consume the data from Kafka using @EnableBinding(Sink::class) and @StreamListener(Sink.INPUT) in the ...
anil kumar's user avatar
0 votes
0 answers
14 views

Creating new Transaction for Spring Batch Writer while handling Error Listener

I have a spring batch job where the Writer gets a list of items. I am using chunk oriented processing. My spring batch chunk size is 100. I have enabled fault tolerant for the step. Since the writer ...
Yash Tailor 's user avatar
0 votes
2 answers
79 views

Microsoft Graph API fails to update user birthday and hireDate in GCC high environment

I have a spring batch job that is scheduled to update the hireDate and birthday attributes of a user profile in SharePoint using MS Graph API in the GCC high environment. Below are the details: PATCH ...
user220982's user avatar
0 votes
0 answers
17 views

MongoDB batch document update strategy with Spring Batch

I am trying to update multiple ‘MongoDB’ documents of a collection in TEST database with the same collection in LIVE database using Spring Batch’. For this purpose I thought to delete the ...
Murat Yıldız's user avatar
0 votes
1 answer
75 views

Spring Batch. DataSource must not be null

I created a new Spring project to test Spring batch in Java 17. The job is simple, print "Hello, World!". My batch-job.xml <?xml version="1.0" encoding="UTF-8"?>...
Shun's user avatar
  • 19
4 votes
2 answers
177 views

Spring Batch issue with MultiResourceItemWriter and ClassifierCompositeItemWriter

I am using Spring Boot + Batch v2.7.1 in my project and looks like there is a bug when Reading from FlatFileItemReader using ClassifierCompositeItemWriter and MultiResourceItemWriter as ...
PAA's user avatar
  • 11.7k
0 votes
0 answers
11 views

How to loop a step until a condition in execution context is met?

My usecase is to fetch records from a sharded database and update. i have startstep that sets the counter and increment the loop and not sure how and where to. specify the loop and then decide if the ...
jyo rani's user avatar
0 votes
1 answer
54 views

How to update existing items during Spring batch migration?

For migration documents between different MongoDB collections, I read items via MongoCursorItemReader based on a filter and then write these data using ItemWriter as shown below. In order to prevent ...
Murat Yıldız's user avatar
-1 votes
0 answers
16 views

Spring Batch scenario, am I thinking this right?

I want to process millions of accounts. I want to read 1K accounts (pageSize) per DB read, call a REST service in processor with 20 account id's per call, then write to network storage (recommends ...
cw22's user avatar
  • 186
0 votes
0 answers
21 views

Spring Batch Asynchronous Chunk Processing With Specific Item Grouping

I'm reading messages from the database, grouping them into chunks and passing those chunks to the thread pool for processing. So, each thread picks one chunk and processes it in the scope of one ...
user18363564's user avatar
1 vote
1 answer
72 views

How to create dynamic steps in Spring Batch?

I have the following MigrationService and BatchoCnfig classes and want to pass jobParameters to the related steps using MigrationJobListener shown below. However, I am getting "Could not autowire....
Murat Yıldız's user avatar
0 votes
1 answer
20 views

How can I run jobs in Spring Batch 5.x without creating Batch Metadata entries in Database

As per Spring Batch 5.x, we have to create meta data table in our DB and because of that I am seeing lots of entries in our DB which I am trying to avoid. Is there a way I can avoid job entries in DB ?...
Dipak Mahato's user avatar
0 votes
2 answers
72 views

How to access JobParameters in Spring Batch ItemReader?

I am trying to access job parameters in an ItemReader as shown below, but when trying some the value cannot be passed or I get some errors e.g. "Failed to initialize the reader: Cannot determine ...
Murat Yıldız's user avatar
0 votes
0 answers
36 views

Spring Boot Batch with TaskExecutor OptimisticLockingFailureException

I use Spring Boot 3 (Batch 5) with Spring Data and Oracle Database. I have one Job with an Item Reader and a Writer (without a processor) Everything is working fine, Now to make job process fast, want ...
Manu's user avatar
  • 1,263
0 votes
0 answers
66 views

Spring Batch 5.1.2: BeanPostProcessor warnings after migrating from @EnableBatchProcessing to DefaultBatchConfiguration

I have a spring batch application (version 5.1.2) and was migrating from the EnableBatchProcessing-Annotation to the programatic configuration by implementing DefaultBatchConfiguration-Class. I did ...
Linde_98's user avatar
  • 428
1 vote
0 answers
22 views

How to persist h2 changes made inside Spring Batch ItemReader

My goal is that while I am processing my files, to also add them in a persistent H2 instance with their current status: "NEW" during reading, "PROCESSING" in the ItemProcessor, and ...
Q_Q_'s user avatar
  • 23
0 votes
0 answers
14 views

How to count the number of files created by composite Item Writer using Spring Batch?

I am using Spring Batch in project and using ClassifierCompositeItemWriter to classify records and then using MultiResourceItemWriterBuilder to create multiple files. In my case more than 1 files is ...
PAA's user avatar
  • 11.7k
0 votes
0 answers
50 views

Spring batch 5.x JpaPagingItemReader Adding "Top ?" to teradata query

I have a spring batch application that was running fine in spring batch 3.x, then it was upgraded to 5.x and it started to fail with: Syntax error, expected something like an integer or a decimal ...
car_dev's user avatar
0 votes
0 answers
43 views

Unable to create scheduled Tasks in SCDF - 2.11.0

I am using SCDF version 2.11.0 . Earlier I am able to create schedule tasks with CRON_TZ and it was working fine . Recently , not able to create schedule with CRON_TZ . We are getting the below error ,...
17_4088_ Subashini.S's user avatar
0 votes
1 answer
21 views

How to use declarative transaction management in spring cloud task with spring batch

Spring boot 2.5.4 My problem is that i dont't succeed in managing transactions : while i annotate my service layer and dao with @Transactionnal, when i have successive dao call in the same service ...
Damien Tacheron's user avatar
0 votes
0 answers
16 views

Dynamic procedure name on StoredProcedureItemReader

I wrote a class that extends StoredProcedureItemReader in the following way: @Component public class SpFirstReader extends StoredProcedureItemReader<StudentResults> implements ...
Vamsi's user avatar
  • 669
0 votes
0 answers
40 views

Why are Spring converters package protected?

In Spring Batch, job parameters are converted from strings to the declared type, using a JobParametersConverter. The default implementation wraps a (Spring Core) ConversionService, which actually ...
dcolazin's user avatar
  • 1,054
0 votes
0 answers
45 views

CompositeItemWriter having issue while persisting data

I wanted a single reader two processors and two writers. my processing logic is correct but data is not being persisted in DB, this is my composite writer @Bean public ...
ANUPAM's user avatar
  • 1
0 votes
0 answers
20 views

Spring Batch how to create fixed size file when reading multiple flat files?

I'd like to further extend Reading data form multiple csv file and writing it into one csv file using Spring Batch, in my case I've to read multiple files (with the same filename) from different sub-...
PAA's user avatar
  • 11.7k
0 votes
1 answer
38 views

Spring cloud data flow cannot find driver class at startup

I'm trying to up spring cloud data flow in my local. By default SCDF use h2 as primary database to store meta information. In my cloud-task-batch application all the properties related to database are ...
Arti Prajapati's user avatar
0 votes
0 answers
37 views

Caused by: org.springframework.jdbc.BadSqlGrammarException: Executing query; bad SQL grammar [SELECT * FROM employee where age in (:ageList)];

I'm using Spring Batch XML code in my project where I'm reading from MySQL DB using IN clause and writing it into CSV/CustomItemWriter etc. Getting below error, it appears that Spring Batch is not ...
PAA's user avatar
  • 11.7k
1 vote
2 answers
48 views

Get the Previous Execution time of @Schedule job in springboot

I'm looking for an option to get the last execution time of @Scheduled task. so that I want to use it for retrieving the database records from the last execution time. Quartz dependency and store the ...
Karkala Srikanth's user avatar
0 votes
0 answers
48 views

How to pass list in SQL query - Spring Batch Caused by: java.sql.SQLException: Parameter index out of range (2 > number of parameters, which is

Before posting the question - I went through below URLS, but it was clear enough to me Using SpringBatch JdbcCursorItemReader with List as NamedParameters How to set list of parameters on prepared ...
PAA's user avatar
  • 11.7k
0 votes
2 answers
35 views

Why I am getting this error? - SLF4J: Class path contains multiple SLF4J bindings

Error :- SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/Users/rm694/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.3/...
Roshan M's user avatar
0 votes
0 answers
63 views

spring batch 5: how to NOT use the datasource?

I am migrating my spring boot spring batch application from spring batch 4.x to 5.1.1 (this is part of spring boot 2.7.18 to 3.2.4 migration). In our batch application we do not use any spring batch ...
Sachin Bisht's user avatar
0 votes
1 answer
61 views

Spring Batch - read date field as a String in readers and convert it to other format

Before posting the question, I went through many links like: Using Spring Batch to parse date from file into LocalDateTime and Spring Batch - how to convert String from file to Date? etc but that ...
PAA's user avatar
  • 11.7k
0 votes
0 answers
54 views

Loading data from file to database, file has field which is primary key, how to implement hibernate bulk inserts. can't have auto generated Id

Entity looks like this. @Entity @Table(name = "TRANSACTION") @Data @AllArgsConstructor @NoArgsConstructor public class Transaction { @Id @Column(name = "ACCOUNT") private String ...
Ramesh N's user avatar
1 vote
0 answers
69 views

Data Migration of multiple tables from one database to other using Spring Boot and Spring Batch

I want to migrate from one database to other using Spring Batch and Spring Boot. I have learnt that we have to declare @Bean for step, Job, JobLauncher, ItemWriter, ItemReader. But for those beans to ...
XPulse's user avatar
  • 11
1 vote
0 answers
23 views

Spring batch read completion indicator

Requirement: Read multiple parquet files from a location Find invalid records, store them in-memory and write them to a file at the end of the batch Writer writes valid data to the database Unsure ...
DS.Sun's user avatar
  • 11
0 votes
1 answer
33 views

Spring Batch - Using ClassifierCompositeItemWriter - how to avoid writting headers when file is empty (no data)

I went through Is there a way to override and not execute the headerCallBack in Spring Batch, but still not clear on the solution. I'm using Spring XML based approach in my project. I'm using ...
PAA's user avatar
  • 11.7k
0 votes
0 answers
25 views

Type Mismatch while using FlatFileItemReader BeanWrapperFieldSetMapper from String to Date

I am trying to batch process a file which has date field. Now when I fetch Date field using FlatFileItemReader it comes as a String. I am trying to map it to following entity @Entity @Data public ...
Gaurav Matta's user avatar
0 votes
0 answers
19 views

Spring Batch - Using Custom Classifier how to create two more files

I'm using Spring Batch XML for my project and have below requirement to fullfill. My task is connect Sybase DB, read all records from the table Based on clientId (there are 10 different clientId), ...
PAA's user avatar
  • 11.7k

1
2 3 4 5
172