How are we doing? Please help us improve Stack Overflow. Take our short survey

Questions tagged [spring-batch]

Spring Batch is a lightweight, comprehensive framework aimed at enabling the development of batch applications vital for the daily operations of enterprise systems. Batch applications in this context refers to automated offline systems targeted towards bulk data processing.

Filter by
Sorted by
Tagged with
0 votes
1 answer
30 views

spring batch RepositoryItemWriter doesnt write data to database

@Configuration public class DataSourceConfig { @Bean @ConfigurationProperties("spring.datasource") public DataSource appDataSource() { DataSourceBuilder builder = ...
Buddhima Chamaranga's user avatar
0 votes
0 answers
5 views

Read from Sybase Database and write to Postgres SQL

I need assistance in knowing whether Spring Batch is correct way of implementing a functionality in which I want to read a table from Sybase and then convert / transform it to the target persistence ...
user3520512's user avatar
0 votes
1 answer
54 views

Spring batch using two data Sources

I'm working on a Spring Batch project where I need to use two different data sources. I'm planning to use RepositoryItemReader and RepositoryItemWriter for reading and writing to these databases. ...
Buddhima Chamaranga's user avatar
0 votes
1 answer
12 views

Sprng batch stepscope reader test with Mockito - org.springframework.batch.item.ReaderNotOpenException: Reader must be open before it can be read

I am testing @stepscope JDBCCUrsorIteamReader but getting below error. I am opening the reader before start reading but still its giving reader is not opened. I want to mock database calls. How can i ...
Pinki Sharma's user avatar
0 votes
1 answer
49 views

How to still run a step spring batch even when an exception occurs

I have a spring batch program where execution flows of various steps are defined via xml files. One of these xml files is structured something like this. <step id="step1" next="step2&...
Luigi's user avatar
  • 183
0 votes
1 answer
23 views

Spring-Batch does not open FlatFileItemWriter if bean is declared as ItemWriter

I'm using Spring Batch 4.3.8 and I'm trying to use the following code in my configuration to define a ItemWriter @Bean @StepScope public ItemWriter<MyData> itemWriter(MyFileConfig fileConfig) { ...
dcolazin's user avatar
  • 992
-1 votes
0 answers
23 views

How are Spring Batch jobs shared between instances of the same application? [closed]

I have an application, of which 3 instances are running. Each one is using Spring Batch, with a job defined with a ThreadPoolTaskExecutor configured with a pool size of 2 threads. Every instance is ...
Onny J's user avatar
  • 1
0 votes
1 answer
31 views

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'javax.sql.DataSource' - Spring Batch Junit5

I was trying to write Junit for the Spring Batch project and was referring to: https://docs.spring.io/spring-batch/docs/4.3.x/api/org/springframework/batch/test/context/SpringBatchTest.html, but ...
Jeff Cook's user avatar
  • 8,076
0 votes
1 answer
56 views

No qualifying bean of type 'org.springframework.batch.core.configuration annotation.JobBuilderFactory' available: in Junit

I went through below few links already, but did not turn to be working, I'm using Spring Boot v2.7.1 and Batch and has below code and using junit 5. https://docs.spring.io/spring-batch/docs/4.0.x/...
Prateek's user avatar
  • 10.7k
-1 votes
0 answers
15 views

How to avoid calling the Reader constructor of step 2 before step 1 is completed?

Reader1 reads from a .csv file, Processor1 generates a different output, and Writer1 stores it in the RECORD table, but also stores the file in the FILE table with an autogenerated ID as the primary ...
Daphne's user avatar
  • 51
0 votes
1 answer
31 views

Spring Batch Job Testing with Mockito

I am new to spring batch application. I am writing test cases for spring batch jobs using Junit & mockito.I tried to write the test cases with in memory DB but we are using some oracle specific ...
Pinki Sharma's user avatar
0 votes
1 answer
21 views

JdbcPagingItemReader: questions about async/sync behaviour

i have a question about JdbcPagingItemReader: When we initialize the QueryProvider the method setSortKeys is is recommended to be used with an unique key constraint to guarantee that no data is lost ...
thefiery77's user avatar
0 votes
1 answer
35 views

Spring batch processing for data migration of a workflow

When performing data migration, it often involves a workflow where data needs to be extracted from multiple repositories, transformed, and then saved to new repositories. One approach to tackle this ...
Buddhima Chamaranga's user avatar
0 votes
0 answers
24 views

PreparedStatementCallback; bad SQL grammar [INSERT INTO BATCH_JOB_EXECUTION_PARAMS(JOB_EXECUTION_ID, PARAMETER_NAME, [duplicate]

I'm working on migration Spring Batch 5. I'm using Jave 17 and Spring Boot 3. I didn't have any issue on step and job. However, In spring batch 5 DefaultBatchConfigurer is removed. setting datasource ...
Arsenal's user avatar
0 votes
1 answer
19 views

Can I use KafkaItemReader in Spring Batch to read from multiple topics?

It's easy to use KafkaItemReader to read messages from one topic: return new KafkaItemReaderBuilder<String, Activity>() .consumerProperties(toProperties(of( ProducerConfig....
banan3'14's user avatar
  • 3,914
0 votes
1 answer
21 views

SkipListener does not work with AsyncItemProcessor, AsyncItemWriter in spring boot 3 & java 17

I'm writing a Spring Boot application that starts up, read->process->write millions file records into a new file. I'm attempting to use Spring Batch for this and using AsyncItemProcessor, ...
snshukla's user avatar
0 votes
1 answer
32 views

Sprint batch fault tolerance issue

I'm using spring batch for my project, my requirement is to write the error record in a file. I used spring batch fault tolerant and skip policy like any exception can be skippable. What my issue is i ...
tarun singh's user avatar
0 votes
1 answer
62 views

First Spring Batch implementation, not executing Reader, Processor and Writer

I'm currently trying to learn spring batch, I did a first implementation, but the code inside Reader, Processor and Writer is not executed. Any advice? JustBatchApplication : @SpringBootApplication ...
Paul Marcelin Bejan's user avatar
0 votes
1 answer
24 views

Before / After steps not called in Item processors of ClassifierCompositeItemProcessor

Here is the sample code block (I remove our stuff and kept only the skeleton code) public class ItemProcessor1 implements ItemProcessor{ private StepExecution stepExecution; @...
Ramkumar S's user avatar
0 votes
1 answer
32 views

JdbcPagingItemReader: PagingQueryProvider add a parameter value after the second iteration

i created a class that extends JdbcPagingItemReader. Inside i have this configuration. @PostConstruct public void init() throws Exception { setDataSource(dsSrv.getUserDataSource()); ...
thefiery77's user avatar
0 votes
0 answers
25 views

Spring Cloud Dataflow - Composite Task Runner Issue

I'm exploring all the features of Spring Cloud Dataflow for our project. Currently I'm doing all development locally(No docker, Cloudfoundry & Kubernates) Currently I have created 2 Spring Batch ...
iCode's user avatar
  • 8,902
0 votes
0 answers
24 views

Tomcat High CPU usage and OutOfMemory issue

Application Stack: Ubuntu 18.04.6 LTS JDK 1.8.0_201 Spring Boot: 2.6.6 Oracle 12c Tomcat 8.5.31 After I have added a batch (runs every 6 hours) which creates 6 excel files (max file size: 12MB) from ...
Tusar's user avatar
  • 759
-1 votes
0 answers
17 views

Spring Batch (JdbcCursorItemReader) [duplicate]

I am Using JdbcCursorItemReader in my spring batch project, JdbcCursorItemReader is working absolutely fine when I am using it without 'TaskExecutor' but when I am giving 'TaskExecutor' in step config ...
Ayush Gupta's user avatar
0 votes
1 answer
20 views

How to implement custom controller that will allow me to stop a step in Spring Batch

Currently the Spring Batch framework does not support an official method to pause a step. This presents a challenge for me and I was hoping to get some insight into how I could go about configuring ...
Jacob Goldverg's user avatar
0 votes
0 answers
17 views

java.lang.NoSuchFieldError: BLOCK_UNSAFE_POLYMORPHIC_BASE_TYPES Issue when upgrading spring batch 3.0.10 to 4.3.8

I am using spring-batch-core version 4.3.8 and jackson-databind version 2.13.4.2. Application build is successful, when I deploying it in jboss eap version 7.1.1 getting below errors. Caused by: java....
kavi's user avatar
  • 1
1 vote
1 answer
32 views

SpringBatch Teradata DatabaseType support

I'm using SpringBatch 5 with Teradata db , I tried to create a job repository to add a datasource but i get an exception about org.springframework.batch.support.DatabaseType : Failed to instantiate [...
Yaz's user avatar
  • 503
-1 votes
1 answer
30 views

Spring Batch with mongodb issue

I'm trying to poc spring batch with mongodb with simple logic. Read csv file Insert data to MongoDB Question: Do i need relational database such as: Mysql for jobRepository? If relational database ...
Rahat Sarawasee's user avatar
0 votes
1 answer
32 views

spring batch read and write same table

I’m using spring batch to read chunk data in a table, send event and when event is send change boolean value in writer. exemple: My reader (RepositoryItemReader) call a JpaRepository that retrieve all ...
Mançaux Pierre-Alexandre's user avatar
0 votes
1 answer
38 views

Scaling Spring Batch remote partitioning with kafka

I need to scale spring batch job by remote partitioning with kafka. I want to run manager step on one VM and worker steps on different VMs. For that I have some questions - If my manager VM instance ...
ashok's user avatar
  • 3
0 votes
0 answers
15 views

DBAs not allowing to set SERIALIZED_CONTEXT as clob for Spring Batch

We are planning to use Spring Batch is our project. As per metadata schema, we need to get some tables created, BATCH_JOB_EXECUTION_CONTEXT and BATCH_STEP_EXECUTION_CONTEXT. In these table we need to ...
Amardeep Pawar's user avatar
0 votes
0 answers
17 views

Spring Batch : Existing transaction detected in JobRepository

I get this trace when i run the code below : 2023-09-08T16:32:38.217+02:00 ERROR 26472 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Encountered an error executing step step in job ...
sman_steel's user avatar
0 votes
1 answer
17 views

Spring Batch, how to skip an entire Job based on a condition

I have a job defined in spring batch like this @Bean public Job myJob(JobBuilderFactory jobBuilderFactory, Step firstStep, Step secondStep, Step thirdStep) { return jobBuilderFactory....
Luca Godano's user avatar
0 votes
1 answer
31 views

Spring Batch - not processing all steps

I have a Sping Batch job with 8 steps that synchronizes 2 databases. The job runs in a Docker container on Apache Mesos. It has been running fine for a very long time until last month when I started ...
Tangrea's user avatar
0 votes
0 answers
12 views

Spring Batch -- Data not getting promoted to future Step

a data i'm putting in the JobExecution map is not getting promoted to a future Step. This is the XML file: <batch:job id="job1"> <step id="step1"> <batch:...
waffledood's user avatar
0 votes
1 answer
32 views

Spring Batch - Promoting multiple pieces of data with ExecutionContextPromotionListener

I have a Spring Batch project where I will need to save 2 keys in the Step ExecutionContext, which will be promoted to the Job ExecutionContext. I understand from online examples, e.g. this code where ...
waffledood's user avatar
0 votes
1 answer
32 views

Temporising a Spring Batch Job

I want to make a spring batch job temporised. That is, I have one CSV file with more than 100 000 records. I want to read this file in chunks each day. For example first 100 records on the first day, ...
Arjun Koroth's user avatar
1 vote
1 answer
55 views

Testing SpringBatch job via independent JUnit 5 based test

My Spring Boot application invoking a sample Spring Batch job is configured as below: MainApplication.java: @SpringBootApplication public class MainApplication { public static void main(...
Simeon Leyzerzon's user avatar
-2 votes
1 answer
48 views

How to solve "The class with java.io.File and name of java.io.File is not trusted." Spring batch test error?

I have a spring batch application and I am writing test cases for it. The application runs without any error. I could also unit tests the Steps in the application without any problem. However when I ...
Jeet_dev's user avatar
0 votes
1 answer
38 views

How to stop Spring Batch job Executing automatically

@ComponentScan({ "com." })@Configuration@EnableBatchProcessing public class InterestDateUpdationConfiguration {@Autowired private DataSource dataSource; @Autowired private ...
vaibhav burgute's user avatar
0 votes
2 answers
73 views

What is the difference between chunk size and partition size in spring batch?

What is the difference between chunk size and partition size in spring batch? I am not referring to spring batch partitioning which is explained briefly here. I am referring to DEFAULT_PARTITION_SIZE ...
CodeBot's user avatar
  • 49
1 vote
0 answers
31 views

Spring Batch not making full use of multi-core CPU

I have a spring batch application which reads lines from a flat file, converts each line to an object, processes them, and writes them to a database. The steps are multi-threaded using spring's ...
ethereal's user avatar
1 vote
1 answer
28 views

Expose spring batch workflow steps/tasks in advance from an api

For a spring batch workflow using java dsl, for example: @Bean public Job myWorkFlow() { return jobBuilders.get("myWorkFlow") .start(myFlowA.doStepA()) .next(...
user518066's user avatar
  • 1,287
0 votes
1 answer
44 views

spring-batch sample: why call the constructor method directly?

I'm looking at the samples for spring-batch and I'm surprised to see a bean constructor method being called directly rather than through a bean reference. I'm wondering if this is an error. In https:/...
al.truisme's user avatar
0 votes
1 answer
37 views

How to Parallelize Web Service Calls in a Spring Batch Job

I have a scheduled Spring Batch job that reads records from a PostgreSQL database. For each record, I make a web service call and store the response back into the database. Each web service call takes ...
SoxoS's user avatar
  • 1
-2 votes
0 answers
45 views

retry.RetryException: Non-skippable exception in recoverer while processing; nested exception is java.lang.OutOfMemoryError: Java heap space

I have an step configuration as follow: stepBuilderFactory.get("step-migracion-documentos-sin-procesar") .<ContenidoDocPendienteVW, ContenidoDocMigrado>chunk(1).reader(...
JBallester's user avatar
0 votes
1 answer
11 views

Open AWS signed URL with FlatFileItemReader

In Spring Batch I am trying to read a file with signed URL from AWS S3 This URL has the following parameters : X-Amz-Algorithm=...&X-Amz-Date=...X-Amz-SignedHeaders=...&X-Amz-Expires=...&X-...
pi lak's user avatar
  • 67
0 votes
1 answer
23 views

Spring batch - processing a subset of multiple items at once

I have a list of numerous items that I split into chunks of 10 each. But the itemProcessor only processes 1 by 1 item in a chunk. How can I aggregate all items in each chunk into a list and then make ...
The Anh Nguyen's user avatar
0 votes
0 answers
35 views

Why does SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=? returns nothing?

I am testing a spring batch application witht he help of the Spring batch testing. I have created the tables that the spring batch test use manually in the h2 database. However when test launching a ...
Jeet_dev's user avatar
0 votes
1 answer
38 views

Best way to share data between steps in Spring Batch?

I'm developing a job that reads a .csv file and stores each record in a database. To do this, I need to use the configuration that I get from the database. As I don't want to be retreiving the same ...
Daphne's user avatar
  • 51
0 votes
1 answer
46 views

BigQuery for Spring batch Reader instead of RDMS

I am new to SpringBatch trying to use Biqquery in spring batch jobs, Itemreader to read the date from BQ table instead of other RDBMS? Trying to build a spring-boot batch application to read from BQ ...
Ashok's user avatar
  • 3

1
2 3 4 5
168