Questions tagged [spring-cloud-task]

Spring Cloud Task allows a user to develop and run short lived microservices using Spring Cloud and run them locally, in the cloud, even on Spring Cloud Data Flow. Just add @EnableTask and run your app as a Spring Boot app (single application context).

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Restarting only failed partitions of a Cloud Task in Spring Cloud Data Flow on K8s platform

We are using Spring Cloud Data Flow running in Kubernetes platform. Sometimes when we run large number of partitions (pods), some partitions are failing. We want to restart those failed tasks only. ...
Madhu's user avatar
  • 31
0 votes
1 answer
42 views

Spring Cloud Data Flow 2.10.3 : may i launch a Task linked to a specific application version, instead of default version

I'd like to run a Task (or create a new Task and run it) whom execution will trigger deployment and execution of a specified (on task level) version of the linked application. I don't find how to do, ...
Damien Tacheron's user avatar
0 votes
0 answers
23 views

Spring cloud task(composed Task) restart from the failed job

I have a spring cloud composed task which has nearly 10 jobs . If any of the job fails from the composed task, I have to continue the flow from the failed job. For Eg: After launching the composed ...
17_4088_ Subashini.S's user avatar
0 votes
0 answers
42 views

SCDF Task Kubernetes Persistent Volume attachment

I have created a simple spring cloud task project, which will generate a file and add to local folder. I am using SCDF(Spring cloud data flow) and kubernetes for deployment. I need to generate file to ...
ThunderBoy's user avatar
0 votes
1 answer
23 views

how to track Spring Cloud Task execution results

I am looking at using Spring Cloud Task for offloading some some asynchronous jobs that need to be executed. I want to be able to track the results of these so the originating application can report ...
Adam Skwersky's user avatar
0 votes
1 answer
224 views

Spring boot cloud task without springs task tables

I've an application that run a task that parses a file and persist the data in the database. After that it shutsdown gracefully. I can see that spring automatically creates 4 tables and populates them ...
user2354898's user avatar
0 votes
1 answer
81 views

An error occurred when running the spring cloud task batch app

An error occurred when running the spring cloud task batch app ERROR : Statement.execute(CREATE TABLE TASK_EXECUTION (...) FAILED! org.postgresql.util.PSQLException: exceptioin: "task_execution&...
이상빈's user avatar
0 votes
1 answer
305 views

Spring cloud task in dataflow dashboard is complete but one partition of spring batch failed

I have a simple spring batch remote partitioning project running with spring cloud task. Im using spring cloud dataflow at run the spring batch. Spring boot version - 2.7.8 Spring cloud task - 2.4.5 ...
Sanjay's user avatar
  • 11
0 votes
2 answers
223 views

The import org.springframework.cloud.task.Task cannot be resolved

Trying to write a custom task in Spring Cloud Dataflow which will create a Spring Batch application that implements the Task interface provided by the Spring Cloud Task. I have try all possible ways ...
Santosh Mallah's user avatar
0 votes
1 answer
68 views

Set SCDF CTR to execute next task immediately when step is complete

I'm using SCDF 2.9.6, which is mariadb 10.3.x(used in my company, cannot update version) compatible with flyway. Compiling SCDF 2.10.x project with lower flyway dependency resulted whole compile ...
Carte's user avatar
  • 21
0 votes
1 answer
824 views

Spring Cloud Dataflow with tasks based on Spring Boot 3 and Spring Batch 5

We are migrating Spring Batch applications based on Spring Boot 2.7.x to Spring Boot 3.0.0. With that migration we noticed changes in versions from Spring Batch 4.3.7 to Spring Batch 5.0.0 and Spring ...
Jochen Reinhardt's user avatar
0 votes
1 answer
422 views

Migrating H2 configuration in Spring Batch for Spring Boot 2.7

I need to migrate a project from Spring Boot 2.5.3 to 2.7.3 (and, if significant, also Spring Cloud from 2020.0.3 to 2021.0.3 and Spring Batch from 4.3.3 to 4.3.6). The main issue is that when using ...
dcolazin's user avatar
  • 1,026
1 vote
1 answer
224 views

Spring cloud Data Flow Task sample python task not working

I am trying to do the sample python task: Install local with docker: https://dataflow.spring.io/docs/installation/local/docker/ Then proceed to create python task: https://dataflow.spring.io/docs/...
Bill Mol's user avatar
0 votes
1 answer
72 views

Spring Cloud Task-Batch - taskBatchDao.saveRelationship doesn't write data

I have a small spring-bach application and deploy it locally in scdf. It works fine so far, taskexecutions run and the metadata tables are filled. Except for one - TASK_TASK_BATCH! I see the log ...
dgerhardt's user avatar
1 vote
0 answers
15 views

If I kill spring cloud task jar it is not reflecting in SCDF dashboard

We triggered spring cloud task using SCDF and abnormally if i kill the running jar which triggered for SCDF. In SCDF Ui still it is running state is there any way we can get the failed status in SCDF
Abhi Ram's user avatar
1 vote
0 answers
78 views

How can we stop spring cloud task in spring cloud data flow?

I am trying to stop a running task in spring cloud data flow , but the spring batch application didn't get terminated and in spring cloud data flow server task status it is still running. Is there any ...
Abhi Ram's user avatar
0 votes
1 answer
672 views

Is the latest Spring cloud task incompatible with spring-boot 2.6.6?

I am trying to use the spring-cloud-task-core version 2.4.1 with the spring-boot version 2.6.6 (from the 2021.0.x release train). This is showing me the below error: Your project setup is incompatible ...
Jatin Kheradiya's user avatar
0 votes
0 answers
239 views

Launch spring cloud task with JVM args via DeployerPartitionHandler

I am planning to execute spring batch job on Pivotal cloud Foundry. The job executes fine on a single JVM with multiple threads(Local partitioning). I am looking to scale the job and the first option ...
GVR's user avatar
  • 320
0 votes
0 answers
283 views

Spring Batch - Job executions launching with same Task execution id

I'm using Spring Batch with Spring Cloud Task for remote partitioning. But for each new Job execution it is created with the same task execution id. Is there any way to create a new task execution id ...
dave's user avatar
  • 19
0 votes
1 answer
100 views

Does Spring Cloud Task supports file listeners?

I have implemented Batch processing with Spring Batch Integration. I’ve achieved the best solutions for file listeners which is way better than scheduler (cron). Now we are migrating to Spring Cloud ...
Eddy Bayonne's user avatar
  • 2,518
2 votes
0 answers
202 views

Spring Batch very slow when using 2 datasources - one for Spring Batch and another for the App

I modified this sample batch job provided by spring to use two custom datasources instead of the one autoconfigured by boot. Both datasources point to the same MySql DB server, but to different ...
vrajkuma's user avatar
1 vote
1 answer
284 views

Helm value not interpreted

I'm trying to set an application property to a spring value in a helm values file. But when I deploy this helm chart to kubernetes, I can see the value is not replaced. I would like to set the ...
Vin's user avatar
  • 11
1 vote
0 answers
165 views

Spring Cloud Data Flow - Handling exceptions before CommandLineRunner

Let's say that we have a Composed Task in Spring Cloud Data Flow defined: JobA && JobB JobA needs to write data to an external database, so I am using 2 datasource configurations. One for ...
Tomas Lukac's user avatar
  • 1,993
0 votes
0 answers
348 views

Spring Cloud Data Flow - Partition Batch Job using Spring Cloud Kubernetes Deployer : Environment properties not getting passed to worker pods

I modified the dataflow sample app partitioned-batch-job to deploy it in a kubernetes cluster via the SCDF server that is running in the cluster. I used the dashboard UI to launch this app as a task. ...
vrajkuma's user avatar
1 vote
0 answers
235 views

Spring Batch task is getting shutdown before RSocket proxy scrapes metrics from it

I am using Spring Cloud Dataflow to launch batch jobs that are wrapped in Spring Cloud task and we also have configured Prometheus and Rsocket proxy to scrape task related metrics. To make the tasks ...
Kranthi's user avatar
  • 37
0 votes
1 answer
343 views

Spring Cloud Data Flow: Docker URI error when running the sample app 'partitioned-batch-job' in Kubernetes cluster

I modified the dataflow sample app partitioned-batch-job to deploy it in a kubernetes cluster via the SCDF server that is running in the cluster. I use the dashboard to launch this app as a task. The ...
vrajkuma's user avatar
0 votes
0 answers
411 views

spring batch and spring cloud starter task : expected single matching bean but found 2: transactionManager, springCloudTransactionManager

I'm using spring cloud starter task and deployerPartitionHandler for spring batch remote partitioning job. Now, When I'm trying to stop a running job using jobOperator.stop() method spring batch is ...
dave's user avatar
  • 19
0 votes
1 answer
235 views

deploymentServiceAccountName does not work for schedules in spring cloud dataflow

I am using spring cloud dataflow for spring batches(with remote partitioning) wrapped with spring cloud task, deployed in kubernetes. I have also created a service account for the same. The batch runs ...
Kranthi's user avatar
  • 37
1 vote
1 answer
316 views

Spring batch worker pods are unable to pick custom service account for spring cloud deployer kubernetes

I am trying to run a spring batch with remote partitioning on K8s cluster using spring-cloud-deployer-kubernetes. Eventhough I have configured a service account and mentioned in my application ...
Kranthi's user avatar
  • 37
0 votes
1 answer
221 views

Spring batch worker pod logs are not visible in spring cloud dataflow

I am trying to run a spring batch with remote partitioning on K8s cluster. Every thing works fine regarding the partitioning, execution etc. The same is integrated with Spring Cloud Dataflow and can ...
Kranthi's user avatar
  • 37
1 vote
0 answers
1k views

Spring Batch pass data from writer to next job or step

I'm fairly new to Spring Batch and I would like to know if it is possible to pass data saved by writer of one job to next job or save in process and pass to next step. I've found similar question here ...
Sujal's user avatar
  • 671
2 votes
1 answer
1k views

Spring Batch with multi - step Spring Cloud Task (PartitionHandler) for Remote Partition

Latest Update (with an image to hope simplify the problem) (thanks for feedback from @Mahmoud) Relate issue reports for other reference (after this original post created, it seem someone filed issues ...
Danil Ko's user avatar
0 votes
1 answer
294 views

Spring Cloud Dataflow - Parallel Tasks

I have about 16 tasks configured in parallel like <AAA && BBB|| CCC && DDD || EEE && FFF || GGG && ......>. My intention is to only have 3 tasks running at one ...
Raicky Derwent's user avatar
1 vote
0 answers
318 views

How to launch a Spring Batch Remote Partitioning with Spring Cloud Dataflow

I am trying to launch a Spring Batch Remote Partitioning with Spring Cloud Dataflow. I have a remote partitioning that is based on https://github.com/mminella/LearningSpringBatch/tree/master/src/...
msuzuki's user avatar
  • 125
0 votes
1 answer
417 views

Spring Cloud Data Flow: Error org.springframework.dao.InvalidDataAccessResourceUsageException

I am trying to run/configure a Spring Data Cloud Data Flow (SCDF) to schedule a task for a Spring Batch Job. I am running in a minikube that connects to a local postgresql(localhost:5432). The ...
msuzuki's user avatar
  • 125
0 votes
1 answer
585 views

Spring cloud data flow task Invalid TaskExecution ID

I am just trying to create a hello world SCDF task. It is my understanding that out of the box the task should be able to read the needed data from the h2 database by just including h2 in the pom, but ...
Jeff I's user avatar
  • 396
-1 votes
1 answer
1k views

Spring batch - Could not commit JDBC transaction; nested exception is java.sql.SQLTransactionRollbackException

I am using spring cloud data flow 2.7.1 with mysql 5.7.24 which executes a task with job related. the architecture used is Externalizing Batch Process Execution - remote partitioning (manger,worker) ...
Ivan Gallo's user avatar
0 votes
1 answer
256 views

Does Spring cloud task handle auto closing of the Kubernetes pods when the worker node has completed [duplicate]

I am writing the Spring batch app on AWS cloud and in order to ensure efficient utilization of resource , I had this question that "Does Spring cloud task handle auto closing of the Kubernetes ...
Ankit Gautam's user avatar
1 vote
0 answers
148 views

Spring Cloud Data Flow : Custom DDL for Data Flow Server

I have setup a Spring Cloud Data Flow server on Kubernetes backed by an in-memory h2 database running on Kubernetes as well. I want to change the length of the columns in some of the tables that are ...
Ping's user avatar
  • 607
0 votes
1 answer
216 views

Spring Cloud Data Flow : Value too long for column "TASK_NAME" (DeployerPartitionHandler with Spring Batch)

I have a simple Spring Batch Job running Kubernetes as a Spring Cloud Task. This job uses Spring Batch Partitioning to further launch partitioned steps as task pods on the same Kubernetes cluster. ...
Ping's user avatar
  • 607
0 votes
1 answer
653 views

Spring Cloud Data Flow : Unable to launch multiple instances of the same Task

TL;DR Spring Cloud Data Flow does not allow multiple executions of the same Task even though the documentation says that this is the default behavior. How can we allow SCDF to run multiple instances ...
Chetan Kinger's user avatar
0 votes
0 answers
238 views

SCDF + Spring Batch : JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={run.id=1}

SCDF is launching job as: Command to be executed: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/java -jar /opt/scdf-batches/foo/foo-1.0.26.jar --spring.cloud.task.executionid=23 ...
e6573's user avatar
  • 55
0 votes
1 answer
231 views

Non-option arguments with Spring Cloud Data Flow

I'm trying to run an application on Spring Cloud Data Flow that requires a non-option argument (e.g. the command would be java -jar task.jar non-option-argument --optionArgument=option etc.) The ...
bmarcj's user avatar
  • 51
0 votes
1 answer
157 views

How to use static spring cloud stream url for launching spring cloud tasks?

Platform used : Kubernetes. I have an issue with Spring cloud stream url. I am launching my spring cloud tasks using spring cloud stream. Streams are deployed in kubernetes platform. Stream contains ...
pvrforpranavvr's user avatar
1 vote
0 answers
508 views

Spring Batch 'jobs must not be null nor empty' Error

I'm running Spring batch apps on Openshift using SCDF. I'm getting error during Spring Batch starting-up like below. What is the main reason of that? 2021-03-08 08:39:50.764 ERROR [ark-batch-template,,...
Umut's user avatar
  • 23
0 votes
0 answers
3k views

Consider defining a bean of type 'org.springframework.transaction.jta.JtaTransactionManager' in your configuration

I am developing a spring cloud task application using spring cloud task and spring batch libraries. I also created a spring boot library containing spring cloud task and spring batch configurations ...
springdeveloper's user avatar
2 votes
1 answer
753 views

Task execution is not working after lunching the task in spring cloud data flow

I have created one Spring boot application with @EnablesTask annotation and try to print the arguments in log. package com.custom.samplejob; import org.springframework.boot.CommandLineRunner; import ...
Pyla Srenu's user avatar
0 votes
1 answer
306 views

Automatic job restart

I have a job that can take up to several hours. It is possible that for some reason (like out of memory, or cluster rebalance) it just fails. The problem is that the job is usually run overnight, and ...
Michał Dębski's user avatar
1 vote
1 answer
602 views

How can I design two different projects with a single entity or a single repository?

We are developing two projects with two different teams. However, there are some common points in the two projects, for example, there are common entities and repository usage cases. How can I manage ...
user avatar
0 votes
0 answers
169 views

Step ExecutionContext not promoted using Spring Cloud Task on Spring Cloud Data Flow

I successfully deployed a remote partitioned job using Spring Cloud Data Flow and Spring Cloud Task; the installation is based on Kubernetes, so I added the Kubernetes implementation of Spring Cloud ...
Claudio Tasso's user avatar

1
2 3 4 5 6