Questions tagged [spring-cloud-task]
Spring Cloud Task allows a user to develop and run short lived microservices using Spring Cloud and run them locally, in the cloud, even on Spring Cloud Data Flow. Just add @EnableTask and run your app as a Spring Boot app (single application context).
254
questions
0
votes
0
answers
8
views
Spring-cloud-task with spring-cloud-sleuth - common traceId
I have a number of microservices that intercommunicate via rabbit queues. One of the services then triggers the launch of a spring-cloud-task.
I am using spring-cloud-sleuth to propagate a common ...
0
votes
0
answers
20
views
Spring cloud dataflow: Issue with deploying stream with square brackets in the property key
I have a spring cloud dataflow app test-app that take an array from properties
@Data
@ConfigurationProperties(prefix = "test")
public class Config {
private List<String> ids;
}
I'm ...
1
vote
0
answers
7
views
If I kill spring cloud task jar it is not reflecting in SCDF dashboard
We triggered spring cloud task using SCDF and abnormally if i kill the running jar which triggered for SCDF. In SCDF Ui still it is running state is there any way we can get the failed status in SCDF
1
vote
0
answers
15
views
How can we stop spring cloud task in spring cloud data flow?
I am trying to stop a running task in spring cloud data flow , but the spring batch application didn't get terminated and in spring cloud data flow server task status it is still running.
Is there any ...
0
votes
1
answer
126
views
Is the latest Spring cloud task incompatible with spring-boot 2.6.6?
I am trying to use the spring-cloud-task-core version 2.4.1 with the spring-boot version 2.6.6 (from the 2021.0.x release train).
This is showing me the below error:
Your project setup is incompatible ...
0
votes
0
answers
63
views
Launch spring cloud task with JVM args via DeployerPartitionHandler
I am planning to execute spring batch job on Pivotal cloud Foundry. The job executes fine on a single JVM with multiple threads(Local partitioning). I am looking to scale the job and the first option ...
0
votes
0
answers
67
views
Spring Batch - Job executions launching with same Task execution id
I'm using Spring Batch with Spring Cloud Task for remote partitioning. But for each new Job execution it is created with the same task execution id. Is there any way to create a new task execution id ...
0
votes
1
answer
38
views
Does Spring Cloud Task supports file listeners?
I have implemented Batch processing with Spring Batch Integration. I’ve achieved the best solutions for file listeners which is way better than scheduler (cron). Now we are migrating to Spring Cloud ...
0
votes
0
answers
46
views
Spring Batch very slow when using 2 datasources - one for Spring Batch and another for the App
I modified this sample batch job provided by spring to use two custom datasources instead of the one autoconfigured by boot. Both datasources point to the same MySql DB server, but to different ...
0
votes
0
answers
144
views
SCDF unable to read application properties from image in local registry: Error: "Could not find a registry configuration for <private-reg:port>"
Running SCDF in a local bare metal kubernetes cluster. Able to register and run task apps from both the public and private docker registry (at hub.docker.com). I re-tagged and pushed the images to a ...
0
votes
0
answers
78
views
spring cloud task does not exit after spring batch job completion when run on PCF
I am trying to run a task via pcf using cf run-task APP_NAME java-buildpack/open_jdk_jre/bin/java org.springframework.boot.loader.JarLauncher --arg0 --arg1
I can see the from the logs that the job is ...
1
vote
0
answers
122
views
Helm value not interpreted
I'm trying to set an application property to a spring value in a helm values file. But when I deploy this helm chart to kubernetes, I can see the value is not replaced. I would like to set the ...
1
vote
0
answers
47
views
Spring Cloud Data Flow - Handling exceptions before CommandLineRunner
Let's say that we have a Composed Task in Spring Cloud Data Flow defined:
JobA && JobB
JobA needs to write data to an external database, so I am using 2 datasource configurations.
One for ...
0
votes
0
answers
93
views
SCDF - Partition Batch Job using Spring Cloud Kubernetes Deployer : Deployer properties not used when deploying worker pods
I modified the dataflow sample app partitioned-batch-job to deploy it in a kubernetes cluster via the SCDF server that is running inside the cluster.
Able to run the task via the SCDF dashboard UI no ...
0
votes
0
answers
29
views
Default properties and arguments in Spring Cloud Data Flow Task
It is possible to create Task with default arguments/properties? For example I would like create 4 task with 4 different config map. I know that it is possible to pass config map name when I launch ...
0
votes
0
answers
15
views
Fetch task name from cloudfoundry to save in logger table
I am running several spring cloud tasks in PCF and I want to save below name which are being generated everytime task runs. In below example it is 339044ef, 8d0618cf.
0
votes
0
answers
145
views
Spring Cloud Data Flow - Partition Batch Job using Spring Cloud Kubernetes Deployer : Environment properties not getting passed to worker pods
I modified the dataflow sample app partitioned-batch-job to deploy it in a kubernetes cluster via the SCDF server that is running in the cluster. I used the dashboard UI to launch this app as a task. ...
1
vote
0
answers
95
views
Spring Batch task is getting shutdown before RSocket proxy scrapes metrics from it
I am using Spring Cloud Dataflow to launch batch jobs that are wrapped in Spring Cloud task and we also have configured Prometheus and Rsocket proxy to scrape task related metrics. To make the tasks ...
0
votes
0
answers
92
views
composed task is giving error " java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels"
I am using spring cloud dataflow server (2.8.1) , it is deployed on openshift (kubernetes deployer). we have set below properties to connect to oracle database.
spring.datasource.url:
spring....
0
votes
1
answer
141
views
Spring Cloud Data Flow: Docker URI error when running the sample app 'partitioned-batch-job' in Kubernetes cluster
I modified the dataflow sample app partitioned-batch-job to deploy it in a kubernetes cluster via the SCDF server that is running in the cluster. I use the dashboard to launch this app as a task. The ...
0
votes
0
answers
195
views
spring batch and spring cloud starter task : expected single matching bean but found 2: transactionManager, springCloudTransactionManager
I'm using spring cloud starter task and deployerPartitionHandler for spring batch remote partitioning job.
Now, When I'm trying to stop a running job using jobOperator.stop() method spring batch is ...
0
votes
1
answer
84
views
deploymentServiceAccountName does not work for schedules in spring cloud dataflow
I am using spring cloud dataflow for spring batches(with remote partitioning) wrapped with spring cloud task, deployed in kubernetes. I have also created a service account for the same. The batch runs ...
1
vote
1
answer
174
views
Spring batch worker pods are unable to pick custom service account for spring cloud deployer kubernetes
I am trying to run a spring batch with remote partitioning on K8s cluster using spring-cloud-deployer-kubernetes. Eventhough I have configured a service account and mentioned in my application ...
0
votes
1
answer
90
views
Spring batch worker pod logs are not visible in spring cloud dataflow
I am trying to run a spring batch with remote partitioning on K8s cluster. Every thing works fine regarding the partitioning, execution etc. The same is integrated with Spring Cloud Dataflow and can ...
1
vote
0
answers
417
views
Spring Batch pass data from writer to next job or step
I'm fairly new to Spring Batch and I would like to know if it is possible to pass data saved by writer of one job to next job or save in process and pass to next step.
I've found similar question here ...
2
votes
1
answer
541
views
Spring Batch with multi - step Spring Cloud Task (PartitionHandler) for Remote Partition
Latest Update (with an image to hope simplify the problem) (thanks for feedback from @Mahmoud)
Relate issue reports for other reference (after this original post created, it seem someone filed issues ...
0
votes
1
answer
112
views
Spring Cloud Dataflow - Parallel Tasks
I have about 16 tasks configured in parallel like <AAA && BBB|| CCC && DDD || EEE && FFF || GGG && ......>.
My intention is to only have 3 tasks running at one ...
1
vote
0
answers
167
views
How to launch a Spring Batch Remote Partitioning with Spring Cloud Dataflow
I am trying to launch a Spring Batch Remote Partitioning with Spring Cloud Dataflow. I have a remote partitioning that is based on https://github.com/mminella/LearningSpringBatch/tree/master/src/...
0
votes
1
answer
190
views
Spring Cloud Data Flow: Error org.springframework.dao.InvalidDataAccessResourceUsageException
I am trying to run/configure a Spring Data Cloud Data Flow (SCDF) to schedule a task for a Spring Batch Job.
I am running in a minikube that connects to a local postgresql(localhost:5432). The ...
0
votes
1
answer
223
views
Spring cloud data flow task Invalid TaskExecution ID
I am just trying to create a hello world SCDF task. It is my understanding that out of the box the task should be able to read the needed data from the h2 database by just including h2 in the pom, but ...
-1
votes
1
answer
641
views
Spring batch - Could not commit JDBC transaction; nested exception is java.sql.SQLTransactionRollbackException
I am using spring cloud data flow 2.7.1 with mysql 5.7.24 which executes a task with job related. the architecture used is Externalizing Batch Process Execution - remote partitioning (manger,worker) ...
0
votes
1
answer
110
views
Does Spring cloud task handle auto closing of the Kubernetes pods when the worker node has completed [duplicate]
I am writing the Spring batch app on AWS cloud and in order to ensure efficient utilization of resource , I had this question that "Does Spring cloud task handle auto closing of the Kubernetes ...
1
vote
0
answers
59
views
Spring Cloud Data Flow : Custom DDL for Data Flow Server
I have setup a Spring Cloud Data Flow server on Kubernetes backed by an in-memory h2 database running on Kubernetes as well.
I want to change the length of the columns in some of the tables that are ...
0
votes
1
answer
134
views
Spring Cloud Data Flow : Value too long for column "TASK_NAME" (DeployerPartitionHandler with Spring Batch)
I have a simple Spring Batch Job running Kubernetes as a Spring Cloud Task. This job uses Spring Batch Partitioning to further launch partitioned steps as task pods on the same Kubernetes cluster.
...
0
votes
1
answer
346
views
Spring Cloud Data Flow : Unable to launch multiple instances of the same Task
TL;DR
Spring Cloud Data Flow does not allow multiple executions of the same Task even though the documentation says that this is the default behavior. How can we allow SCDF to run multiple instances ...
0
votes
0
answers
118
views
SCDF + Spring Batch : JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={run.id=1}
SCDF is launching job as:
Command to be executed: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/java -jar /opt/scdf-batches/foo/foo-1.0.26.jar --spring.cloud.task.executionid=23
...
0
votes
1
answer
78
views
Non-option arguments with Spring Cloud Data Flow
I'm trying to run an application on Spring Cloud Data Flow that requires a non-option argument (e.g. the command would be java -jar task.jar non-option-argument --optionArgument=option etc.)
The ...
0
votes
1
answer
110
views
How to use static spring cloud stream url for launching spring cloud tasks?
Platform used : Kubernetes.
I have an issue with Spring cloud stream url. I am launching my spring cloud tasks using spring cloud stream. Streams are deployed in kubernetes platform. Stream contains ...
1
vote
0
answers
218
views
Spring Batch 'jobs must not be null nor empty' Error
I'm running Spring batch apps on Openshift using SCDF.
I'm getting error during Spring Batch starting-up like below.
What is the main reason of that?
2021-03-08 08:39:50.764 ERROR [ark-batch-template,,...
0
votes
0
answers
1k
views
Consider defining a bean of type 'org.springframework.transaction.jta.JtaTransactionManager' in your configuration
I am developing a spring cloud task application using spring cloud task and spring batch libraries. I also created a spring boot library containing spring cloud task and spring batch configurations ...
1
vote
1
answer
455
views
Task execution is not working after lunching the task in spring cloud data flow
I have created one Spring boot application with @EnablesTask annotation and try to print the arguments in log.
package com.custom.samplejob;
import org.springframework.boot.CommandLineRunner;
import ...
0
votes
1
answer
105
views
Automatic job restart
I have a job that can take up to several hours. It is possible that for some reason (like out of memory, or cluster rebalance) it just fails. The problem is that the job is usually run overnight, and ...
0
votes
1
answer
187
views
How can I design two different projects with a single entity or a single repository?
We are developing two projects with two different teams. However, there are some common points in the two projects, for example, there are common entities and repository usage cases. How can I manage ...
0
votes
0
answers
115
views
Step ExecutionContext not promoted using Spring Cloud Task on Spring Cloud Data Flow
I successfully deployed a remote partitioned job using Spring Cloud Data Flow and Spring Cloud Task; the installation is based on Kubernetes, so I added the Kubernetes implementation of Spring Cloud ...
-1
votes
1
answer
106
views
Spring Cloud Deployer Local is unable to spin up worker remote partitions when server.port property is set in master's application properties file
I am trying to build a batch service in an existing application that has server.port=8080 property configured in application.properties file. When I run the batch process and Spring Batch trying to ...
0
votes
1
answer
125
views
Forbidden error using paritioned job with Spring Cloud Data Flow on Kubernetes
I want to implement a remote partitioned job using Spring Cloud Data Flow on Kuberentes. The Skipper server is not installed because I just need to run tasks and jobs.
I modified the partitioned batch ...
1
vote
0
answers
86
views
How to define business custom properties for a task in SCDF?
I would like to develop a workflow in Spring Cloud Data Flow where each task instance would have its behavior dictate by the parameters I would have give to it at creation.
For example, if I have a ...
2
votes
1
answer
269
views
Violation of Foreign Key constraint in "task_metadata_fk" when launching a Task in SCDF Dashboard
I am trying to launch a task using the SCDF Dashboard.
I am using CockroachDB as the underlying persistence layer with Hibernate ORM version 5.4.22.Final which is supported by the 2.7.0 release of ...
0
votes
1
answer
451
views
Getting null pointer exception in launchWorker for spring batch running through spring cloud task using DeployerPartitionHandler
I'm using DeployerPartitionHandler (local variant) to partition my Spring batch job. When I run my job, I'm getting a null pointer exception in the launch step of the worker as below
at org....
1
vote
1
answer
719
views
Spring Cloud Data Flow Pod Cleanup
We are repeatedly seeing resource quota limitation issues in logs and Task jobs fail on the SCDF running on Kubernetes. Problem is, there are so many pods in "running" status even after they ...