I am running a pyspark dataproc serverless batch. It has been running for too long so I decided to delete it. But neither the GCP console nor the CLI allow me to delete the batch.
The command I tried is
gcloud dataproc batches delete <Batch ID> --region=us-central1
I get the following error:
ERROR: (gcloud.dataproc.batches.delete) FAILED_PRECONDITION: Cannot delete non terminal batch 'Batch(<project-id/batch-id>)'; current state: 'RUNNING'
gcloud dataproc batches cancel