In the case of Scala, we can use SparkContext.applicationId
but I was not able to find any way to do the same using Spark's java API.
Using SparkConf.getAppId()
results in NoSuchElementException: spark.app.id
Is there any other unique Id that can be used?
My use case: I want the Spark job to not fail in case output from the previous job already exist and want to avoid deleting the output of previous job and avoid using random UUID