20

Is there a sql fucntion in spark sql which returns back current timestamp , example in impala NOW() is the function which returns back current timestamp is there similar in spark sql ?

Thanks

3 Answers 3

31

Try current_timestamp function.

current_timestamp() - Returns the current timestamp at the start of query evaluation. All calls of current_timestamp within the same query return the same value. 
1
0

It is possible to use Date and Timestamp functions from pyspark sql functions.

Example:

spark-sql> select current_timestamp();
2022-05-07 16:43:43.207
Time taken: 0.17 seconds, Fetched 1 row(s)
spark-sql> select current_date();
2022-05-07
Time taken: 5.224 seconds, Fetched 1 row(s)
spark-sql>

Reference: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.functions.current_timestamp.html

1
0

You can use the following code to get the date and timestamp in Spark with Scala code.

import org.apache.spark.sql.functions._
val newDf = df.withColumn("current_date",current_date())
.withColumn("current_timestamp",current_timestamp())

The result will be something like this.

+------------+-----------------------+
|current_date|current_timestamp      |
+------------+-----------------------+
|2022-06-06  |2022-06-06 12:25:55.349|
+------------+-----------------------+
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.