In SQL there is:
REFRESH TABLE table_name
and in pyspark the equivalent is:
spark.refreshTable(table_name)
So, what would be the Pyspark's equivalent to:
MSCK REPAIR TABLE table_name
refreshTable is integrated with spark session catalog.
as steven suggested, you can go with spark.sql('MSCK REPAIR TABLE table_name')
There is something called recoverPartitions
(Only works with a partitioned table, and not a view) in the above link. I believe this is aliased version of msck repair table
. you can go ahead and try this..
see this ddl.scala seems like its equalent by documentation.
example usage :
spark.catalog.recoverPartitions(tableName)
spark.sql('MSCK REPAIR TABLE table_name')