0

I am running Pyspark locally and trying to read a parquet file and load into a data frame from notebook.

df = spark.read.parquet("metastore_db/tmp/userdata1.parquet")

I am getting this exception

An error occurred while calling o738.parquet.
: org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;

Does anyone know how to do it?

1 Answer 1

1

Assuming that you are running spark on your local, you should be doing something like

df = spark.read.parquet("file:///metastore_db/tmp/userdata1.parquet")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.