1

I'm currently testing one application using sparkR. This are the my platform & application details:

Platform: Windows server 2008 SparkR version : R version 3.1.2 (2014-10-31) Spark Verion : version 1.4.1

What I did?

Step I: Load package into R environment

library(SparkR) -- Working

Step II: Set the system environment variables

Sys.setenv(SPARK_HOME = "C:\hdp\spark-1.4.1-bin-hadoop2.6") -- Working .libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"),.libPaths()))

Step III: Create a spark context and a SQL context

sc <- sparkR.init(master = "local",sparkHome = "C:\hdp\spark-1.4.1-bin-hadoop2.6",appName = "TestSparR")

Getting error at this line i.e JVM is not ready after 10 seconds

Please, help me resolve this issue. Thanks.

1

2 Answers 2

2

I had the same problem, and I can tell you I tried many many things.

But finally the following worked for me, after restarting my computer (and R and RStudio by the way):

SPARK_HOME <- "C:\\Apache\\spark-1.5.2-bin-hadoop2.6\\"
Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.databricks:spark-csv_2.10:1.2.0" "sparkr-shell"')
library(SparkR, lib.loc = "C:\\Apache\\spark-1.5.2-bin-hadoop2.6\\R\\lib")

library(SparkR)
library(rJava)

sc <- sparkR.init(master = "local", sparkHome = SPARK_HOME)

Maybe this can help: after restarting the system this was included in my environment variables PATH:

C:\ProgramData\Oracle\Java\javapath

0

This worked for me

sparkPath <- 'C:/Users/YOUR PATH'
Sys.setenv(SPARK_HOME=sparkPath)
.libPaths(c(file.path(Sys.getenv('SPARK_HOME'), 'R', 'lib'), .libPaths()))
library(SparkR)
library(sparklyr)
sc <- spark_connect(master='local')

1
  • If this doesn't work, it may be because you have a space in your path. example C:/Users/Rick Ross/etc., when you have a space in your path is sometimes glitchy. Try installing Docker!
    – rad15f
    Jan 25, 2022 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.