1

I have set up Datastax Enterprice in three nodes in the local network. Two nodes are debian servers and i used apt package manager for installation. The last node is iMac and i used the .dmg package for installation.

Node #1:
OS: Debian GNU/Linux 8.10 (jessie)
Local IP: 172.16.21.18
Datastax Enterprice: 5.1.7

Node #2:
OS: Ubuntu 16.04.3 LTS
Local IP: 172.16.21.25
Datastax Enterprice: 5.1.7

Node #1:
OS: macOS 10.13.2
Local IP: 192.168.1.108
Datastax Enterprice: 5.1.7

Nodes are up and running in analytics and search mode: ($ dse cassandra -k -s) spark cluster webui Now, I'm trying to connect on Spark Cluster using Apache Zeppelin 0.7.3. Apache Zeppelin is installed and configured in Node #1.

I followed these instructions for configuration. Below you can see some basic changes in config files:

zeppelin-0.7.3-bin-all/conf/zeppelin-env.sh

[..]
export MASTER=spark://172.16.21.18:7077                                 # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode.
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export DSE_HOME=/usr
[..]

zeppelin-0.7.3-bin-all/bin/interpreter.sh

[..]
# set spark related env variables
if [[ "${INTERPRETER_ID}" == "spark" ]]; then
  if [[ -n "${SPARK_HOME}" ]]; then
    export SPARK_SUBMIT="${DSE_HOME}/bin/dse spark-submit"
[..]

Zeppelin Spark Intepreter:

enter image description here

Zeppelin CQL intepreter works perfect with Apache Cassandra but then i'm trying to use Spark Intepreter to execute some queries i'm getting this error:

%spark
val results = spark.sql("SELECT * from keyspace.table")

java.lang.NullPointerException
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
    [..]

complete zeppelin log file:

 INFO [2018-02-21 04:25:36,185] ({Thread-0} RemoteInterpreterServer.java[run]:97) - Starting remote interpreter server on port 52127
 INFO [2018-02-21 04:25:36,562] ({pool-1-thread-3} RemoteInterpreterServer.java[createInterpreter]:198) - Instantiate interpreter org.apache.zeppelin.spark.SparkInterpreter
 INFO [2018-02-21 04:25:36,589] ({pool-1-thread-3} RemoteInterpreterServer.java[createInterpreter]:198) - Instantiate interpreter org.apache.zeppelin.spark.SparkSqlInterpreter
 INFO [2018-02-21 04:25:36,601] ({pool-1-thread-3} RemoteInterpreterServer.java[createInterpreter]:198) - Instantiate interpreter org.apache.zeppelin.spark.DepInterpreter
 INFO [2018-02-21 04:25:36,619] ({pool-1-thread-3} RemoteInterpreterServer.java[createInterpreter]:198) - Instantiate interpreter org.apache.zeppelin.spark.PySparkInterpreter
 INFO [2018-02-21 04:25:36,622] ({pool-1-thread-3} RemoteInterpreterServer.java[createInterpreter]:198) - Instantiate interpreter org.apache.zeppelin.spark.SparkRInterpreter
 INFO [2018-02-21 04:25:36,683] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1519205136682 started by scheduler org.apache.zeppelin.spark.SparkInterpreter269729544
 INFO [2018-02-21 04:25:40,733] ({pool-2-thread-2} SparkInterpreter.java[createSparkSession]:318) - ------ Create new SparkContext spark://172.16.21.18:7077 -------
 WARN [2018-02-21 04:25:40,740] ({pool-2-thread-2} SparkInterpreter.java[setupConfForSparkR]:577) - sparkr.zip is not found, sparkr may not work.
 INFO [2018-02-21 04:25:40,786] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Running Spark version 2.1.0
 WARN [2018-02-21 04:25:41,760] ({pool-2-thread-2} NativeCodeLoader.java[<clinit>]:62) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 WARN [2018-02-21 04:25:41,958] ({pool-2-thread-2} Logging.scala[logWarning]:66) - 
SPARK_CLASSPATH was detected (set to ':/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/dep/*:/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/*:/home/cassandra/zeppelin-0.7.3-bin-all/lib/interpreter/*:').
This is deprecated in Spark 1.0+.

Please instead use:
 - ./spark-submit with --driver-class-path to augment the driver classpath
 - spark.executor.extraClassPath to augment the executor classpath

 WARN [2018-02-21 04:25:41,959] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Setting 'spark.executor.extraClassPath' to ':/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/dep/*:/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/*:/home/cassandra/zeppelin-0.7.3-bin-all/lib/interpreter/*:' as a work-around.
 WARN [2018-02-21 04:25:41,960] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Setting 'spark.driver.extraClassPath' to ':/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/dep/*:/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/*:/home/cassandra/zeppelin-0.7.3-bin-all/lib/interpreter/*:' as a work-around.
 WARN [2018-02-21 04:25:41,986] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Your hostname, XPLAIN005 resolves to a loopback address: 127.0.1.1; using 172.16.21.18 instead (on interface eth0)
 WARN [2018-02-21 04:25:41,987] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Set SPARK_LOCAL_IP if you need to bind to another address
 INFO [2018-02-21 04:25:42,017] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Changing view acls to: cassandra
 INFO [2018-02-21 04:25:42,017] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Changing modify acls to: cassandra
 INFO [2018-02-21 04:25:42,018] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Changing view acls groups to: 
 INFO [2018-02-21 04:25:42,019] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Changing modify acls groups to: 
 INFO [2018-02-21 04:25:42,019] ({pool-2-thread-2} Logging.scala[logInfo]:54) - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(cassandra); groups with view permissions: Set(); users  with modify permissions: Set(cassandra); groups with modify permissions: Set()
 INFO [2018-02-21 04:25:42,417] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Successfully started service 'sparkDriver' on port 51240.
 INFO [2018-02-21 04:25:42,445] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Registering MapOutputTracker
 INFO [2018-02-21 04:25:42,476] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Registering BlockManagerMaster
 INFO [2018-02-21 04:25:42,481] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
 INFO [2018-02-21 04:25:42,482] ({pool-2-thread-2} Logging.scala[logInfo]:54) - BlockManagerMasterEndpoint up
 INFO [2018-02-21 04:25:42,507] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Created local directory at /tmp/blockmgr-797ea400-69f1-4228-a6da-fe424edce8d4
 INFO [2018-02-21 04:25:42,524] ({pool-2-thread-2} Logging.scala[logInfo]:54) - MemoryStore started with capacity 408.9 MB
 INFO [2018-02-21 04:25:42,591] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Registering OutputCommitCoordinator
 INFO [2018-02-21 04:25:42,700] ({pool-2-thread-2} Log.java[initialized]:186) - Logging initialized @6930ms
 INFO [2018-02-21 04:25:42,864] ({pool-2-thread-2} Server.java[doStart]:327) - jetty-9.2.z-SNAPSHOT
 INFO [2018-02-21 04:25:42,902] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@2cbd702d{/jobs,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,903] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@240b993c{/jobs/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,903] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@5b7d8292{/jobs/job,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,908] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@4c2353ff{/jobs/job/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,909] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@bd87e4e{/stages,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,910] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@73e2d470{/stages/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,917] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@44bca18c{/stages/stage,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,918] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@1256be4f{/stages/stage/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,919] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@5a349845{/stages/pool,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,919] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@3f108627{/stages/pool/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,926] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@1e01f088{/storage,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,927] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@390281c1{/storage/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,927] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@470ac014{/storage/rdd,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,927] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@7c90476c{/storage/rdd/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,928] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@6d847dc6{/environment,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,936] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@40a5e53e{/environment/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,937] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@513e975e{/executors,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,937] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@2f6b1132{/executors/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,938] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@61cf2354{/executors/threadDump,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,939] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@eacb646{/executors/threadDump/json,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,951] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@2b8d44aa{/static,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,953] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@5c982268{/,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,954] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@44556f2c{/api,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,955] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@2fa0ef66{/jobs/job/kill,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,955] ({pool-2-thread-2} ContextHandler.java[doStart]:744) - Started o.s.j.s.ServletContextHandler@6e49562c{/stages/stage/kill,null,AVAILABLE}
 INFO [2018-02-21 04:25:42,970] ({pool-2-thread-2} AbstractConnector.java[doStart]:266) - Started ServerConnector@53405611{HTTP/1.1}{0.0.0.0:4040}
 INFO [2018-02-21 04:25:42,971] ({pool-2-thread-2} Server.java[doStart]:379) - Started @7201ms
 INFO [2018-02-21 04:25:42,971] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Successfully started service 'SparkUI' on port 4040.
 INFO [2018-02-21 04:25:42,974] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Bound SparkUI to 0.0.0.0, and started at http://172.16.21.18:4040
 INFO [2018-02-21 04:25:43,214] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Added file file:/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/pyspark/pyspark.zip at spark://172.16.21.18:51240/files/pyspark.zip with timestamp 1519205143214
 INFO [2018-02-21 04:25:43,217] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Copying /home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/pyspark/pyspark.zip to /tmp/spark-2e9292e3-8c4d-445a-92f0-7d54188818db/userFiles-4e8301a5-91bc-4753-8436-6cced0bdc5c5/pyspark.zip
 INFO [2018-02-21 04:25:43,226] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Added file file:/home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/pyspark/py4j-0.10.4-src.zip at spark://172.16.21.18:51240/files/py4j-0.10.4-src.zip with timestamp 1519205143226
 INFO [2018-02-21 04:25:43,227] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Copying /home/cassandra/zeppelin-0.7.3-bin-all/interpreter/spark/pyspark/py4j-0.10.4-src.zip to /tmp/spark-2e9292e3-8c4d-445a-92f0-7d54188818db/userFiles-4e8301a5-91bc-4753-8436-6cced0bdc5c5/py4j-0.10.4-src.zip
 INFO [2018-02-21 04:25:43,279] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Created default pool default, schedulingMode: FIFO, minShare: 0, weight: 1
 INFO [2018-02-21 04:25:43,325] ({appclient-register-master-threadpool-0} Logging.scala[logInfo]:54) - Connecting to master spark://172.16.21.18:7077...
 INFO [2018-02-21 04:25:43,391] ({netty-rpc-connection-0} TransportClientFactory.java[createClient]:250) - Successfully created connection to /172.16.21.18:7077 after 33 ms (0 ms spent in bootstraps)
 INFO [2018-02-21 04:26:03,326] ({appclient-register-master-threadpool-0} Logging.scala[logInfo]:54) - Connecting to master spark://172.16.21.18:7077...
 INFO [2018-02-21 04:26:23,326] ({appclient-register-master-threadpool-0} Logging.scala[logInfo]:54) - Connecting to master spark://172.16.21.18:7077...
ERROR [2018-02-21 04:26:43,328] ({appclient-registration-retry-thread} Logging.scala[logError]:70) - Application has been killed. Reason: All masters are unresponsive! Giving up.
 WARN [2018-02-21 04:26:43,328] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Application ID is not initialized yet.
 INFO [2018-02-21 04:26:43,336] ({stop-spark-context} AbstractConnector.java[doStop]:306) - Stopped ServerConnector@53405611{HTTP/1.1}{0.0.0.0:4040}
 INFO [2018-02-21 04:26:43,339] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 40068.
 INFO [2018-02-21 04:26:43,498] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Server created on 172.16.21.18:40068
 INFO [2018-02-21 04:26:43,499] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6e49562c{/stages/stage/kill,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,500] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@2fa0ef66{/jobs/job/kill,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,501] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@44556f2c{/api,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,501] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@5c982268{/,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,505] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@2b8d44aa{/static,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,506] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@eacb646{/executors/threadDump/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,507] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@61cf2354{/executors/threadDump,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,508] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
 INFO [2018-02-21 04:26:43,508] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@2f6b1132{/executors/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,509] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@513e975e{/executors,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,510] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@40a5e53e{/environment/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,511] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6d847dc6{/environment,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,511] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@7c90476c{/storage/rdd/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,512] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@470ac014{/storage/rdd,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,513] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@390281c1{/storage/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,513] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@1e01f088{/storage,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,513] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@3f108627{/stages/pool/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,514] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Registering BlockManager BlockManagerId(driver, 172.16.21.18, 40068, None)
 INFO [2018-02-21 04:26:43,514] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@5a349845{/stages/pool,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,515] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@1256be4f{/stages/stage/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,515] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@44bca18c{/stages/stage,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,516] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@73e2d470{/stages/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,516] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@bd87e4e{/stages,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,517] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@4c2353ff{/jobs/job/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,517] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@5b7d8292{/jobs/job,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,518] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@240b993c{/jobs/json,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,518] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@2cbd702d{/jobs,null,UNAVAILABLE}
 INFO [2018-02-21 04:26:43,521] ({dispatcher-event-loop-0} Logging.scala[logInfo]:54) - Registering block manager 172.16.21.18:40068 with 408.9 MB RAM, BlockManagerId(driver, 172.16.21.18, 40068, None)
 INFO [2018-02-21 04:26:43,522] ({stop-spark-context} Logging.scala[logInfo]:54) - Stopped Spark web UI at http://172.16.21.18:4040
 INFO [2018-02-21 04:26:43,526] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Registered BlockManager BlockManagerId(driver, 172.16.21.18, 40068, None)
 INFO [2018-02-21 04:26:43,527] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Initialized BlockManager: BlockManagerId(driver, 172.16.21.18, 40068, None)
 INFO [2018-02-21 04:26:43,530] ({stop-spark-context} Logging.scala[logInfo]:54) - Shutting down all executors
 INFO [2018-02-21 04:26:43,546] ({dispatcher-event-loop-1} Logging.scala[logInfo]:54) - Asking each executor to shut down
 WARN [2018-02-21 04:26:43,561] ({dispatcher-event-loop-0} Logging.scala[logWarning]:66) - Drop UnregisterApplication(null) because has not yet connected to master
 INFO [2018-02-21 04:26:43,583] ({dispatcher-event-loop-2} Logging.scala[logInfo]:54) - MapOutputTrackerMasterEndpoint stopped!
 INFO [2018-02-21 04:26:43,596] ({stop-spark-context} Logging.scala[logInfo]:54) - MemoryStore cleared
 INFO [2018-02-21 04:26:43,597] ({stop-spark-context} Logging.scala[logInfo]:54) - BlockManager stopped
 INFO [2018-02-21 04:26:43,605] ({stop-spark-context} Logging.scala[logInfo]:54) - BlockManagerMaster stopped
 INFO [2018-02-21 04:26:43,608] ({dispatcher-event-loop-1} Logging.scala[logInfo]:54) - OutputCommitCoordinator stopped!
ERROR [2018-02-21 04:26:43,748] ({pool-2-thread-2} Logging.scala[logError]:91) - Error initializing SparkContext.
java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem
    at scala.Predef$.require(Predef.scala:224)
    at org.apache.spark.metrics.MetricsSystem.getServletHandlers(MetricsSystem.scala:91)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:524)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:378)
    at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:233)
    at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:841)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
 INFO [2018-02-21 04:26:43,751] ({pool-2-thread-2} Logging.scala[logInfo]:54) - SparkContext already stopped.
ERROR [2018-02-21 04:26:43,751] ({pool-2-thread-2} Utils.java[invokeMethod]:40) - 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:378)
    at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:233)
    at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:841)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem
    at scala.Predef$.require(Predef.scala:224)
    at org.apache.spark.metrics.MetricsSystem.getServletHandlers(MetricsSystem.scala:91)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:524)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
    ... 20 more
 INFO [2018-02-21 04:26:43,752] ({stop-spark-context} Logging.scala[logInfo]:54) - Successfully stopped SparkContext
 INFO [2018-02-21 04:26:43,752] ({pool-2-thread-2} SparkInterpreter.java[createSparkSession]:379) - Created Spark session
ERROR [2018-02-21 04:26:43,753] ({pool-2-thread-2} Job.java[run]:181) - Job failed
java.lang.NullPointerException
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
    at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:398)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:387)
    at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:146)
    at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:843)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
 INFO [2018-02-21 04:26:43,759] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1519205136682 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter269729544

What do you think?

UPDATE:

All nodes upgraded to Datastax Enterprice 5.1.7

1
  • I have exactly the same issue. Mar 29, 2018 at 0:46

2 Answers 2

1

With DSE 5.1 any reference to the Spark Master should look like this example:

export MASTER=dse://1.20.300.10

0

ERROR [2018-02-21 04:26:43,328] ({appclient-registration-retry-thread} Logging.scala[logError]:70) - Application has been killed. Reason: All masters are unresponsive! Giving up. WARN [2018-02-21 04:26:43,328] ({pool-2-thread-2} Logging.scala[logWarning]:66) - Application ID is not initialized yet.

It seems the app is killed. Could you check the logs in spark master ?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.