13

I know there have been many posts regarding this exception, but I am not able to fix this issue. Classpath has to be edited I think to resolve it. I am trying to run a program called DistMap in hadoop infrastructure. This is the error I am getting.

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.PlatformName.  Program will exit.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FsShell
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.fs.FsShell.  Program will exit.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.PlatformName.  Program will exit.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FsShell
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.fs.FsShell.  Program will exit.
Error could not create input directory /distmap_output_input folder on hdfs file system

which java says

/usr/java/jdk1.6.0_32/bin/java

echo $CLASSPATH gives a blank line

cat ~/.bash_profile says

cat ~/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

Update:

$HADOOP_HOME /usr/lib/hadoop

$HADOOP_CLASSPATH

/usr/lib/hadoop-0.20-mapreduce/hadoop-ant-2.0.0-mr1-cdh4.4.0.jar:/usr/lib/hadoop‌​-0.20-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-examples-2.0‌​.0-mr1-cdh4.4.0.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar:/usr/lib/h‌​adoop-0.20-mapreduce/hadoop-core.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2‌​.0.0-mr1-cdh4.4.0.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-test-2.0.0-mr1-cdh4.4‌​.0.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-test.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-tools-2.0.0-mr1-cdh4.4.0.jar:/usr/lib/hado‌​op-0.20-mapreduce/hadoop-tools.jar:/usr/lib/hadoop-0.20-mapreduce/lib/*jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar

The last 2 jar files have those classes PlatformName and FsShell. Still its not working.

Can anyone please help me fix this issue?

Thanks

10
  • Seems it can not locate hadoop-core.jar, just make sure the jar file is incluced in your CLASSPATH.
    – Aaron
    Sep 24, 2013 at 6:13
  • is that done like this? PATH=$PATH:$HOME/bin:/usr/lib/hadoop/hadoop-core.jar
    – Ashwin
    Sep 24, 2013 at 6:26
  • Do I have to change that PATH to $PATH:$usr/java/jdk1.6.0_32/bin?
    – Ashwin
    Sep 24, 2013 at 6:27
  • No. It should be CLASSPATH not PATH. Should be something like CLASSPATH=.:/usr/lib/hadoop/hadoop-core.jar. Or you can specify the CLASSPATH when you start your program.
    – Aaron
    Sep 24, 2013 at 6:29
  • 1
    IS there a hadoop-core jar in 2.2.0? I just download the last stable version from Apache and I cant find it. There is hadoop-mapreduce-client-core, but only hadoop-core I believe there is not. I'm also having the same exception and cant figure out why Dec 28, 2013 at 10:33

4 Answers 4

18

hadoop-auth-2.2.0.jar should be there for hadoop2.2.0
Add hadoop auth jar for your version

3

If you are a maven user and faced this issue -

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>${hadoop.client.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>${hadoop.client.version}</version>
    </dependency>

NOTE: :hadoop-client:2.5.2 alone didn't bring in all the required hadoop dependencies. That is why I added hadoop-common, which brought all the required deps.

0

Hadoop HADOOP_CLASSPATH issues

This would be helpful.

Thanks & Regards,
Alok Thaker

1
  • I didn't quite understand as there are many hadoop folders with many jars. I am not understanding which one to set.
    – Ashwin
    Sep 24, 2013 at 6:56
0

In my particular case, I am using Maven, and although I had hadoop-auth in both the POM and the classpath, I was still getting this exception.

What finally fixed it for me was changing the scope for the dependency in the POM from "provided" to "compile".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.