9

I tried to run a simple program in hadoop using Windows-Cygwin.

I am able to start the namenode .

The jobtracker start however fails with exception :

FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: local
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:162)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:128)
        at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2560)
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2200)
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2192)
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2186)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:291)
        at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4978)

I tried all possible methods to resolve this ,but in vain. Any pointers will greatly help me.

Hdfs-site.xml configurations :

 <configuration><br>
   <property>
     <name>fs.default.name< /name>
     <value>hdfs://localhost:9100</value>
   </property>
   <property>
     <name>mapred.job.tracker< /name>
     <value>localhost:9101< /value>
   </property>
   <property>
     <name>dfs.replication< /name>
     <value>1</value>
   </property> 
 </configuration>

2 Answers 2

13

The problem is the following lines should on into mapred-site.xml and NOT hdfs-site.xml,

<property> 
<name>mapred.job.tracker</name> 
<value>localhost:9101</value> 
</property>

By the way why are you trying to run Hadoop in Windows? For development? You don't have a linux machine or reluctant to install one?

One more thing, you usually put this property in core-site.xml not hdfs-site.xml,

<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9100</value>
</property>
3
  • 8
    Why is it unreasonable to want to run Hadoop on Windows, other than in doing so one runs against the Hadoop community's continual blinkered view of the world that suggests there is only one acceptable operating system?
    – DeejUK
    Mar 19, 2013 at 16:45
  • Normally Hadoop people think Windows machine as a child's video game. Not any serious thing. I think when they get tired of work, they use windows machine's wall paper things.
    – Siva Tumma
    Dec 16, 2013 at 9:17
  • 6
    Because while it may be possible to run Hadoop on Windows, the code was developed entirely in the Linux environment and as such running it on Windows will introduce variables that are not expected and lead to further complications of something that is already very complicated. Hadoop does a lot of file operations, and while Java may run on Windows, not all native operations work quite the same, especially filesystem operations. That it runs on windows at all is something of a Miracle. The question isnt why would you want to run it on Windows, it is why would you want the misery of unknowns.
    – AaronM
    Mar 8, 2014 at 21:05
0

I faced the same issue when working on the "Pseudo Distributed" examples as at this page: http://hadoop.apache.org/docs/r1.1.2/single_node_setup.html#PseudoDistributed

It turned out that hadoop simply wasn't picking up my conf files. The examples at the link above assume you are running in your install of hadoop (i.e. /Usr/jane/hadoop-1.1.2). I was trying to run the examples in another directory. I'm sure you could configure hadoop to recognize other 'conf' directories, but I took the easy route and just started running in my hadoop directory.

This thread helped me figure it out: https://issues.apache.org/jira/browse/HDFS-2515

1
  • Btw, I found this page a helpful corollary to the hadoop setup page: javacodegeeks.com/2012/01/… This page shows pretty much the same examples but with additional info on expected output.
    – fahrradler
    Mar 12, 2013 at 0:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.