1

As a start, I've installed Hadoop (0.15.2) and setup a cluster of 3 nodes: one each for NameNode, DataNode and the JobTracker. All the daemons are up and running. But when I issue any command I get the above error. For instance, when I do a copyFromLocal, I get the following error:

Am I missing something?

More details: I am trying to install Hadoop on an NFS file system. I've installed 1.0.4 version and tried running it but to of no avail. The 1.0.4 version doesn't start the datanode. And the log files for the datanode are empty. Hence I switched back to 0.15 version which started all the daemons atleast.

I believe the problem is due to the underlying NFS file system i.e. all the datanodes and masters using the same files and folders. But I am not sure if that is actually the case. But I don't see any reason why I shouldn't be able to run Hadoop on NFS (after appropriately setting the configuration parameters).

Currently I am trying and figuring out if I could set the name and data directories differently for different machines based on the individual machine names.

Configuration file: (hadoop-site.xml)

 <property>
 <name>fs.default.name</name>
 <value>mumble-12.cs.wisc.edu:9001</value>
 </property>
 <property>
 <name>mapred.job.tracker</name>
 <value>mumble-13.cs.wisc.edu:9001</value>
 </property>
 <property>
 <name>dfs.replication</name>
 <value>1</value>
 </property>
 <property>
 <name>dfs.secondary.info.port</name>
 <value>9002</value>
 </property>
 <property>
 <name>dfs.info.port</name>
 <value>9003</value>
 </property>
 <property>
 <name>mapred.job.tracker.info.port</name>
 <value>9004</value>
 </property>
 <property>
 <name>tasktracker.http.port</name>
 <value>9005</value>
 </property>

Error using Hadoop 1.0.4 (DataNode doesn't get started):

2013-04-22 18:50:50,438 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on     9001, call addBlock(/tmp/hadoop-akshar/mapred/system/jobtracker.info, DFSClient_502734479, null) from 128.105.112.13:37204: error: java.io.IOException: File /tmp/hadoop-akshar/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /tmp/hadoop-akshar/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

Error using Hadoop 0.15.2:

[akshar@mumble-12] (38)$ bin/hadoop fs -copyFromLocal lib/junit-3.8.1.LICENSE.txt input

13/04/17 03:22:11 WARN fs.DFSClient: Error while writing.
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:189)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at java.net.SocketInputStream.read(SocketInputStream.java:203)
    at java.io.DataInputStream.readShort(DataInputStream.java:312)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1660)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1733)
    at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
    at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:55)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:83)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:140)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:826)
    at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:120)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:1360)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.fs.FsShell.main(FsShell.java:1478)
13/04/17 03:22:12 WARN fs.DFSClient: Error while writing.
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:189)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at java.net.SocketInputStream.read(SocketInputStream.java:203)
    at java.io.DataInputStream.readShort(DataInputStream.java:312)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1660)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1733)
    at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
    at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:55)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:83)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:140)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:826)
    at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:120)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:1360)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.fs.FsShell.main(FsShell.java:1478)
13/04/17 03:22:12 WARN fs.DFSClient: Error while writing.
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:189)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at java.net.SocketInputStream.read(SocketInputStream.java:203)
    at java.io.DataInputStream.readShort(DataInputStream.java:312)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1660)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1733)
    at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
    at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:55)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:83)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:140)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:826)
    at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:120)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:1360)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.fs.FsShell.main(FsShell.java:1478)
copyFromLocal: Connection reset
3
  • Welcome to StackOverflow! You may need to provide more details on what you're trying to do in order to get useful feedback.
    – ASGM
    Apr 17, 2013 at 9:12
  • If you are starting fresh with Hadoop, I highly recommend you use a more recent version than 0.15.2. 1.0.4 is the current stable release.
    – Matt D
    Apr 17, 2013 at 13:40
  • @MattD I've addressed your suggestion in the edited question above.
    – Rocky Inde
    Apr 22, 2013 at 5:31

1 Answer 1

0

I was able to get Hadoop to run over NFS using version 1.1.2. It might work for other versions, but I can't guarantee anything.

If you have an NFS file system then each node should have access to the filesystem. The fs.default.name tells Hadoop the filesystem URI to use, so it should be pointed to the local disk. I'll assume that your NFS directory is mounted to each node at /nfs.

In core-site.xml you should define:

<property>
  <name>fs.default.name</name>
  <value>file:///</value>
</property>

<property>
  <name>hadoop.tmp.dir</name>
  <value>/nfs/tmp</value>
</property>

In mapred-site.xml you should define:

<property>
  <name>mapred.job.tracker</name>
  <value>node1:8021</value>
</property>

<property>
  <name>mapred.local.dir</name>
  <value>/tmp/mapred-local</value>
</property>

Since hadoop.tmp.dir is pointed to the nfs drive then the default locations of mapred.system.dir and mapreduce.jobtracker.staging.root.dir point to locations on the nfs drive. It might run with leaving the default value for mapred.local.dir, but it is supposed to point to the local filesystem so to be safe you can put that in /tmp.

You don't have to worry about hdfs-site.xml. This configuration file is used when you start the namenode, but with everything being distributed on the nfs drive you shouldn't run HDFS.

Now you can run start-mapred.sh on the jobtracker node and run a hadoop job. Don't run start-all.sh or start-dfs.sh because those will start HDFS. If you run multiple DataNodes that point to the same NFS directory, then one DataNode will lock that directory and the others will shutdown because they are unable to obtain a lock.

I tested the configuration with:

bin/hadoop jar hadoop-examples-1.1.2.jar wordcount /nfs/data/test.text /nfs/out

Note that you need to specify full paths to the input and output locations.

I also tried:

bin/hadoop jar hadoop-examples-1.1.2.jar grep /nfs/data/loremIpsum.txt /nfs/out2 lorem

It gave me the same output as when I run it in Standalone, so I assume it is performing correctly.

Here is more information on fs.default.name: http://www.greenplum.com/blog/dive-in/usage-and-quirks-of-fs-default-name-in-hadoop-filesystem

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.