Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - RemoteException writing files


Copy link to this message
-
Re: RemoteException writing files
samir das mohapatra 2012-05-19, 18:00
Hi
  This Could be due to the Following reason

1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have
any available DataNodes
 2) Namenode not able to start properly
 3) other wise some IP Issue .
    Note:- Pleaes  mention localhost instead of 127.0.0.1 (If it is in
local)

   Follow URL:
http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F
Thanks
 samir
On Sat, May 19, 2012 at 8:59 PM, Todd McFarland <[EMAIL PROTECTED]>wrote:

> Hi folks,
>
> (Resending to this group, sent to common-dev before, pretty sure that's for
> Hadoop internal development - sorry for that..)
>
> I'm pretty stuck here.  I've been researching for hours and I haven't made
> any forward progress on this one.
>
> I have a vmWare installation of Cloudera Hadoop 0.20.  The following
> commands to create a directory and copy a file from the shared folder *work
> fine*, so I'm confident everything is setup correctly:
>
> [cloudera@localhost bin]$ hadoop fs -mkdir /user/cloudera/testdir
> [cloudera@localhost bin]$ hadoop fs -put /mnt/hgfs/shared_folder/file1.txt
> /user/cloudera/testdir/file1.txt
>
> The file shows up fine in the HDFS doing it this way on the Linux VM.
>
> *However*, when I try doing the equivalent operation in Java everything
> works great until I try to close() FSDataOutputStream.
> I'm left with the new directory and a zero byte size file.  One suspicious
> thing is that the user is "admin" instead of "cloudera" which I haven't
> figured out why.  Here is the error:
>
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> 127.0.0.1:50010 java.net.ConnectException: Connection refused: no further
> information
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Abandoning block
> blk_1931357292676354131_1068
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
> 12/05/19 09:45:46 WARN hdfs.DFSClient: DataStreamer Exception:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /user/admin/testdir/file1.txt could only be replicated to 0 nodes, instead
> of 1
>    at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1533)
>    at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:667)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> There are certainly lots of search references to "*could only be replicated
> to 0 nodes, instead of 1*" but chasing down those suggestions hasn't
> helped.
> I have run *jps* and* netstat* and that looks good.  All services are
> running, all port seem to be good.  The *health check* looks good, plenty
> of disk space, no failed nodes...
>
> Here is the java (it fails when it hits "fs.close()":
>
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.FileReader;
> import java.io.IOException;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.fs.FSDataInputStream;
> import org.apache.hadoop.fs.FSDataOutputStream;
> import org.apache.hadoop.fs.FileSystem;
> import org.apache.hadoop.fs.Path;
>
> public class TestFileTrans {
>
>    public static void main(String[] args) {
>
>        Configuration config = new Configuration();
>
>        config.addResource(new
> Path("c:/_bigdata/client_libs/core-site.xml"));
>        config.addResource(new
> Path("c:/_bigdata/client_libs/hdfs-site.xml"));
>
>        System.out.println("hadoop.tmp.dir: " +
> config.get("hadoop.tmp.dir"));
>        try{
>            FileSystem dfs = FileSystem.get(config);
>
>            // this will default to admin unless the workingDirectory is
> explicitly set..
>
>            System.out.println("HDFS Working Directory: " +
> dfs.getWorkingDirectory().toString());
>
>            String dirName = "testdir";