Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> RemoteException writing files


Copy link to this message
-
Re: RemoteException writing files
Hi
  This Could be due to the Following reason

1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have
any available DataNodes
 2) Namenode not able to start properly
 3) other wise some IP Issue .
    Note:- Pleaes  mention localhost instead of 127.0.0.1 (If it is in
local)

   Follow URL:
http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F
Thanks
 samir
On Sat, May 19, 2012 at 8:59 PM, Todd McFarland <[EMAIL PROTECTED]>wrote:

> Hi folks,
>
> (Resending to this group, sent to common-dev before, pretty sure that's for
> Hadoop internal development - sorry for that..)
>
> I'm pretty stuck here.  I've been researching for hours and I haven't made
> any forward progress on this one.
>
> I have a vmWare installation of Cloudera Hadoop 0.20.  The following
> commands to create a directory and copy a file from the shared folder *work
> fine*, so I'm confident everything is setup correctly:
>
> [cloudera@localhost bin]$ hadoop fs -mkdir /user/cloudera/testdir
> [cloudera@localhost bin]$ hadoop fs -put /mnt/hgfs/shared_folder/file1.txt
> /user/cloudera/testdir/file1.txt
>
> The file shows up fine in the HDFS doing it this way on the Linux VM.
>
> *However*, when I try doing the equivalent operation in Java everything
> works great until I try to close() FSDataOutputStream.
> I'm left with the new directory and a zero byte size file.  One suspicious
> thing is that the user is "admin" instead of "cloudera" which I haven't
> figured out why.  Here is the error:
>
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> 127.0.0.1:50010 java.net.ConnectException: Connection refused: no further
> information
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Abandoning block
> blk_1931357292676354131_1068
> 12/05/19 09:45:46 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
> 12/05/19 09:45:46 WARN hdfs.DFSClient: DataStreamer Exception:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /user/admin/testdir/file1.txt could only be replicated to 0 nodes, instead
> of 1
>    at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1533)
>    at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:667)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> There are certainly lots of search references to "*could only be replicated
> to 0 nodes, instead of 1*" but chasing down those suggestions hasn't
> helped.
> I have run *jps* and* netstat* and that looks good.  All services are
> running, all port seem to be good.  The *health check* looks good, plenty
> of disk space, no failed nodes...
>
> Here is the java (it fails when it hits "fs.close()":
>
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.FileReader;
> import java.io.IOException;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.fs.FSDataInputStream;
> import org.apache.hadoop.fs.FSDataOutputStream;
> import org.apache.hadoop.fs.FileSystem;
> import org.apache.hadoop.fs.Path;
>
> public class TestFileTrans {
>
>    public static void main(String[] args) {
>
>        Configuration config = new Configuration();
>
>        config.addResource(new
> Path("c:/_bigdata/client_libs/core-site.xml"));
>        config.addResource(new
> Path("c:/_bigdata/client_libs/hdfs-site.xml"));
>
>        System.out.println("hadoop.tmp.dir: " +
> config.get("hadoop.tmp.dir"));
>        try{
>            FileSystem dfs = FileSystem.get(config);
>
>            // this will default to admin unless the workingDirectory is
> explicitly set..
>
>            System.out.println("HDFS Working Directory: " +
> dfs.getWorkingDirectory().toString());
>
>            String dirName = "testdir";
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB