Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> RemoteException writing files


Copy link to this message
-
Re: RemoteException writing files
Hi
  This Could be due to the Following reason

1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have
any available DataNodes
 2) Namenode not able to start properly
 3) other wise some IP Issue .
    Note:- Pleaes  mention localhost instead of 127.0.0.1 (If it is in
local)

   Follow URL:
http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F

Thanks
 samir

On Sat, May 19, 2012 at 11:30 PM, samir das mohapatra <
[EMAIL PROTECTED]> wrote:

> Hi
>   This Could be due to the Following reason
>
> 1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have
> any available DataNodes
>  2) Namenode not able to start properly
>  3) other wise some IP Issue .
>     Note:- Pleaes  mention localhost instead of 127.0.0.1 (If it is in
> local)
>
>    Follow URL:
>
> http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F
>
>
> Thanks
>  samir
>
>
>
> On Sat, May 19, 2012 at 8:59 PM, Todd McFarland <[EMAIL PROTECTED]>wrote:
>
>> Hi folks,
>>
>> (Resending to this group, sent to common-dev before, pretty sure that's
>> for
>> Hadoop internal development - sorry for that..)
>>
>> I'm pretty stuck here.  I've been researching for hours and I haven't made
>> any forward progress on this one.
>>
>> I have a vmWare installation of Cloudera Hadoop 0.20.  The following
>> commands to create a directory and copy a file from the shared folder
>> *work
>> fine*, so I'm confident everything is setup correctly:
>>
>> [cloudera@localhost bin]$ hadoop fs -mkdir /user/cloudera/testdir
>> [cloudera@localhost bin]$ hadoop fs -put
>> /mnt/hgfs/shared_folder/file1.txt
>> /user/cloudera/testdir/file1.txt
>>
>> The file shows up fine in the HDFS doing it this way on the Linux VM.
>>
>> *However*, when I try doing the equivalent operation in Java everything
>> works great until I try to close() FSDataOutputStream.
>> I'm left with the new directory and a zero byte size file.  One suspicious
>> thing is that the user is "admin" instead of "cloudera" which I haven't
>> figured out why.  Here is the error:
>>
>> 12/05/19 09:45:46 INFO hdfs.DFSClient: Exception in
>> createBlockOutputStream
>> 127.0.0.1:50010 java.net.ConnectException: Connection refused: no further
>> information
>> 12/05/19 09:45:46 INFO hdfs.DFSClient: Abandoning block
>> blk_1931357292676354131_1068
>> 12/05/19 09:45:46 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
>> 12/05/19 09:45:46 WARN hdfs.DFSClient: DataStreamer Exception:
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>> /user/admin/testdir/file1.txt could only be replicated to 0 nodes, instead
>> of 1
>>    at
>>
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1533)
>>    at
>>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:667)
>>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>    at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> There are certainly lots of search references to "*could only be
>> replicated
>> to 0 nodes, instead of 1*" but chasing down those suggestions hasn't
>> helped.
>> I have run *jps* and* netstat* and that looks good.  All services are
>> running, all port seem to be good.  The *health check* looks good, plenty
>> of disk space, no failed nodes...
>>
>> Here is the java (it fails when it hits "fs.close()":
>>
>> import java.io.BufferedReader;
>> import java.io.FileInputStream;
>> import java.io.FileReader;
>> import java.io.IOException;
>> import org.apache.hadoop.conf.Configuration;
>> import org.apache.hadoop.fs.FSDataInputStream;
>> import org.apache.hadoop.fs.FSDataOutputStream;
>> import org.apache.hadoop.fs.FileSystem;
>> import org.apache.hadoop.fs.Path;
>>
>> public class TestFileTrans {