Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> HFTP based distcp failing.


Copy link to this message
-
Re: HFTP based distcp failing.
Hi Ravi,

With distcp you usually run it on the target cluster and specify the
source with the "hftp://" URI and the target with "hdfs://". hftp is a
read only, http based access to the data.

Lars

On Tue, Jan 4, 2011 at 11:10 PM, Ravi Phulari <[EMAIL PROTECTED]> wrote:
> Hello Hadoopers,
>  I need to distcp data across two clusters. For security reasons I can not
> use hdfs based distcp.
> HFTP based distcp is failing  with following Ioexception.
>
> Stack trace.
>
> Copy failed: java.io.IOException: Not supported
>    at org.apache.hadoop.hdfs.HftpFileSystem.delete(HftpFileSystem.java:360)
>    at org.apache.hadoop.tools.DistCp.fullyDelete(DistCp.java:939)
>    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:655)
>    at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>    at org.apache.hadoop.tools.DistCp.main(DistCp.java:884)
>
> I am using following command for distcp.
>
> Hadoop distcp  hftp://nn1.hadoop1:50070/data
> hftp://nn2.hadoop2:50070/user/hadoop/
> Hadoop distcp  /data/logs    hftp://nn2.hadoop2:50070/user/hadoop/
>
> Any idea why this distcp could be failing.
> I don’t see any logs in JT and NN.
>
> Any help will be greatly appreciated.
>
> -
> Thanks,
> Ravi
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB