Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Fastest way to transfer files


Copy link to this message
-
Re: Fastest way to transfer files
Not sure why you are implying a contradiction when you say: "... distcp is useful _but_ you want to do 'it' in java..."

First of all distcp _is_ written in Java.
You can call distcp or any other MR job from Java just fine.

Cheers,

Joep

Sent from my iPhone

On Dec 28, 2012, at 12:01 PM, burakkk <[EMAIL PROTECTED]> wrote:

> Hi,
> I have two different hdfs cluster. I need to transfer files between these environments. What's the fastest way to transfer files for that situation?
>
> I've researched about it. I found distcp command. It's useful but I want to do in java so is there any way to do this?
>
> Is there any way to transfer files chunk by chunk from one hdfs cluster to another one or is there any way to implement a process using chunks without whole file?
>
> Thanks
> Best Regards...
>
> --
> BURAK ISIKLI | http://burakisikli.wordpress.com
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB