Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Fastest way to transfer files


Copy link to this message
-
Re: Fastest way to transfer files
Not sure why you are implying a contradiction when you say: "... distcp is useful _but_ you want to do 'it' in java..."

First of all distcp _is_ written in Java.
You can call distcp or any other MR job from Java just fine.

Cheers,

Joep

Sent from my iPhone

On Dec 28, 2012, at 12:01 PM, burakkk <[EMAIL PROTECTED]> wrote:

> Hi,
> I have two different hdfs cluster. I need to transfer files between these environments. What's the fastest way to transfer files for that situation?
>
> I've researched about it. I found distcp command. It's useful but I want to do in java so is there any way to do this?
>
> Is there any way to transfer files chunk by chunk from one hdfs cluster to another one or is there any way to implement a process using chunks without whole file?
>
> Thanks
> Best Regards...
>
> --
> BURAK ISIKLI | http://burakisikli.wordpress.com
>