Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - HDFS data transfer is faster than SCP based transfer?


Copy link to this message
-
Re: HDFS data transfer is faster than SCP based transfer?
Harsh J 2014-01-25, 10:40
There's a lot of difference here, although both do use TCP underneath,
but do note that SCP securely encrypts data but stock HDFS
configuration does not.

You can also ask SCP to compress data transfer via the "-C" argument
btw - unsure if you already applied that pre-test - it may help show
up some difference. Also, the encryption algorithm can be changed to a
weaker one if security is not a concern during the transfer, via "-c
arcfour".

On Fri, Jan 24, 2014 at 10:55 AM, rab ra <[EMAIL PROTECTED]> wrote:

Harsh J