Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> HDFS question


Copy link to this message
-
Re: HDFS question
Do you have a jobtracker?  Without a jobtracker and tasktrackers, distcp is
running in LocalRunner mode.  I.E. it is running a single-threaded process
on the local machine.  The default behavior of the DFSClient is to write
data locally first, with replicas being placed off-rack then on-rack.

This would explain why everything seems to be going locally, it is also
probably much slower than it could be.
On Tue, Jan 28, 2014 at 11:42 AM, Ognen Duzlevski
<[EMAIL PROTECTED]>wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB